/cache volume is added automatically on gitlab-runner installation and managed by cache_dir setting
more about cache_dir:https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-sectiongitlab.comgitlab-org/gitlab-runner/blob/master/docs/executors/docker.md#the-builds-and-cache-storage# The Docker executorGitLab Runner can use Docker to run jobs on user provided images. This ispossible with the use of **Docker** executor.The **Docker** executor when used with GitLab CI, connects to [Docker Engine]and runs each build in a separate and isolated container using the predefinedimage that is [set up in `.gitlab-ci.yml`][yaml] and in accordance in[`config.toml`][toml].That way you can have a simple and reproducible build environment that can alsorun on your workstation. The added benefit is that you can test all thecommands that we will explore later from your shell, rather than having to testthem on a dedicated CI server.The following table lists what combinations of containers, executors,and OS are supported.| Container Type| Executor | OS Type | Supported ||:-----------------:|:----------------:|:-------:|:---------:| This file has been truncated. show original
If you modify the /cache storage path, you also need to make sure to mark thisdirectory as persistent by defining it in volumes = ["/my/cache/"] under the[runners.docker] section in config.toml.TLDR/cache volume is different from cache config in gitlab-ci.yml
/cache dir in job container is where the cached files are storedfiles matching to cache config in gitlab-ci.yml are copied to /cache/CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/- at the end of jobThe “Clear Runner Caches” button schedules NOT TO extract /cache/CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/-/cache.zip to dir specified in cache config in gitlab-ci.ymlP.S.
There is container named gitlab-runner-cache (https://gitlab.com/gitlab-org/gitlab-runner/blob/af343971874198a1923352107409583b78e8aa80/executors/docker/executor_docker.go#L382)Seems like this container is used to create anonymous volume where /cache data is stored and then stoppedSeems like the job containers mount this anonymous volume using --volumes-from
I may be wrong
ProofsHAVING gitlab-ci.yml
image: srghma/docker-nixos-with-git-cryptcache: key: "test00000" # to reset cache - change this key OR clear cache in project settings page paths:- .mycache # gitlab allows only cache dirs that are relative to project root OR /cache (created automatically)testtest: script:- nix-env -i tree- tree --dirsfirst -L 4 /cache- ls -al ./.mycache || true- echo "test" > /cache/test- mkdir -p ./.mycache- echo "test" > ./.mycache/test- tree --dirsfirst -L 4 /cache- ls -al ./.mycache || trueOutput:
on first runRunning with gitlab-runner 11.6.0 (f100a208) on srghma_gitlab_runner 9b3980daUsing Docker executor with image srghma/docker-nixos-with-git-crypt ...Pulling docker image srghma/docker-nixos-with-git-crypt ...Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...Running on runner-9b3980da-project-222-concurrent-0 via stage.getgreenbadger.com...Fetching changes...Removing .mycache/HEAD is now at 675caa7 feat: cache updateFrom https://gitlab.nordicresults.com/srghma/greenbadger675caa7..3d1e223 nix-> origin/nixChecking out 3d1e2237 as nix...Skipping Git submodules setupChecking cache for test00000-11...No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.Successfully extracted cache$ nix-env -i treeinstalling 'tree-1.8.0'these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked): /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...created 233 symlinks in user environment$ tree --dirsfirst -L 4 /cache/cache0 directories, 0 files$ ls -al ./.mycache || true$ echo "test" > /cache/testls: ./.mycache: No such file or directory$ mkdir -p ./.mycache$ echo "test" > ./.mycache/test$ tree --dirsfirst -L 4 /cache/cache`-- test0 directories, 1 file$ ls -al ./.mycache || truetotal 12drwxr-xr-x2 root root 4096 Feb 24 11:44 .drwxrwxrwx20 root root 4096 Feb 24 11:44 ..-rw-r--r--1 root root 5 Feb 24 11:44 testCreating cache test00000-11....mycache: found 2 matching filesNo URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.Created cacheJob succeededon second runRunning with gitlab-runner 11.6.0 (f100a208) on srghma_gitlab_runner 9b3980daUsing Docker executor with image srghma/docker-nixos-with-git-crypt ...Pulling docker image srghma/docker-nixos-with-git-crypt ...Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...Running on runner-9b3980da-project-222-concurrent-0 via stage.getgreenbadger.com...Fetching changes...Removing .mycache/HEAD is now at 3d1e223 feat: cache updateChecking out 3d1e2237 as nix...Skipping Git submodules setupChecking cache for test00000-11...No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.Successfully extracted cache$ nix-env -i treeinstalling 'tree-1.8.0'these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked): /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...created 233 symlinks in user environment$ tree --dirsfirst -L 4 /cache/cache|-- srghma|`-- greenbadger|`-- test00000-11|`-- cache.zip`-- test3 directories, 2 files$ ls -al ./.mycache || truetotal 12drwxr-xr-x2 root root 4096 Feb 24 11:44 .drwxrwxrwx20 root root 4096 Feb 24 11:44 ..-rw-r--r--1 root root 5 Feb 24 11:44 test$ echo "test" > /cache/test$ mkdir -p ./.mycache$ echo "test" > ./.mycache/test$ tree --dirsfirst -L 4 /cache/cache|-- srghma|`-- greenbadger|`-- test00000-11|`-- cache.zip`-- test3 directories, 2 files$ ls -al ./.mycache || truetotal 12drwxr-xr-x2 root root 4096 Feb 24 11:44 .drwxrwxrwx20 root root 4096 Feb 24 11:44 ..-rw-r--r--1 root root 5 Feb 24 11:44 testCreating cache test00000-11....mycache: found 2 matching filesNo URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.Created cacheJob succeededafter you clear cache by clicking on “Clear Runner Caches” in your project “Pipelines” pageRunning with gitlab-runner 11.6.0 (f100a208) on srghma_gitlab_runner 9b3980daUsing Docker executor with image srghma/docker-nixos-with-git-crypt ...Pulling docker image srghma/docker-nixos-with-git-crypt ...Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...Running on runner-9b3980da-project-222-concurrent-0 via stage.getgreenbadger.com...Fetching changes...Removing .mycache/HEAD is now at 3d1e223 feat: cache updateChecking out 3d1e2237 as nix...Skipping Git submodules setupChecking cache for test00000-12...No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.Successfully extracted cache$ nix-env -i treeinstalling 'tree-1.8.0'these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked): /nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...created 233 symlinks in user environment$ tree --dirsfirst -L 4 /cache/cache|-- srghma|`-- greenbadger|`-- test00000-11|`-- cache.zip`-- test3 directories, 2 files$ ls -al ./.mycache || truels: ./.mycache: No such file or directory$ echo "test" > /cache/test$ mkdir -p ./.mycache$ echo "test" > ./.mycache/test$ tree --dirsfirst -L 4 /cache/cache|-- srghma|`-- greenbadger|`-- test00000-11|`-- cache.zip`-- test3 directories, 2 files$ ls -al ./.mycache || truetotal 12drwxr-xr-x2 root root 4096 Feb 24 11:45 .drwxrwxrwx20 root root 4096 Feb 24 11:45 ..-rw-r--r--1 root root 5 Feb 24 11:45 testCreating cache test00000-12....mycache: found 2 matching filesNo URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.Created cacheJob succeeded