Overview

Host cleanup removes irrelevant data and reduces cache size automatically as part of the basic werf command invocation and for all projects at once. If necessary, the cleanup can be performed manually using the werf host cleanup command.

Notification about a recent automatic cleanup

Automatic host cleanup runs in background, so its output is not visible. If the last background cleanup freed any space, the next suitable werf command reports it: how long ago the cleanup ran, how many local images werf explicitly removed and how much space was freed in total. Commands with machine-readable output preserve the report for a later command that can display it. The report is shown only once per cleanup run.

If the cleanup was unable to bring the storage volume usage down to the allowed level, the report is shown as a warning instead: recently built images may keep being removed on subsequent runs until disk space is freed on the host or the allowed level is raised.

Changing build backend (Docker or Buildah) storage directory

The --backend-storage-path parameter (or the WERF_BACKEND_STORAGE_PATH environment variable) allows you to explicitly specify the backend storage directory in case werf fails to detect it automatically.

Changing the space usage threshold and cleanup depth of build backend (Docker or Buildah) storage

The --allowed-backend-storage-volume-usage (WERF_ALLOWED_BACKEND_STORAGE_VOLUME_USAGE) parameter allows you to adjust the volume usage threshold (the default is 70%). Reaching it will trigger a backend storage cleanup. You can specify the value as a percentage (e.g., 70) or in absolute units (e.g., 50GB, 100GiB).

The --allowed-backend-storage-volume-usage-margin (WERF_ALLOWED_BACKEND_STORAGE_VOLUME_USAGE_MARGIN) parameter allows you to set the extra cleanup margin relative to the backend storage usage threshold (the default is 5%).

Note: Options within the same group (e.g., usage and margin for backend storage) must use the same units. Mixing percentages and absolute units (e.g., --allowed-backend-storage-volume-usage=100GB --allowed-backend-storage-volume-usage-margin=5) is not allowed.

Cleaning a build backend cache (Docker or Buildah)

When cleaning up the host, werf removes containers, images, and unused volumes. However, the strategy for cleaning the build cache of the build backend is left to the user.

For example, when using the Docker backend, the following approaches can be used:

  • Set a cache size limit in the builder configuration.
  • Configure periodic cleanup using cron and the docker buildx prune --max-used-space=bytes command.

Similarly, users can set up a custom cleanup strategy for Buildah.

Changing the space usage threshold and cleanup depth of the local cache

The --allowed-local-cache-volume-usage (WERF_ALLOWED_LOCAL_CACHE_VOLUME_USAGE) parameter allows you to adjust the threshold of space used on the volume at which the local cache cleanup is triggered (the default is 70%). You can specify the value as a percentage (e.g., 70) or in absolute units (e.g., 10GB, 500MiB).

The --allowed-local-cache-volume-usage-margin (WERF_ALLOWED_LOCAL_CACHE_VOLUME_USAGE_MARGIN) parameter allows to set the cleanup margin relative to the local cache usage threshold (the default is 5%).

Note: Options within the same group (e.g., usage and margin for local cache) must use the same units. Mixing percentages and absolute units (e.g., --allowed-local-cache-volume-usage=10GB --allowed-local-cache-volume-usage-margin=5) is not allowed.

The local git data cache is cleaned up by removing the data that is cheapest to rebuild first: git archives and patches, then worktrees without submodules, then worktrees with submodules, and git repository mirrors last. Within each group the least recently used data goes first. Entries with valid access metadata are eligible for this cleanup only after 3 hours without use; invalid entries and stale cache format versions are handled separately.

The threshold applies to space used on the filesystem containing the cache, not just to the size of Git data. Reported freed bytes are an estimate: hard links, compression and filesystem snapshots can make the actual change differ. Cleanup stops early if the measured usage reaches the target, but does not expand its original estimated removal budget to chase delayed space reclamation.

Turning off automatic cleaning

The user can disable automatic cleanup of outdated host data using the --disable-auto-host-cleanup parameter (WERF_DISABLE_AUTO_HOST_CLEANUP). In this case, we recommend adding the werf host cleanup command to the list of cron jobs, e.g., as follows:

# /etc/cron.d/werf-host-cleanup
SHELL=/bin/bash
*/30 * * * * gitlab-runner source ~/.profile ; source $(trdl use werf 3 stable) ; werf host cleanup