Overview

Host cleanup removes irrelevant data and reduces cache size automatically as part of the basic werf command invocation and for all projects at once. If necessary, the cleanup can be performed manually using the werf host cleanup command.

Notification about a recent automatic cleanup

Automatic host cleanup runs in background, so its output is not visible. If the last background cleanup freed any space, the next suitable werf command reports it: how long ago the cleanup ran, how many local images werf explicitly removed and how much space was freed in total. Commands with machine-readable output preserve the report for a later command that can display it. The report is shown only once per cleanup run.

If the cleanup was unable to bring the storage volume usage down to the allowed level, the report is shown as a warning instead: recently built images may keep being removed on subsequent runs until disk space is freed on the host or the allowed level is raised.

Changing build backend (Docker or Buildah) storage directory

The --backend-storage-path parameter (or the WERF_BACKEND_STORAGE_PATH environment variable) allows you to explicitly specify the backend storage directory in case werf fails to detect it automatically.

Changing the space usage threshold and cleanup depth of build backend (Docker or Buildah) storage

The --allowed-backend-storage-volume-usage (WERF_ALLOWED_BACKEND_STORAGE_VOLUME_USAGE) parameter allows you to adjust the volume usage threshold (the default is 70%). Reaching it will trigger a backend storage cleanup. You can specify the value as a percentage (e.g., 70) or in absolute units (e.g., 50GB, 100GiB).

The --allowed-backend-storage-volume-usage-margin (WERF_ALLOWED_BACKEND_STORAGE_VOLUME_USAGE_MARGIN) parameter allows you to set the extra cleanup margin relative to the backend storage usage threshold (the default is 5%).

Note: Options within the same group (e.g., usage and margin for backend storage) must use the same units. Mixing percentages and absolute units (e.g., --allowed-backend-storage-volume-usage=100GB --allowed-backend-storage-volume-usage-margin=5) is not allowed.

Cleaning a build backend cache (Docker or Buildah)

When cleaning up the host, werf removes containers, images, and unused volumes. However, the strategy for cleaning the build cache of the build backend is left to the user.

For example, when using the Docker backend, the following approaches can be used:

  • Set a cache size limit in the builder configuration.
  • Configure periodic cleanup using cron and the docker buildx prune --max-used-space=bytes command.

Similarly, users can set up a custom cleanup strategy for Buildah.

Changing the space usage threshold and cleanup depth of the local cache

The --allowed-local-cache-volume-usage (WERF_ALLOWED_LOCAL_CACHE_VOLUME_USAGE) parameter allows you to adjust the threshold of space used on the volume at which the local cache cleanup is triggered (the default is 70%). You can specify the value as a percentage (e.g., 70) or in absolute units (e.g., 10GB, 500MiB).

The --allowed-local-cache-volume-usage-margin (WERF_ALLOWED_LOCAL_CACHE_VOLUME_USAGE_MARGIN) parameter allows to set the cleanup margin relative to the local cache usage threshold (the default is 5%).

Note: Options within the same group (e.g., usage and margin for local cache) must use the same units. Mixing percentages and absolute units (e.g., --allowed-local-cache-volume-usage=10GB --allowed-local-cache-volume-usage-margin=5) is not allowed.

Turning off automatic cleaning

The user can disable automatic cleanup of outdated host data using the --disable-auto-host-cleanup parameter (WERF_DISABLE_AUTO_HOST_CLEANUP). In this case, we recommend adding the werf host cleanup command to the list of cron jobs, e.g., as follows:

# /etc/cron.d/werf-host-cleanup
SHELL=/bin/bash
*/30 * * * * gitlab-runner source ~/.profile ; source $(trdl use werf 2 stable) ; werf host cleanup