Build process

Container registry authentication

You must log in to the container registry to work with the images. Use the werf cr login command as follows:

werf cr login <registry url>

For example:

# Log in with a username and password from the command line
werf cr login -u username -p password registry.example.com

# Log in with a token from the command line
werf cr login -u username -p token registry.example.com

# Log in to an insecure registry (over HTTP)
werf cr login --insecure-registry registry.example.com

Note: In supported CI/CD systems, the user gets authenticated to the integrated container registries as part of the ci-env command — you do not have to use the werf cr login command in this case.

Tagging images

The tagging of werf images is performed automatically as part of the build process. werf uses an optimal tagging scheme based on the contents of the image, thus preventing unnecessary rebuilds and application wait times during deployment.

registry.example.org/group/project  d4bf3e71015d1e757a8481536eeabda98f51f1891d68b539cc50753a-1589714365467  7c834f0ff026  20 hours ago  66.7MB
registry.example.org/group/project  e6073b8f03231e122fa3b7d3294ff69a5060c332c4395e7d0b3231e3-1589714362300  2fc39536332d  20 hours ago  66.7MB

Adding custom tags

The user can add any number of custom tags using the --add-custom-tag option:

werf build --repo REPO --add-custom-tag main

# You can add some alias tags.
werf build --repo REPO --add-custom-tag main --add-custom-tag latest --add-custom-tag prerelease

The tag template may include the following parameters:

• %image%, %image_slug% or %image_safe_slug% to use an image name defined in werf.yaml (mandatory when building multiple images);
• %image_content_based_tag% to use the content-based werf tag.

werf build --repo REPO --add-custom-tag "%image%-latest"

NOTE: When you use the options listed above, werf still creates the additional alias tags that reference the automatic hash tags. It is not possible to completely disable auto-tagging.

Layer-by-layer image caching

Layer-by-layer image caching is essential part of the werf build process. werf saves and reuses the build cache in the container registry and synchronizes parallel builders.

NOTE: It is assumed that the image repository for the project will not be deleted or cleaned by third-party tools, otherwise it will have negative consequences for users of a werf-based CI/CD (see image cleanup).

Dockerfile

By default, Dockerfile images are cached by a single image in the container registry.

To enable layered caching of Dockerfile instructions in the container registry, use the staged directive in werf.yaml. The directive can be set globally at the root level of werf.yaml for all images, or locally for specific images where the local setting will override the global one:

# werf.yaml
image: example
dockerfile: ./Dockerfile
staged: true

IMPORTANT: The staged: true option is supported only when using the Buildah builder.

Stapel

Stapel images are cached layer-by-layer in the container registry by default and do not require any configuration.

Cache versioning

You can use the global directive build.cacheVersion or its local alternative <image>.cacheVersion to explicitly manage the cache version of images through configuration and ensure reproducibility of all previous builds. If both directives are specified, the local one takes precedence.

Usage example:

project: test
configVersion: 1
build:
  cacheVersion: global-cache-version
---
image: backend
cacheVersion: user-cache-version
dockerfile: Dockerfile
---
image: frontend
cacheVersion: frontend-cache-version
dockerfile: Dockerfile
staged: true
---
image: user
cacheVersion: user-cache-version
from: alpine:3.14

Checking that images are built

werf build --check-built-images (aliases: --require-built-images, -Z, $WERF_CHECK_BUILT_IMAGES), and --require-built-images on the commands that process images without building them, check that every image the project needs is already published. Missing stages produce stages required; unavailable output images or custom tags also cause the check to fail.

The check is read-only, and stage discovery is limited to the main repository:

• a stage found in a --secondary-repo is not promoted into the main repository, and secondary repositories are not listed at all — so a project whose stages only exist in a secondary repository fails the check until a regular build copies them over;
• nothing is published: no stage, no manifest list for a multi-platform image, no custom tag, no managed-image record and no Git metadata. Custom tags are verified to exist instead of being created. werf build --check-built-images and its aliases also skip synchronization registration and use host-local locks; other commands using --require-built-images retain their normal synchronization initialization. The automatic host cleanup is not run either, so a check never deletes local cache data or local backend images;
• the configured output is still validated, read-only: with a --final-repo, the final image is required to exist there and is not copied into it, so the check never reports an image as available at an address that does not have it.

Unlike a regular build, the check never trusts a negative result of the per-command listing: when the listing shows no stage for a digest, the main repository is listed afresh, so that a stage published while the check runs is reported as built rather than missing. A stage already present in the listing is used as is.

Parallelism and image assembly order

All the images described in werf.yaml are built in parallel on the same build host. Each image starts building as soon as all the images it depends on have been built — an image never waits for unrelated images.

Note that the build log output is grouped per worker rather than per image dependency: the order of image log blocks may differ from the dependency order and may vary between runs.

When Dockerfile stages are used, the parallelism of their assembly is also determined based on the dependency tree. On top of that, if different images use a Dockerfile stage declared in werf.yaml, werf will make sure that this common stage is built only once, without any redundant rebuilds.

The parallel assembly in werf is regulated by two parameters: --parallel and --parallel-tasks-limit. By default, the parallel build is enabled and no more than 5 images can be built at a time. Setting --parallel-tasks-limit to 0 or a negative value runs one worker per image, so all currently eligible images are built simultaneously.

Let’s look at the following example:

# backend/Dockerfile
FROM node as backend
WORKDIR /app
COPY package*.json /app/
RUN npm ci
COPY . .
CMD ["node", "server.js"]

# frontend/Dockerfile

FROM ruby as application
WORKDIR /app
COPY Gemfile* /app
RUN bundle install
COPY . .
RUN bundle exec rake assets:precompile
CMD ["rails", "server", "-b", "0.0.0.0"]

FROM nginx as assets
WORKDIR /usr/share/nginx/html
COPY configs/nginx.conf /etc/nginx/conf.d/default.conf
COPY --from=application /app/public/assets .
COPY --from=application /app/vendor .
ENTRYPOINT ["nginx", "-g", "daemon off;"]

project: my-project
configVersion: 1
---
image: backend
dockerfile: Dockerfile
context: backend
---
image: frontend
dockerfile: Dockerfile
context: frontend
target: application
---
image: frontend-assets
dockerfile: Dockerfile
context: frontend
target: assets

There are 3 images: backend, frontend and frontend-assets. They have no werf-level dependencies on each other — COPY --from=application is resolved inside the Dockerfile build, and staged defaults to false — so all of them land in Level #0. werf-level dependencies (dependencies:, import:, fromImage: or the stages of a Dockerfile image with staged: true) produce later levels.

In this case, werf will print the following build plan:

┌ Concurrent build plan (no more than 5 images at the same time)
│ Level #0:
│ - 🛳️  (1/3) image backend
│ - 🛳️  (2/3) image frontend
│ - 🛳️  (3/3) image frontend-assets
└ Concurrent build plan (no more than 5 images at the same time)

Network isolation

werf supports configuring the networking mode for the build containers. This allows you to restrict network access during the build process, which can be useful for security or reproducibility.

Network isolation is currently supported only when using the Docker container backend. It works for both Dockerfile and Stapel image syntaxes.

Configuration

You can specify the network mode in the werf.yaml configuration for each image using the network parameter:

project: my-project
configVersion: 1
---
image: backend
dockerfile: Dockerfile
network: none # network is disabled during the build
---
image: frontend
from: alpine:3.14
network: host # use host's network

CLI option

You can also set the network mode globally for the build process using the --backend-network CLI option:

werf build --backend-network none

The --backend-network CLI option has priority over the network parameter defined in the werf.yaml configuration.

Using the SSH agent

werf allows using the SSH agent for authentication when accessing remote Git repositories or executing commands in build containers.

Note: Only the root user inside the build container can access the UNIX socket specified by the SSH_AUTH_SOCK environment variable.

By default, werf attempts to use the system’s running SSH agent by detecting it via the SSH_AUTH_SOCK environment variable.

If an SSH agent is not running, werf can automatically launch a temporary agent and load available keys (~/.ssh/id_rsa|id_dsa). This agent operates only during the execution of the command and does not conflict with the system’s SSH agent.

Specifying specific SSH keys

The --ssh-key PRIVATE_KEY_FILE_PATH flag allows restricting the SSH agent to specific keys (it can be used multiple times to add several keys). werf will start a temporary SSH agent with only the specified keys.

werf build --ssh-key ~/.ssh/private_key_1 --ssh-key ~/.ssh/private_key_2

Limitations on macOS

When working on macOS, keep in mind that containers are always launched inside the Linux VM of Docker Desktop. Docker Desktop provides its own proxy socket, which forwards the system SSH socket (typically the one started by launchd for the current user). It is not possible to use an arbitrary agent.

As a result, the temporary SSH agent and the --ssh-key option are not supported on macOS. To use SSH keys, you must add them in advance to the system SSH agent.

Multi-platform and cross-platform building

werf can build images for either the native host platform in which it is running, or for arbitrary platform in cross-platform mode using emulation. It is also possible to build images for multiple target platforms at once (i.e. manifest-list images).

Multi-platform builds use the cross-platform instruction execution mechanics provided by the Linux kernel and the QEMU emulator. List of supported architectures. Refer to the Installation section for more information on how to configure the host system to do cross-platform builds.

The table below summarizes support of multi-platform building for different configuration syntaxes, building modes, and build backends:

Building for single target platform

The user can choose the target platform for the images to be built using the --platform parameter:

werf build --platform linux/arm64

— werf builds all the final images from werf.yaml for the target platform using emulation.

You can also set the target platform using the build.platform configuration directive:

# werf.yaml
project: example
configVersion: 1
build:
  platform:
  - linux/arm64
---
image: frontend
dockerfile: frontend/Dockerfile
---
image: backend
dockerfile: backend/Dockerfile

In this case, running the werf build command without parameters will start the image build process for the specified platform (the explicitly passed --platform parameter will override the werf.yaml settings).

Building for multiple target platforms

werf supports simultaneous image building for multiple platforms. In this case, werf publishes a special manifest in the container registry, which includes the built images for each specified target platform (pulling such an image will provide the client with the image built for the client platform).

Here is how you can define a common list of target platforms for all images in the werf.yaml:

# werf.yaml
project: example
configVersion: 1
build:
  platform:
  - linux/arm64
  - linux/amd64
  - linux/arm/v7

It is possible to define list of target platforms separately per image in the werf.yaml (this setting will have a priority over common list of target platforms):

# werf.yaml
project: example
configVersion: 1
---
image: mysql
dockerfile: ./Dockerfile.mysql
platform:
- linux/amd64
---
image: backend
dockerfile: ./Dockerfile.backend
platform:
- linux/amd64
- linux/arm64

You can also override this list using the --platform parameter as follows:

werf build --platform=linux/amd64,linux/i386

— this parameter will override all platform settings specified in the werf.yaml (both common list and per-image list).

Using mirrors for docker.io

You can set up mirrors for the default docker.io container registry.

Docker

If you are using Docker backend for building images, add registry-mirrors to the /etc/docker/daemon.json file:

{
  "registry-mirrors": ["https://<my-docker-io-mirror-host>"]
}

Then restart the Docker daemon and logout from docker.io with:

werf cr logout

Buildah

If you are using Buildah backend, instead of editing daemon.json, add the --container-registry-mirror option to werf commands. For example:

werf build --container-registry-mirror=mirror.gcr.io

To add multiple mirrors, use the --container-registry-mirror option multiple times.

In addition to the command-line option, you can use the environment variables WERF_CONTAINER_REGISTRY_MIRROR_*, for example:

export WERF_CONTAINER_REGISTRY_MIRROR_GCR=mirror.gcr.io
export WERF_CONTAINER_REGISTRY_MIRROR_LOCAL=docker.mirror.local

Mirrors configured this way are treated as secure (https) mirrors by default. If needed, you can enable werf global insecure mode for registry access with --insecure-registry or --skip-tls-verify-registry.

werf also reads container registry mirrors and standalone insecure registries from registries.conf.

The following paths are supported in priority order:

1. the path from CONTAINERS_REGISTRIES_CONF;
2. ~/.config/containers/registries.conf;
3. /etc/containers/registries.conf.

If CONTAINERS_REGISTRIES_CONF is set, werf uses only that file and its neighboring <path>.d directory.

If CONTAINERS_REGISTRIES_CONF is not set, werf uses the first existing file from the standard paths and the neighboring <path>.d directory for that file.

werf uses the following data from this configuration:

• mirrors for docker.io;
• standalone insecure registries from [[registry]] insecure = true.

Insecure mirrors should be configured through registries.conf. An insecure mirror for docker.io does not automatically make the same host a standalone insecure registry. If the same host must be used both as a docker.io mirror and as a standalone insecure registry, it must be described by two separate entries.

Using container registry

In werf, the container registry is used not only to store the final images, but also to store the build cache and service data required for werf (e.g., metadata for cleaning the container registry based on Git history). The container registry is set by the --repo parameter:

werf converge --repo registry.mycompany.org/project

There are a number of additional repositories on top of the main repository:

• --final-repo to store the final images in a dedicated repository;
• --meta-repo to store werf service metadata (used for cleanup based on Git history) in a dedicated repository;
• --secondary-repo to use the repository in read-only mode (e.g. to use a container registry CI that you cannot push into, but you can reuse the build cache);
• --cache-repo to set the repository containing the build cache alongside the builders.

Caution! For werf to operate properly, the container registry must be persistent, and cleaning should only be done with the werf cleanup special command.

When listing tags, werf requests up to 1 000 000 tags per page, so listing even a large repository usually takes a few requests instead of hundreds. A container registry may answer with a smaller page and a pagination link, and werf follows such links. The larger the page, the larger the response werf holds in memory: a tag is at most 128 bytes, so a full page of a million tags can weigh over a hundred megabytes. Set WERF_DOCKER_REGISTRY_TAGS_PAGE_SIZE to another number of tags per page to change that, or to 0 to use the page size of the container registry client library (1000 tags). A negative or non-numeric value is an error.

Amazon ECR limits pages to 1000 tags, so for the ecr container registry implementation and for public.ecr.aws werf uses the client library page size right away, whatever the environment variable says. If any other registry answers a listing with a recognized page size rejection, werf repeats that listing with the client library page size and, if that succeeds, keeps using this page size for that registry host until the werf command ends. Any other error is returned as is.

Extra repository for final images

If necessary, the so-called final repositories can be used to exclusively store the final images.

werf build --repo registry.mycompany.org/project --final-repo final-registry.mycompany.org/project-final

Final repositories reduce image retrieval time and network load by bringing the container registry closer to the Kubernetes cluster on which the application is being deployed. Final repositories can also be used in the same container registry as the main repository (--repo), if necessary.

Extra repository for service metadata

By default, werf stores service metadata in the main repository (--repo) alongside image stages. If necessary, that metadata — image-metadata used for cleanup based on Git history, the managed images list, custom-tag metadata, and the last-cleanup record — can be stored in a separate repository via --meta-repo:

werf build --repo registry.mycompany.org/project --meta-repo registry.mycompany.org/project-meta

(Rejected-stage markers and the custom-tag aliases on stage images stay in --repo because they are tied to stage images.)

Separating metadata from stages is useful when the repository accumulates a large number of service tags. image-metadata records are written per image, per commit and per stage, so under active development they quickly outnumber the stage images themselves, while every werf operation that resolves a stage lists the whole tag set of --repo and filters it. Moving the metadata into --meta-repo keeps that listing proportional to the stages alone. Every werf command (build, cleanup, purge, etc.) must be invoked with the same --meta-repo value.

Safeguard against inconsistent --meta-repo usage

To prevent metadata from splitting between the two repositories (which can make cleanup delete in-use images), werf records a per-project marker in --repo on the first metadata write to a --meta-repo. Afterwards the marker is enforced on every run:

• omitting --meta-repo, or passing a different one, is a hard error — including a --meta-repo that resolves to the same repository as --repo;
• read-only commands only validate the marker and never write it, so pull-only credentials keep working.

Adopting --meta-repo on an existing project

Passing --meta-repo to a project that already has metadata in --repo is allowed: new metadata goes to the meta-repo right away. The metadata already in --repo does not follow, though, and werf reading only --meta-repo cannot see it — so move it before the first cleanup, which would otherwise decide the fate of stages from an incomplete picture and delete images that are still in use:

werf meta-repo migrate --from registry.mycompany.org/project --to registry.mycompany.org/project-meta

migrate copies metadata from --from into --to (copy-first, verified, idempotent — safe to re-run), records the marker in --from, and then deletes each original from --from once its copy is verified present in --to. Pass --remove-source=false to keep the originals.

To release the safeguard, run werf meta-repo detach --repo registry.mycompany.org/project. This removes only the marker; the metadata is not moved back, so subsequent runs without --meta-repo will read stale or empty metadata from --repo.

werf purge removes the marker as its last step, after the project’s stages and metadata are deleted. A re-created project can therefore be built without --meta-repo again, with no werf meta-repo detach needed. If the purge fails partway, the marker is left in place so that the metadata still in --meta-repo stays protected.

Extra repository for quick access to the build cache

You can specify one or more so-called caching repositories using the --cache-repo parameter.

# An extra caching repository on the local network.
werf build --repo registry.mycompany.org/project --cache-repo localhost:5000/project

A caching repository can help reduce build cache loading times. However, for this to work, download speeds from a caching repository must be significantly higher than those from the main repository. This is usually achieved by hosting a container registry on the local network, but it is not mandatory.

Caching repositories have higher priority than the main repository when the build cache is retrieved. When caching repositories are used, the build cache remains stored in the main repository as well.

You can clean up a caching repository by deleting it entirely without any risks.

Synchronizing builders

To coordinate publication of built images, werf synchronizes parallel builders. By default, the public synchronization service at https://synchronization.werf.io/ is used and no extra user interaction is required.

Using your own synchronization service

HTTP server

The synchronization server can be run with the werf synchronization command. In the example below, port 55581 (the default one) is used:

werf synchronization --host 0.0.0.0 --port 55581

By default, the HTTP server stores locks in process memory. Separate server processes do not share locks, even when given the same directory options; restarting the server loses its locks. Use werf synchronization --kubernetes to store locks in ConfigMaps in the fixed werf-synchronization namespace. The --local, --local-lock-manager-base-dir, --local-stages-storage-cache-base-dir, --kubernetes-namespace-prefix and --ttl options remain accepted for compatibility but have no effect.

— This server only supports HTTP mode. To use HTTPS, you have to configure additional SSL termination by third-party tools (e.g., via the Kubernetes Ingress).

Then, for all werf commands that use the --repo parameter, the --synchronization=http[s]://DOMAIN parameter must be specified as well, for example:

werf build --repo registry.mydomain.org/repo --synchronization https://synchronization.domain.org
werf converge --repo registry.mydomain.org/repo --synchronization https://synchronization.domain.org

Kubernetes synchronization

Use Kubernetes ConfigMap locks directly with --synchronization=kubernetes://NAMESPACE[:CONTEXT][@CONFIG_PATH]. An embedded kubeconfig is also supported: kubernetes://NAMESPACE@base64:BASE64_CONFIG_DATA. All builders must use the same cluster and namespace. The client needs permission to create the namespace and to read, create and update its ConfigMaps.

Local synchronization

Local synchronization is enabled by the --synchronization=:local option. The local lock manager uses file locks provided by the operating system.

werf build --repo registry.mydomain.org/repo --synchronization :local
werf converge --repo registry.mydomain.org/repo --synchronization :local

NOTE: This method is only suitable if all werf runs are triggered by the same runner in your CI/CD system.

Build report

A build report captures the results of a build: image names, tags, digests, and other metadata. It can be saved to a file and then consumed by other werf commands to skip rebuilding.

Saving a build report

Use --save-build-report to save a build report to a file:

werf build --save-build-report --repo REPO

By default, the report is saved to .werf-build-report.json in JSON format. Use --build-report-path to specify a custom path — the format is auto-detected by the file extension (.json, .env):

werf build --save-build-report --build-report-path .werf-build-report.env --repo REPO

The --save-build-report flag is supported by all commands that perform a build.

Important: It is impossible to get the tags before the build — they are generated during the build process. To use the tags, save them after the build with --save-build-report.

JSON format

The JSON report contains detailed information about the build:

• Runtime — build runtime information:
  • werf version that produced the report (WerfVersion)
  • Selected container backend (Backend: docker or buildah)
  • Whether werf is running inside a container (InContainer).
• Images — list of built images:
  • Image name in werf (WerfImageName)
  • Image config type (ConfigType: stapel, dockerfile, staged, unknown)
  • Image tags (DockerImageName, DockerRepo, DockerTag)
  • Target platform (TargetPlatform), for example linux/amd64
  • Whether the image was rebuilt (Rebuilt)
  • Whether the image is final or intermediate (Final). Final images are available in Helm chart values, can be tagged with custom tags, published to the final repository, and exported. Intermediate images (final: false) are used only as build dependencies
  • Image size in bytes (Size) and build time in seconds (BuildTime)
  • Git commit the image was built on (Commit)
  • Build stages (Stages) with details:
    • Stage name (Name)
    • Tags (DockerImageName, DockerTag, DockerImageID, DockerImageDigest)
    • Stage image creation time in Unix time nanoseconds (CreatedAt)
    • Size in bytes (Size)
    • Source of the base image (SourceType: local, secondary, cache-repo, registry)
    • Whether the base image was pulled (BaseImagePulled)
    • Whether the stage was rebuilt (Rebuilt)
    • Stage build time in seconds (BuildTime)
    • Git commit the stage was built on (Commit).

• ImagesByPlatform — per-platform breakdown for multiarch builds. This field is populated only when the WERF_ENABLE_REPORT_BY_PLATFORM=1 environment variable is set. The record structure is the same as in Images, but the data is grouped by image name and platform.

• Operations — aggregated timings of low-level operations collected for the whole command run (backend image builds and inspections, registry API calls, git operations, lock acquisitions and so on). Populated only when the --build-report-operations flag ($WERF_BUILD_REPORT_OPERATIONS) is set or debug logging is enabled (--log-debug). Operation keys are named subsystem: operation, for example registry: tags list, docker: image build or git: clone; they name backend calls, not individual HTTP requests. For each operation: the number of calls (Count), summed duration across parallel workers (TotalTimeSeconds), wall-clock duration as the union of possibly overlapping intervals (WallTimeSeconds), average (AvgTimeSeconds) and maximum (MaxTimeSeconds) durations. Operations mostly counts the calls that actually reached the backend: where the timer sits inside the caching layer, a lookup answered from that cache adds no call here. This does not hold everywhere: the git: patch and git: archive timers start before the disk cache is probed, so a call served from the disk cache is still counted — Operations figures cannot in general be derived from the cache outcomes in CacheOperations. Some recorded calls internally make another recorded call — a Buildah image pull inspects the image it has just pulled — so the rows may overlap and nest; do not add them up into the duration of the command. The console summary covers the whole command run, while a saved report covers the operations recorded since the previous report of the same command: with --follow each report includes everything since the previous one — the polling between builds and failed retry attempts included.

• CacheOperations — per-layer counters of how the caches answered the lookups that went through them, keyed by the same operation names as Operations and then by the caching layer the lookup reached (memory or disk). Every call through a caching layer is classified exactly once and recorded when it completes: Hit (a usable cached result, an empty one included), Miss (the cache had no usable result — which does not by itself mean the underlying call ran, as a shared or canceled lookup may never reach the backend) or Bypass (the caller asked for a fresh result and never consulted the cache). Lookups is their sum. A layered cache counts one lookup per layer the call actually reached: a result found in memory leaves the disk record of that operation untouched, while a memory miss answered from disk is one lookup on each of the two layers. The records of one operation therefore must not be added up into the number of requests the application made — each of them describes its own layer. Shared counts the calls that joined an already in-flight request instead of starting one, and is therefore included in Miss or Bypass: a local image list that attached to a listing another caller had already started is counted as shared as well, even when it then rejected that listing as too old and waited for the next one, with the outcome its own cache option gives it (Miss when it consulted the cache, Bypass when it asked for a fresh result). The hit rate is Hit / (Hit + Miss) per layer and is not serialized separately. A caching layer that received no lookup has no record at all rather than a row of zeros. Populated under the same conditions as Operations, and a saved report covers only the lookups recorded since the previous report of the same command.

• StageCache — per-source counters of how stages were satisfied during the build, counted in stages: found in the local or repo stages storage, copied from a secondary storage, built, or discarded. A run that answered every image from the content-based fast path works with no stage at all, so the section and the console Stages: line are omitted rather than reporting zeros. discarded counts stages that were built locally and then thrown away because another builder had already published a suitable stage by the time this one finished: such a stage is also counted as found in the stages storage, so discarded is a subset of the reused stages and not an additional outcome. It says nothing about the published stage being broken or about old images being removed. Populated only when the --build-report-operations flag ($WERF_BUILD_REPORT_OPERATIONS) is set or debug logging is enabled (--log-debug).

• RegistryCache — counters of tag-list requests using a cached or shared result: registry tags cache hit (the listing came from the in-memory tags cache) and registry tags shared result (the result was shared by concurrent requests for the same repository). A shared result is counted for every caller, including the one that initiated the registry request, so this is not a count of avoided network requests. These counters are kept apart from StageCache because they count requests, not stages. They are kept for compatibility and are superseded by CacheOperations, which classifies every lookup instead of counting two particular situations. Populated only when the --build-report-operations flag ($WERF_BUILD_REPORT_OPERATIONS) is set or debug logging is enabled (--log-debug), and omitted when no such request was recorded. Both cache sections follow the same rules as Operations: the console summary covers the whole command run, while a saved report covers only the interval since the previous report of the same command.

• Recovery — counters of recovering from a broken or conflicting storage state, kept apart from StageCache because they count failures, not how a stage was satisfied: broken stage detections (a stage read, fetch or mutation that the repo stages storage rejected as a broken image; a stage that is merely missing, rejected or unavailable is not broken, and every independent detection is counted again, including repeated lookups of the same stage) and conveyor restarts (conveyor attempts after the first one, caused by an unexpected stages storage state; a planned backoff or a cancellation before the next attempt adds nothing). The section and the console Recovery: line are omitted when nothing was detected. Populated under the same conditions as Operations.

Example report in JSON format (the Operations, CacheOperations, StageCache, RegistryCache and Recovery sections are present because the report was generated with --build-report-operations). The operation names and the figures below are illustrative:

{
  "Runtime": {
    "WerfVersion": "v3.6.1",
    "Backend": "docker",
    "InContainer": false
  },
  "Images": {
    "frontend": {
      "WerfImageName": "frontend",
      "ConfigType": "dockerfile",
      "DockerRepo": "localhost:5000/demo-app",
      "DockerTag": "079dfdd3f51a800c269cdfdd5e4febfcc1676b2c0d533f520255961c-1752501317353",
      "DockerImageID": "sha256:9b3a32dfe5a4aa46d96547e3f8e678626f96741776d78656ea72cab7117612bf",
      "DockerImageDigest": "sha256:54f564edebb6e0699dc0e43de4165488f86fbc76b0c89d88311d7cc06ae397f5",
      "DockerImageName": "localhost:5000/demo-app:079dfdd3f51a800c269cdfdd5e4febfcc1676b2c0d533f520255961c-1752501317353",
      "Rebuilt": true,
      "Final": true,
      "Size": 20960980,
      "BuildTime": "4.08",
      "Commit": "9d1bb68ca2f4e8b0e2b6e5f5a3c7d1e4f2a0b3c9",
      "Stages": [
        {
          "Name": "from",
          "DockerImageName": "localhost:5000/demo-app:6f40fd07cdb62e03d7238e1fccb3341379bcd677ff6d7575317f3783-1752501287209",
          "DockerTag": "6f40fd07cdb62e03d7238e1fccb3341379bcd677ff6d7575317f3783-1752501287209",
          "DockerImageID": "sha256:2ae3fbc31d2b5f0d7d105a74693ed14bec6b106ad43c660b9162dfa00d24d4d0",
          "DockerImageDigest": "sha256:c4449ccfaee03e5b601290909e4d69178c40e39c9d2daf57d1fd74093beb4e10",
          "CreatedAt": 1752501286000000000,
          "Size": 20960798,
          "SourceType": "",
          "BaseImagePulled": false,
          "Rebuilt": true,
          "BuildTime": "3.42",
          "Commit": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2"
        },
        {
          "Name": "install",
          "DockerImageName": "localhost:5000/demo-app:079dfdd3f51a800c269cdfdd5e4febfcc1676b2c0d533f520255961c-1752501317353",
          "DockerTag": "079dfdd3f51a800c269cdfdd5e4febfcc1676b2c0d533f520255961c-1752501317353",
          "DockerImageID": "sha256:9b3a32dfe5a4aa46d96547e3f8e678626f96741776d78656ea72cab7117612bf",
          "DockerImageDigest": "sha256:54f564edebb6e0699dc0e43de4165488f86fbc76b0c89d88311d7cc06ae397f5",
          "CreatedAt": 1752501316000000000,
          "Size": 20960980,
          "SourceType": "",
          "BaseImagePulled": false,
          "Rebuilt": true,
          "BuildTime": "0.46",
          "Commit": "9d1bb68ca2f4e8b0e2b6e5f5a3c7d1e4f2a0b3c9"
        }
      ]
    }
  },
  "ImagesByPlatform": {},
  "Operations": {
    "registry: tags list": {
      "Count": 2,
      "TotalTimeSeconds": 2.905423333,
      "WallTimeSeconds": 1.8,
      "AvgTimeSeconds": 1.4527116665,
      "MaxTimeSeconds": 1.611141611
    },
    "docker: image build": {
      "Count": 2,
      "TotalTimeSeconds": 0.831474958,
      "WallTimeSeconds": 0.831474958,
      "AvgTimeSeconds": 0.415737479,
      "MaxTimeSeconds": 0.421835292
    },
    "docker: image list": {
      "Count": 1,
      "TotalTimeSeconds": 0.110243333,
      "WallTimeSeconds": 0.110243333,
      "AvgTimeSeconds": 0.110243333,
      "MaxTimeSeconds": 0.110243333
    },
    "git: clone": {
      "Count": 1,
      "TotalTimeSeconds": 0.213458291,
      "WallTimeSeconds": 0.213458291,
      "AvgTimeSeconds": 0.213458291,
      "MaxTimeSeconds": 0.213458291
    },
    "git: patch": {
      "Count": 1,
      "TotalTimeSeconds": 0.042113250,
      "WallTimeSeconds": 0.042113250,
      "AvgTimeSeconds": 0.042113250,
      "MaxTimeSeconds": 0.042113250
    },
    "sync: lock acquire": {
      "Count": 2,
      "TotalTimeSeconds": 0.001153668,
      "WallTimeSeconds": 0.001153668,
      "AvgTimeSeconds": 0.000576834,
      "MaxTimeSeconds": 0.000661667
    }
  },
  "CacheOperations": {
    "registry: tags list": {
      "memory": {
        "Lookups": 12,
        "Hit": 9,
        "Miss": 3,
        "Bypass": 0,
        "Shared": 1
      }
    },
    "git: patch": {
      "memory": {
        "Lookups": 3,
        "Hit": 2,
        "Miss": 1,
        "Bypass": 0,
        "Shared": 0
      },
      "disk": {
        "Lookups": 1,
        "Hit": 0,
        "Miss": 1,
        "Bypass": 0,
        "Shared": 0
      }
    },
    "docker: image list": {
      "memory": {
        "Lookups": 5,
        "Hit": 4,
        "Miss": 1,
        "Bypass": 0,
        "Shared": 0
      }
    }
  },
  "StageCache": {
    "built": 2,
    "found in repo stages storage": 1,
    "discarded": 1
  },
  "RegistryCache": {
    "registry tags cache hit": 9,
    "registry tags shared result": 2
  },
  "Recovery": {
    "broken stage detections": 1,
    "conveyor restarts": 1
  }
}

To extract final image tags from a JSON report, you can use the jq utility:

jq -r '.Images | to_entries | map({key: .key, value: .value.DockerImageName}) | from_entries' .werf-build-report.json

Result:

{
  "backend": "localhost:5000/demo-app:caeb9005a06e34f0a20ba51b98d6b99b30f5cf3b8f5af63c8f3ab6c3-1752510176215",
  "frontend": "localhost:5000/demo-app:079dfdd3f51a800c269cdfdd5e4febfcc1676b2c0d533f520255961c-1752501317353"
}

envfile format

The envfile report contains a subset of image fields as environment variables. For each image, the following variables are generated:

• WERF_<IMAGE>_DOCKER_IMAGE_NAME — full image name with tag
• WERF_<IMAGE>_DOCKER_IMAGE_ID — image ID
• WERF_<IMAGE>_DOCKER_IMAGE_DIGEST — image digest
• WERF_<IMAGE>_DOCKER_REPO — image repository
• WERF_<IMAGE>_DOCKER_TAG — image tag
• WERF_<IMAGE>_WERF_IMAGE_NAME — original image name in werf
• WERF_<IMAGE>_FINAL — whether the image is final or intermediate (true/false). Final images are available in Helm chart values, can be tagged with custom tags, published to the final repository, and exported. Intermediate images (final: false) are used only as build dependencies

Where <IMAGE> is the uppercased image name with /, -, ., + replaced by _.

Example report in envfile format:

WERF_BACKEND_DOCKER_IMAGE_NAME=localhost:5000/demo-app:b94607bcb6e03a6ee07c8dc912739d6ab8ef2efc985227fa82d3de6f-1752510311968
WERF_BACKEND_DOCKER_IMAGE_ID=sha256:a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4e5f6a1b2
WERF_BACKEND_DOCKER_IMAGE_DIGEST=sha256:f6e5d4c3b2a1f6e5d4c3b2a1f6e5d4c3b2a1f6e5d4c3b2a1f6e5d4c3b2a1f6e5
WERF_BACKEND_DOCKER_REPO=localhost:5000/demo-app
WERF_BACKEND_DOCKER_TAG=b94607bcb6e03a6ee07c8dc912739d6ab8ef2efc985227fa82d3de6f-1752510311968
WERF_BACKEND_WERF_IMAGE_NAME=backend
WERF_BACKEND_FINAL=true
WERF_FRONTEND_DOCKER_IMAGE_NAME=localhost:5000/demo-app:079dfdd3f51a800c269cdfdd5e4febfcc1676b2c0d533f520255961c-1752501317353
WERF_FRONTEND_DOCKER_IMAGE_ID=sha256:9b3a32dfe5a4aa46d96547e3f8e678626f96741776d78656ea72cab7117612bf
WERF_FRONTEND_DOCKER_IMAGE_DIGEST=sha256:54f564edebb6e0699dc0e43de4165488f86fbc76b0c89d88311d7cc06ae397f5
WERF_FRONTEND_DOCKER_REPO=localhost:5000/demo-app
WERF_FRONTEND_DOCKER_TAG=079dfdd3f51a800c269cdfdd5e4febfcc1676b2c0d533f520255961c-1752501317353
WERF_FRONTEND_WERF_IMAGE_NAME=frontend
WERF_FRONTEND_FINAL=true

Using a build report

A build report serves as a contract between CI/CD pipeline stages: the build stage produces it, and downstream stages (deploy, export, render) consume it. This lets you build images once and reuse the results across multiple jobs or environments without rebuilding. See Deploying using a build report for a detailed CI/CD example.

Use --use-build-report to skip building and read image data from a previously saved report. The report path and format are specified with --build-report-path (format is auto-detected by file extension). Both JSON and envfile formats are supported.

The --use-build-report flag is supported by all commands that use build results.

Example of a two-step CI pipeline — build in one job, deploy in another:

# Step 1: Build and save the report
werf build --save-build-report --build-report-path .werf-build-report.env --repo REPO

# Step 2: Deploy using the saved report (no rebuild)
werf converge --use-build-report --build-report-path .werf-build-report.env --repo REPO