This chapter provides essential information for using werf comfortably:

  • Principles of working with source code and giterminism. These ensure reliability and reproducibility and unify all processes.
  • werf’s image tagging strategy. This strategy frees you of worries about tagging rules, building and deploying images.
  • What is a release and how to debug it.
  • How to free up space in the image repository that will run out sooner or later.

Working with source code and giterminism

Generally, some of the settings that affect the configuration of the deployed application are based on external data, such as parameters of the environment used for building and deploying an application, pre-defined environment variables, generated files, external resources involved in the process, etc. This directly leads to the inability to guarantee the reproducibility of the application state.

Why do we need reproducibility?

The ability to reproduce a specific application state at any moment makes the debugging process easier, allowing us to deploy the copy of the project for development or testing purposes. Plus, it makes the results of testing more credible and dependable thanks to a lower number of hidden parameters affecting the application state.

The reproducibility is essential for implementing the Infrastructure as Code approach and immutable infrastructure.

The standard mode

werf follows the giterminism principle to ensure reproducibility. That is, the current Git state (HEAD commit) determines the state of the application. By default, werf does not allow uncommitted and untracked files in the build context/configuration of the images being built. It also excludes the features that potentially have external dependencies.

We strongly recommend following this approach. However, if necessary, you can loosen giterminism restrictions explicitly and enable the features that require careful use. You can do this using the werf-giterminism.yaml file.

During development or debugging, editing the project files might be annoying due to the need to create intermediate commits. We recommend using the developer mode (or simply dev — see below for details).

The detailed documentation on determinism is available here.

The dev mode

With the dev mode, you can loosen giterminism restrictions and work with non-committed changes. The mode is enabled by the --dev option or WERF_DEV environment variable.

In the developer mode, werf uses all tracked and untracked changes while taking into account the .gitignore rules as well as user-defined exceptions set by the --dev-ignore=<glob> option or the WERF_DEV_IGNORE_<ANY> environment variable.

The follow mode

In the follow mode, werf automatically re-runs the command in response to the Git state changes:

  • the command is restarted when a new commit is created;
  • together with the dev mode (--dev) in response to any changes.

You can enable it via the --follow parameter or WERF_FOLLOW environment variable.

Choosing the mode for local development

Summary table of werf operating modes:

Mode no flags --dev --follow --dev --follow
working with committed files only + - + -
working with non-committed changes - + - +
restarting the deploy process automatically - - + +

Building and tagging images


The classic approach to building images using native Docker tools involves docker build and docker tag commands. After the build is complete, you have to push the newly created image to the registry (using the docker push command).

In werf, the entire process can be carried out with just one command — werf build. Read this article to learn more about the differences between werf and Docker.

In addition, werf supports the following features out of the box (they are enabled by default):

  • parallel builds, when multiple independent images get assembled simultaneously;
  • distributed immutable cache. In this mode, thanks to the synchronization and locking mechanisms, an arbitrary number of builders can use the shared cache in the container registry without breaking the reproducibility of previously assembled layers and images. This approach is based on the ideas of MVCC and optimistic locking.

Please, refer to the werf documentation to learn more about how data is stored in the registry. This document provides information on how the build process is structured, how the dependencies between the build stages are accounted for, and how the resulting images get their names.

Tagging images

In the manual deployment process (i.e., without werf), you have to define strict rules for tagging images and follow them (TBH, that’s no easy task). On the other hand, in werf, tagging is automatic, and the user need not worry about the tags of the built images throughout the delivery steps. For example, when templating the chart, the tags of the built images can be retrieved using the following construct: {{ .Values.werf.image.<IMAGE_NAME> }}:

      - name: app
        image: {{ }}
        command: ["/app/"]

The tag is generated based on the configuration and the image assembly context (see this article for more information on content-based tagging).

This way, werf frees you of worries about tagging rules: the tagging is done in an optimal way. The reproducibility of artifacts from earlier builds is preserved and unnecessary re-deployment of Kubernetes resources is eliminated.

Releases and Helm

werf uses Helm to deploy the application. Helm chart is a set of files that describes a related set of Kubernetes objects. A release is created when the chart is rolled out to a particular environment in a cluster.

Learn more

The werf documentation provides a detailed description of working with releases, storing and naming them.

When working with releases, werf follows giterminism principles. For Helm releases, it uses the 3-way merge approach: manual changes to the cluster that conflict with the state described in Git are corrected to conform to the latter. Note that manual changes that do not conflict with the state defined in Git remain outside the control of Helm and werf.

werf manages releases on its own, but if you want to do it the hard way, you can use the werf helm <...> commands.

You can read more about werf and Helm in this article.

How do I list the installed components?

A specific state of the deployed application is called a release. Releases provide information about components installed in the cluster (using werf and Helm), their state, and the environment they are running in.

To browse the list of releases or find out the name of the release you need, use the werf helm list -A command.

How do I delete the unneeded component?

Use werf dismiss to uninstall a specific release of an application.

Debugging the installation process

Debugging allows you to see what went wrong while werf was running. Frequently, mistakes made in chart configurations lead to problems when rolling out a release. The werf render command can help you in debugging such problems.

werf render performs all the tasks related to the building and generating charts, showing you the resulting charts instead of deploying the release to Kubernetes. It is a compute-intensive task that shows you the final result with all the required values filled in.

Note that werf render only works with files committed to Git (like all other werf commands); however, it supports the --dev mode.

Cleaning up

Over time, a lot of data can accumulate in the storage (either local or in the registry). werf has two types of cleaning commands with different purposes: werf cleanup and werf purge (refer to the documentation to learn more). The first one cleans up data in the container registry only, while the second one can clean up all werf-related data.

werf cleanup

The werf cleanup command cleans up outdated data in the registry. It safely deletes images that are no longer required using an advanced algorithm that takes into account the Git history, the contents of the registry, and the state of the cluster. If the --repo <reponame> key is provided, the command will clean a specific repository.

The werf host cleanup command is used to clean up the outdated data on the host.

werf purge

The werf host purge command allows you to free up disk space by deleting all images and other data.

The werf purge command purges all werf-related data on a host. Caution! werf purge (NOT SAFE) deletes all images, including those linked to applications running in the cluster!