We propose to divide the assembly process into steps. Every step corresponds to the intermediate image (like layers in Docker) with specific functions and assignments. In werf, we call every such step a stage. So the final image consists of a set of built stages. All stages are kept in a stages storage. You can view it as a building cache of an application, however, that isn’t a cache but merely a part of a building context.

Stages

Stages are steps in the assembly process. They act as building blocks for constructing images. A stage is built from a logically grouped set of config instructions. It takes into account the assembly conditions and rules. Each stage relates to a single Docker image.

The werf assembly process involves a sequential build of stages using the stage conveyor. A stage conveyor is an ordered sequence of conditions and rules for carrying out stages. werf uses different stage conveyors to assemble various types of images depending on their configuration.

The user only needs to write a correct configuration: werf performs the rest of the work with stages

For each stage at every build, werf calculates the unique identifier of the stage called stage signature. Each stage is assembled in the assembly container that is based on the previous stage and saved in the stages storage. The stage signature is used for tagging a stage in the stages storage. werf does not build stages that already exist in the stages storage (similar to caching in Docker yet more complex).

The stage signature is calculated as the checksum of stage dependencies and previous stage signature. In the absence of stage dependencies, the stage is skipped.

It means that the stage conveyor can be reduced to several stages or even to a single from stage.

Stage dependencies

Stage dependency is a piece of data that affects the stage signature. Stage dependency may be represented by:

  • some file from a git repo with its contents;
  • instructions to build stage defined in the werf.yaml;
  • the arbitrary string specified by the user in the werf.yaml;
  • and so on.

Most stage dependencies are specified in the werf.yaml, others relate to a runtime.

The tables below illustrate dependencies of a Dockerfile image, a Stapel image, and a Stapel artifact stages dependencies. Each row describes dependencies for a certain stage. Left column contains a short description of dependencies, right column includes related werf.yaml directives and contains relevant references for more information.

stage dockerfile

target dockerfile instructions
hashsum of files related with ADD and COPY dockerfile instructions
args used in target dockerfile instructions
addHost
image: <image name... || ~>
dockerfile: <relative path>
context: <relative path>
target: <docker stage name>
args:
  <build arg name>: <value>
addHost:
- <host:ip>

stage from

from
or from image id
or from artifact id
actual digest from registry (if fromLatest: true)
fromCacheVersion
mounts
from: <image[:<tag>]>
fromLatest: <bool>
fromCacheVersion: <arbitrary string>
fromImage: <image name>
fromImageArtifact: <artifact name>
mount:
- from: build_dir
  to: <absolute or relative path>
- from: tmp_dir
  to: <absolute path>
- fromPath: <absolute or relative path>
  to: <absolute path>

stage beforeInstall

beforeInstall bash commands or ansible tasks
cacheVersion
beforeInstallCacheVersion
shell:
  beforeInstall:
  - <bash command>
  cacheVersion: <arbitrary string>
  beforeInstallCacheVersion: <arbitrary string>

or

ansible:
  beforeInstall:
  - <task>
  cacheVersion: <arbitrary string>
  beforeInstallCacheVersion: <arbitrary string>

stage importsBeforeInstall

imports before install
import:
- artifact: <artifact name>
  before: install
  add: <absolute path>
  to: <absolute path>
  owner: <owner>
  group: <group>
  includePaths:
  - <relative path or glob>
  excludePaths:
  - <relative path or glob>

stage gitArchive

git mappings
git:
- add: <absolute path>
  to: <absolute path>
  owner: <owner>
  group: <group>
  includePaths:
  - <relative path or glob>
  excludePaths:
  - <relative path or glob>
- url: <git repo url>
  branch: <branch name>
  commit: <commit>
  tag: <tag>
  add: <absolute path>
  to: <absolute path>
  owner: <owner>
  group: <group>
  includePaths:
  - <relative path or glob>
  excludePaths:
  - <relative path or glob>

stage install

install bash commands or ansible tasks
installCacheVersion
git files hashsum by install stageDependency
git:
- stageDependencies:
    install:
    - <relative path or glob>

shell:
  install:
  - <bash command>
  installCacheVersion: <arbitrary string>

or

ansible:
  install:
  - <task>
  installCacheVersion: <arbitrary string>

stage importsAfterInstall

imports after install
import:
- artifact: <artifact name>
  after: install
  add: <absolute path>
  to: <absolute path>
  owner: <owner>
  group: <group>
  includePaths:
  - <relative path or glob>
  excludePaths:
  - <relative path or glob>

stage beforeSetup

beforeSetup bash commands or ansible tasks
beforeSetupCacheVersion
git files hashsum by beforeSetup stageDependency
git:
- stageDependencies:
    beforeSetup:
    - <relative path or glob>

shell:
  beforeSetup:
  - <bash command>
  beforeSetupCacheVersion: <arbitrary string>

or

ansible:
  beforeSetup:
  - <task>
  beforeSetupCacheVersion: <arbitrary string>

stage importsBeforeSetup

imports before setup
import:
- artifact: <artifact name>
  before: setup
  add: <absolute path>
  to: <absolute path>
  owner: <owner>
  group: <group>
  includePaths:
  - <relative path or glob>
  excludePaths:
  - <relative path or glob>

stage setup

setup bash commands or ansible tasks
setupCacheVersion
git files hashsum by setup stageDependency
git:
- stageDependencies:
    setup:
    - <relative path or glob>

shell:
  setup:
  - <bash command>
  setupCacheVersion: <arbitrary string>

or

ansible:
  setup:
  - <task>
  setupCacheVersion: <arbitrary string>

stage gitCache

size of git diff between last used commit and actual

stage importsAfterSetup

imports after setup
import:
- artifact: <artifact name>
  after: setup
  add: <absolute path>
  to: <absolute path>
  owner: <owner>
  group: <group>
  includePaths:
  - <relative path or glob>
  excludePaths:
  - <relative path or glob>

stage gitLatestPatch

presence of git diff changes between last used commit and actual

stage dockerInstructions

docker instructions
docker:
  VOLUME:
  - <volume>
  EXPOSE:
  - <expose>
  ENV:
    <env name>: <env value>
  LABEL:
    <label name>: <label value>
  ENTRYPOINT: <entrypoint>
  CMD: <cmd>
  WORKDIR: <workdir>
  USER: <user>
  HEALTHCHECK: <healthcheck>

Stages storage

The stages storage contains the stages of the project. Stages can be stored in the Docker Repo or locally on a host machine.

Most commands use stages and require the reference to a specific stages storage, defined by the --stages-storage option or WERF_STAGES_STORAGE environment variable. At the moment, only the local storage, :local, is supported.

Stage naming

Stages in the local stages storage are named using the following schema — werf-stages-storage/PROJECT_NAME:STAGE_SIGNATURE.

Images

Image is a ready-to-use Docker image corresponding to a specific application state and tagging strategy.

As mentioned above, stages are steps in the assembly process. They act as building blocks for constructing images. Unlike images, stages are not intended for the direct use. The main difference between images and stages is in cleaning policies due to the stored meta-information. The process of cleaning up the stages storage is only based on the related images in the images repo.

werf creates images using the stages storage. Currently, images can only be created during the publishing process and saved in the images repo.

Images should be defined in the werf configuration file werf.yaml.

Learn more about the build process.