Werf represents the new breed of CI/CD tools that integrates lowlevel building, cleaning and deploying tools into a single tool, which can easily be plugged into any existing CI/CD system. This is possible because werf follows established concepts of all such systems.

Werf plugs into CI/CD system using so called ci-env command. Ci-env command is responsible to gather required info from CI/CD system and define corresponding werf params using environment variables which will be referred to as ci-env.

ci-env

In the following chapters we will see:

What is ci-env

With ci-env command werf gathers data from CI/CD systems and sets modes of operation to accomplish the following tasks:

  • Docker registry integration;
  • Git integration;
  • CI/CD pipelines integration;
  • CI/CD configuration integration;
  • Configure modes of operation in CI/CD systems.

Docker registry integration

Typically CI/CD system can provide each job with:

  1. Docker registry address.
  2. Credentials to access Docker registry.

Werf ci-env command should perform login into detected Docker registry using detected credentials. See more info about docker login below. DOCKER_CONFIG=PATH_TO_TMP_CONFIG will be set.

Docker registry address will also be used as --images-repo parameter value. WERF_IMAGES_REPO=DOCKER_REGISTRY_REPO will be set.

Git integration

Typically CI/CD system that uses git runs each job in the detached commit state of git worktree. And current git-commit, git-tag or git-branch are passed to the job using environment variables.

Werf ci-env command detects current git-commit, git-tag or git-branch and uses this info to tag images from werf.yaml config. This usage of git info depends on the selected tagging scheme, more info below.

WERF_TAG_GIT_TAG=GIT_TAG or WERF_TAG_GIT_BRANCH=GIT_BRANCH will be set.

CI/CD pipelines integration

Werf can embed any info into deployed Kubernetes resources annotations and labels. Typically CI/CD system exposes for users such info as link to CI/CD web page of the project, link to job itself, job id and pipeline id and other info.

Werf ci-env command automatically detects CI/CD web page project url and embeds this url into annotations of every resource deployed with werf.

Annotation name depends on the selected CI/CD system and constructed as follows: "project.werf.io/CI_CD_SYSTEM_NAME-url": URL.

WERF_ADD_ANNOTATION_PROJECT_GIT="project.werf.io/git": URL will be set.

There are another auto annotations set by werf using any CI/CD system, also custom annotations and labels can be passed, see deploy article for details.

CI/CD configuration integration

There is a concept used in CI/CD systems named environment. Environment can define used host nodes, access parameters, Kubernetes cluster connection info, job parameters (using environment variables for example) and other info. Typical environments are: development, staging, testing, production and review environments with dynamical names.

Werf also uses concept of environment name in the deploy process.

Ci-env command detects current environment name of CI/CD system and passes it to all subsequent werf commands automatically. Slugged name of the environment will be preferred if CI/CD system exports such a name.

WERF_ENV=ENV will be set.

Configure modes of operation in CI/CD systems

Ci-env command configures cleanup policies as follows:

Colorized output of werf will be forced if CI/CD system has support for it. WERF_LOG_COLOR_MODE=on will be set.

Logging of project directory where werf runs will be forced: convinient common output to debug problems within CI/CD system, by default when running werf it will not print the directory. WERF_LOG_PROJECT_DIR=1 will be set.

So called werf process exterminator will be enabled. Some CI/CD systems will kill job process with SIGKILL linux signal when user hits the Cancel button in the user interface. Child processes of this job will continue to run till termination in this case. Werf with enabled process exterminator will constantly check in the background for its parent processes pids and if one of them has died, then werf will be terminated by itself. This mode will be enabled in CI/CD system and is disabled by default. WERF_ENABLE_PROCESS_EXTERMINATOR=1 will be set.

Logging output widht will be forced to 100 symbols, which is experimentally proven universal width to support most of the typical today screens. WERF_LOG_TERMINAL_WIDTH=100 will be set.

Ci-env tagging modes

Tagging mode determines how images from werf.yaml built by werf will be named in publish process.

The only available mode of ci-env for now is tag-or-branch mode.

Content based tagging support and corresponding ci-env tagging mode is coming soon.

tag-or-branch

Current git-tag or git-branch will be used to tag images from werf.yaml built by werf.

Image related with git-tag and git-branch will be kept in the Docker registry according to cleanup policies.

This mode makes use of werf publish params: --tag-git-tag or --tag-git-branch — automatically choosing appropriate one. These params also used in the werf deploy command.

This tagging mode is selected by --tagging-strategy=tag-or-branch option of werf ci-env command.

If used tag or branch value does not match regex ^[\w][\w.-]*$ or consists of more than 128 characters, werf slugifies this tag (read more in slug reference).

For example:

  • branch developer-feature is valid and resulted tag will be unchanged;
  • branch developer/feature is not valid and resulted tag will be developer-feature-6e0628fc.

How ci-env works

Ci-env command passes all parameters to werf using environment variables, see pass cli params as environment variables below.

werf ci-env command should be called in the begin of any CI/CD job prior running any other werf commands.

NOTE Werf ci-env command prints bash script which exports werf params using environment variables. So to actually use ci-env command user must source command output using bash. For example:

source <(werf ci-env gitlab --tagging-strategy tag-or-branch --verbose)
werf build-and-publish --stages-storage :local

Sourcing ci-env command output will also result in printing all exported variables with values to the screen.

Example of werf ci-env command output script without sourcing:

### DOCKER CONFIG
echo '### DOCKER CONFIG'
export DOCKER_CONFIG="/tmp/werf-docker-config-204033515"
echo 'export DOCKER_CONFIG="/tmp/werf-docker-config-204033515"'

### IMAGES REPO
echo
echo '### IMAGES REPO'
export WERF_IMAGES_REPO="registry.domain.com/project/x"
echo 'export WERF_IMAGES_REPO="registry.domain.com/project/x"'

### TAGGING
echo
echo '### TAGGING'
export WERF_TAG_GIT_TAG="v1.0.3"
echo 'export WERF_TAG_GIT_TAG="v1.0.3"'

### DEPLOY
echo
echo '### DEPLOY'
export WERF_ENV="dev"
echo 'export WERF_ENV="dev"'
export WERF_ADD_ANNOTATION_PROJECT_GIT="project.werf.io/git=https://gitlab.domain.com/project/x"
echo 'export WERF_ADD_ANNOTATION_PROJECT_GIT="project.werf.io/git=https://gitlab.domain.com/project/x"'
export WERF_ADD_ANNOTATION_CI_COMMIT="ci.werf.io/commit=b9a1ddd366aa6a20a0fd43fb6612f349d33465ff"
echo 'export WERF_ADD_ANNOTATION_CI_COMMIT="ci.werf.io/commit=b9a1ddd366aa6a20a0fd43fb6612f349d33465ff"'
export WERF_ADD_ANNOTATION_GITLAB_CI_PIPELINE_URL="gitlab.ci.werf.io/pipeline-url=https://gitlab.domain.com/project/x/pipelines/43107"
echo 'export WERF_ADD_ANNOTATION_GITLAB_CI_PIPELINE_URL="gitlab.ci.werf.io/pipeline-url=https://gitlab.domain.com/project/x/pipelines/43107"'
export WERF_ADD_ANNOTATION_GITLAB_CI_JOB_URL="gitlab.ci.werf.io/job-url=https://gitlab.domain.com/project/x/-/jobs/110681"
echo 'export WERF_ADD_ANNOTATION_GITLAB_CI_JOB_URL="gitlab.ci.werf.io/job-url=https://gitlab.domain.com/project/x/-/jobs/110681"'

### IMAGE CLEANUP POLICIES
echo
echo '### IMAGE CLEANUP POLICIES'
export WERF_GIT_TAG_STRATEGY_LIMIT="10"
echo 'export WERF_GIT_TAG_STRATEGY_LIMIT="10"'
export WERF_GIT_TAG_STRATEGY_EXPIRY_DAYS="30"
echo 'export WERF_GIT_TAG_STRATEGY_EXPIRY_DAYS="30"'

### OTHER
echo
echo '### OTHER'
export WERF_LOG_COLOR_MODE="on"
echo 'export WERF_LOG_COLOR_MODE="on"'
export WERF_LOG_PROJECT_DIR="1"
echo 'export WERF_LOG_PROJECT_DIR="1"'
export WERF_ENABLE_PROCESS_EXTERMINATOR="1"
echo 'export WERF_ENABLE_PROCESS_EXTERMINATOR="1"'
export WERF_LOG_TERMINAL_WIDTH="95"
echo 'export WERF_LOG_TERMINAL_WIDTH="95"'

Pass CLI params as environment variables

Any parameter of any werf command (except ci-env command itself) that can be passed via CLI, can also be passed using environment variables.

There is a common rule of conversion of CLI param to environment variable name: environment variable have WERF_ prefix, consists of capital letters, underscore symbol _ instead of dash -.

Examples:

  • --tag-git-tag=mytag can be specified by WERF_TAG_GIT_TAG=mytag;
  • --env=staging can be specified by WERF_ENV=staging;
  • --images-repo=myregistry.myhost.com/project/x can be specified by WERF_IMAGES_REPO=myregistry.myhost.com/project/x; etc.

Exception to this rule is --add-label and --add-annotation params, which can be specified multiple times. To specify this params using environment variables use following pattern: WERF_ADD_ANNOTATION_<ARBITRARY_VARIABLE_NAME_SUFFIX>="annoName1=annoValue1". For example:

export WERF_ADD_ANNOTATION_MYANNOTATION_1="annoName1=annoValue1"
export WERF_ADD_ANNOTATION_MYANNOTATION_2="annoName2=annoValue2"
export WERF_ADD_LABEL_MYLABEL_1="labelName1=labelValue1"
export WERF_ADD_LABEL_MYLABEL_2="labelName2=labelValue2"

Note that _MYANNOTATION_1, _MYANNOTATION_2, _MYLABEL_1, _MYLABEL_2 suffixes does not affect annotations and labels names or values and used only to differentiate between multiple environment variables.

Docker registry login

As noted in the Docker registry integration werf performs autologin into detected docker registry.

Ci-env command always creates a new temporal docker config.

Temporal docker config needed so that parallel running jobs cannot interfere with each other. Also temporal docker config is a more secure way, than logging into Docker registry using system-wide default docker config (~/.docker).

If DOCKER_CONFIG variable is defined or there is ~/.docker, then werf will copy this already existing config into a new temporal one. So any docker logins already made before running werf ci-env command will still be active and available using this new temporal config.

After new tmp config was created werf performs additional login into detected Docker registry.

As a result of Docker registry integration procedure werf ci-env will export DOCKER_CONFIG variable with new temporal docker config.

This config then will be used by any following werf command and also can be used by docker login command to perform any additional custom logins on your needs.

Complete list of ci-env params and customizing

As an output of ci-env command werf exports following list of variables. To customize these variables user may predefine any variable with the name prefix WERF_ prior running werf ci-env command (ci-env command will detect already defined environment variable and will use this variable as is). This can be accomplished using project environment variables or simply by exporting variables in shell.

Following variables are defined by werf ci-env command.

DOCKER_CONFIG

Path to new temporal docker config generated by werf ci-env command.

WERF_IMAGES_REPO

Within Docker registry integration procedure werf ci-env command will detect docker registry and define --images-repo param using WERF_IMAGES_REPO environment variable.

WERF_TAG_GIT_TAG

Within git integration procedure werf ci-env command will detect if this job is running for git-tag and optionally define --tag-git-tag param using WERF_TAG_GIT_TAG environment variable.

WERF_TAG_GIT_BRANCH

Within git integration procedure werf ci-env command will detect if this job is running for git-branch and optionally define --tag-git-branch param using WERF_TAG_GIT_BRANCH environment variable.

WERF_ENV

Within CI/CD configuration integration procedure werf ci-env command will detect environment name and define --env param using WERF_ENV environment variable.

WERF_ADD_ANNOTATION_PROJECT_GIT

Within CI/CD pipelines integration procedure werf ci-env command will detect web page project url and set --add-annotation param using WERF_ADD_ANNOTATION_PROJECT_GIT environment variable.

WERF_ADD_ANNOTATION_CI_COMMIT

Within CI/CD pipelines integration procedure werf ci-env command will detect current commit and set --add-annotation param using WERF_ADD_ANNOTATION_CI_COMMIT environment variable.

WERF_GIT_TAG_STRATEGY_LIMIT

Within configure modes of operation in CI/CD systems procedure werf ci-env command will set --git-tag-strategy-limit param using WERF_GIT_TAG_STRATEGY_LIMIT environment variable.

WERF_GIT_TAG_STRATEGY_EXPIRY_DAYS

Within configure modes of operation in CI/CD systems procedure werf ci-env command will set --git-tag-strategy-expiry-days param using WERF_GIT_TAG_STRATEGY_EXPIRY_DAYS environment variable.

WERF_LOG_COLOR_MODE

Within configure modes of operation in CI/CD systems procedure werf ci-env command will set --log-color-mode param using WERF_LOG_COLOR_MODE.

WERF_LOG_PROJECT_DIR

Within configure modes of operation in CI/CD systems procedure werf ci-env command will set --log-project-dir param using WERF_LOG_PROJECT_DIR.

WERF_ENABLE_PROCESS_EXTERMINATOR

Within configure modes of operation in CI/CD systems procedure werf ci-env command will set --enable-process-exterminator param using WERF_ENABLE_PROCESS_EXTERMINATOR.

WERF_LOG_TERMINAL_WIDTH

Within configure modes of operation in CI/CD systems procedure werf ci-env command will set --log-terminal-width param using WERF_LOG_TERMINAL_WIDTH.

Further reading