werf represents the new breed of CI/CD tools that integrates lowlevel building, cleaning and deploying tools into a single tool, which can easily be plugged into any existing CI/CD system. This is possible because werf follows established concepts of all such systems.
werf plugs into CI/CD system using so called ci-env command. Ci-env command is responsible to gather required info from CI/CD system and define corresponding werf params using environment variables which will be referred to as ci-env.
In the following chapters we will see:
- what is ci-env — what information werf gathers from CI/CD system and why;
- ci-env tagging modes — about different images tagging modes available;
- how ci-env works — how ci-env command should be used and how it passes info to other werf commands;
- complete list of ci-env params and customizing — complete list of all ci-env params passed to other werf commands and how to customize common params.
What is ci-env
With ci-env command werf gathers data from CI/CD systems and sets modes of operation to accomplish the following tasks:
- Docker registry integration;
- Git integration;
- CI/CD pipelines integration;
- CI/CD configuration integration;
- Configure modes of operation in CI/CD systems.
Docker registry integration
Typically CI/CD system can provide each job with:
- Docker registry address.
- Credentials to access Docker registry.
Docker registry address will also be used as
--images-repo parameter value.
WERF_IMAGES_REPO=DOCKER_REGISTRY_REPO will be set.
Typically CI/CD system that uses git runs each job in the detached commit state of git worktree. And current git-commit, git-tag or git-branch are passed to the job using environment variables.
werf ci-env command detects current git-commit, git-tag or git-branch and uses this info to tag images from
werf.yaml config. This usage of git info depends on the selected tagging scheme, more info below.
CI/CD pipelines integration
werf can embed any info into deployed Kubernetes resources annotations and labels. Typically CI/CD system exposes for users such info as link to CI/CD web page of the project, link to job itself, job id and pipeline id and other info.
werf ci-env command automatically detects CI/CD web page project url and embeds this url into annotations of every resource deployed with werf.
Annotation name depends on the selected CI/CD system and constructed as follows:
There are another auto annotations set by werf using any CI/CD system, also custom annotations and labels can be passed, see deploy article for details.
CI/CD configuration integration
There is a concept used in CI/CD systems named environment. Environment can define used host nodes, access parameters, Kubernetes cluster connection info, job parameters (using environment variables for example) and other info. Typical environments are: development, staging, testing, production and review environments with dynamical names.
werf also uses concept of environment name in the deploy process.
Ci-env command detects current environment name of CI/CD system and passes it to all subsequent werf commands automatically. Slugged name of the environment will be preferred if CI/CD system exports such a name.
WERF_ENV=ENV will be set.
Configure modes of operation in CI/CD systems
Ci-env command configures cleanup policies as follows:
- keep no more than 10 images built for git-tags,
WERF_GIT_TAG_STRATEGY_LIMIT=10will be set;
- keep images built for git-tags no more than 30 days,
WERF_GIT_TAG_STRATEGY_EXPIRY_DAYS=30will be set.
Colorized output of werf will be forced if CI/CD system has support for it.
WERF_LOG_COLOR_MODE=on will be set.
Logging of project directory where werf runs will be forced: convinient common output to debug problems within CI/CD system, by default when running werf it will not print the directory.
WERF_LOG_PROJECT_DIR=1 will be set.
So called werf process exterminator will be enabled. Some CI/CD systems will kill job process with
SIGKILL linux signal when user hits the
Cancel button in the user interface. Child processes of this job will continue to run till termination in this case. werf with enabled process exterminator will constantly check in the background for its parent processes pids and if one of them has died, then werf will be terminated by itself. This mode will be enabled in CI/CD system and is disabled by default.
WERF_ENABLE_PROCESS_EXTERMINATOR=1 will be set.
Logging output widht will be forced to 100 symbols, which is experimentally proven universal width to support most of the typical today screens.
WERF_LOG_TERMINAL_WIDTH=100 will be set.
Ci-env tagging modes
The only available mode of ci-env for now is tag-or-branch mode.
Content-based tagging support and corresponding ci-env tagging mode is coming soon.
Current git-tag or git-branch will be used to tag images from
werf.yaml built by werf.
Image related with git-tag and git-branch will be kept in the Docker registry according to cleanup policies.
This tagging mode is selected by
--tagging-strategy=tag-or-branch option of
werf ci-env command.
If used tag or branch value does not match regex
^[\w][\w.-]*$or consists of more than 128 characters, werf slugifies this tag (read more in slug reference).
developer-featureis valid and resulted tag will be unchanged;
developer/featureis not valid and resulted tag will be
How ci-env works
Ci-env command passes all parameters to werf using environment variables, see pass cli params as environment variables below.
werf ci-env command should be called in the begin of any CI/CD job prior running any other werf commands.
NOTE werf ci-env command prints bash script which exports werf params using environment variables. So to actually use ci-env command user must
source command output using bash. For example:
source <(werf ci-env gitlab --tagging-strategy tag-or-branch --verbose) werf build-and-publish --stages-storage :local
Sourcing ci-env command output will also result in printing all exported variables with values to the screen.
werf ci-env command output script without sourcing:
### DOCKER CONFIG echo '### DOCKER CONFIG' export DOCKER_CONFIG="/tmp/werf-docker-config-204033515" echo 'export DOCKER_CONFIG="/tmp/werf-docker-config-204033515"' ### IMAGES REPO echo echo '### IMAGES REPO' export WERF_IMAGES_REPO="registry.domain.com/project/x" echo 'export WERF_IMAGES_REPO="registry.domain.com/project/x"' ### TAGGING echo echo '### TAGGING' export WERF_TAG_GIT_TAG="v1.0.3" echo 'export WERF_TAG_GIT_TAG="v1.0.3"' ### DEPLOY echo echo '### DEPLOY' export WERF_ENV="dev" echo 'export WERF_ENV="dev"' export WERF_ADD_ANNOTATION_PROJECT_GIT="project.werf.io/git=https://gitlab.domain.com/project/x" echo 'export WERF_ADD_ANNOTATION_PROJECT_GIT="project.werf.io/git=https://gitlab.domain.com/project/x"' export WERF_ADD_ANNOTATION_CI_COMMIT="ci.werf.io/commit=b9a1ddd366aa6a20a0fd43fb6612f349d33465ff" echo 'export WERF_ADD_ANNOTATION_CI_COMMIT="ci.werf.io/commit=b9a1ddd366aa6a20a0fd43fb6612f349d33465ff"' export WERF_ADD_ANNOTATION_GITLAB_CI_PIPELINE_URL="gitlab.ci.werf.io/pipeline-url=https://gitlab.domain.com/project/x/pipelines/43107" echo 'export WERF_ADD_ANNOTATION_GITLAB_CI_PIPELINE_URL="gitlab.ci.werf.io/pipeline-url=https://gitlab.domain.com/project/x/pipelines/43107"' export WERF_ADD_ANNOTATION_GITLAB_CI_JOB_URL="gitlab.ci.werf.io/job-url=https://gitlab.domain.com/project/x/-/jobs/110681" echo 'export WERF_ADD_ANNOTATION_GITLAB_CI_JOB_URL="gitlab.ci.werf.io/job-url=https://gitlab.domain.com/project/x/-/jobs/110681"' ### IMAGE CLEANUP POLICIES echo echo '### IMAGE CLEANUP POLICIES' export WERF_GIT_TAG_STRATEGY_LIMIT="10" echo 'export WERF_GIT_TAG_STRATEGY_LIMIT="10"' export WERF_GIT_TAG_STRATEGY_EXPIRY_DAYS="30" echo 'export WERF_GIT_TAG_STRATEGY_EXPIRY_DAYS="30"' ### OTHER echo echo '### OTHER' export WERF_LOG_COLOR_MODE="on" echo 'export WERF_LOG_COLOR_MODE="on"' export WERF_LOG_PROJECT_DIR="1" echo 'export WERF_LOG_PROJECT_DIR="1"' export WERF_ENABLE_PROCESS_EXTERMINATOR="1" echo 'export WERF_ENABLE_PROCESS_EXTERMINATOR="1"' export WERF_LOG_TERMINAL_WIDTH="95" echo 'export WERF_LOG_TERMINAL_WIDTH="95"'
Pass CLI params as environment variables
Any parameter of any werf command (except ci-env command itself) that can be passed via CLI, can also be passed using environment variables.
There is a common rule of conversion of CLI param to environment variable name: environment variable have
WERF_ prefix, consists of capital letters, underscore symbol
_ instead of dash
--tag-git-tag=mytagcan be specified by
--env=stagingcan be specified by
--images-repo=myregistry.myhost.com/project/xcan be specified by
Exception to this rule is
--add-annotation params, which can be specified multiple times. To specify this params using environment variables use following pattern:
WERF_ADD_ANNOTATION_<ARBITRARY_VARIABLE_NAME_SUFFIX>="annoName1=annoValue1". For example:
export WERF_ADD_ANNOTATION_MYANNOTATION_1="annoName1=annoValue1" export WERF_ADD_ANNOTATION_MYANNOTATION_2="annoName2=annoValue2" export WERF_ADD_LABEL_MYLABEL_1="labelName1=labelValue1" export WERF_ADD_LABEL_MYLABEL_2="labelName2=labelValue2"
_MYLABEL_2 suffixes does not affect annotations and labels names or values and used only to differentiate between multiple environment variables.
Docker registry login
As noted in the Docker registry integration werf performs autologin into detected docker registry.
Ci-env command always creates a new temporal docker config.
Temporal docker config needed so that parallel running jobs cannot interfere with each other. Also temporal docker config is a more secure way, than logging into Docker registry using system-wide default docker config (
DOCKER_CONFIG variable is defined or there is
~/.docker, then werf will copy this already existing config into a new temporal one. So any docker logins already made before running
werf ci-env command will still be active and available using this new temporal config.
After new tmp config was created werf performs additional login into detected Docker registry.
As a result of Docker registry integration procedure
werf ci-env will export
DOCKER_CONFIG variable with new temporal docker config.
This config then will be used by any following werf command and also can be used by
docker login command to perform any additional custom logins on your needs.
Complete list of ci-env params and customizing
As an output of ci-env command werf exports following list of variables. To customize these variables user may predefine any variable with the name prefix
WERF_ prior running
werf ci-env command (ci-env command will detect already defined environment variable and will use this variable as is). This can be accomplished using project environment variables or simply by exporting variables in shell.
Following variables are defined by
werf ci-env command.
Path to new temporal docker config generated by
werf ci-env command.