Overview

werf supports importing configuration files from external sources using the includes mechanism.

This is especially useful when you have many similar applications and want to manage and reuse configuration in a centralized way.

What can be imported

The following types of files are supported for import:

  • The main configuration file (werf.yaml), templates (.werf/**/*.tmpl), as well as arbitrary files used during templating.
  • Helm charts and individual files.
  • Dockerfile and .dockerignore.

What cannot be imported

The mechanism does not allow importing files for the build context, as well as the following configuration files:

  • werf-includes.yaml
  • werf-includes.lock
  • werf-giterminism.yaml

How the includes works

  1. You define external sources and the files to import in werf-includes.yaml.
  2. You lock the versions of the external sources using the werf includes update command, which creates the werf-includes.lock file. The lock file is required for reproducible builds and deployments.
  3. All werf commands (e.g., werf build, werf converge) will respect the lock file and apply the configuration according to the overlay rules.

Overlay rules

If the same file exists in multiple places, the following rules apply:

  1. Local project files always take precedence over imported files.
  2. If a file exists in multiple includes, the version from the source listed lower in the includes list in werf-includes.yaml (from general to specific) is used.

Example of applying overlay rules

For instance, suppose the local repository contains file c, and the following includes configuration is used:

# werf-includes.yaml
includes:
  - git: repo1 # imports a, b, c
    branch: main
    add: /
    to: /
  - git: repo2 # imports b, c
    branch: main
    add: /
    to: /

Then the resulting merged file structure will look like this:

.
├── a      # from repo1
├── b      # from repo2
├── c      # from local repo

Connecting configuration from external sources

Below is an example configuration of external sources (a full description of directives can be found on the corresponding page werf-includes.yaml):

# werf-includes.yaml
includes:
  - git: https://github.com/werf/werf
    branch: main
    add: /docs/examples/includes/common
    to: /
    includePaths:
      - .werf

After the configuration, it is necessary to lock the versions of external dependencies. You can do this using the command werf includes update, which will create the file werf-includes.lock in the project root:

includes:
  - git: https://github.com/werf/werf
    branch: main
    commit: 21640b8e619ba4dd480fedf144f7424aa217a2eb

IMPORTANT. According to giterminism policies, the files werf-includes.yaml and werf-includes.lock must be committed. During configuration and debugging, for convenience, it is recommended to use the --dev flag.

Example of using external sources for configuring similar applications

Suppose you decided to centrally manage the configuration of applications in your organization, keeping common configuration in one source and per-project-type configuration in others (in one or several Git repositories — at your discretion).

The configuration of a typical project in this case might look like this:

  • Project:
.helm
backend
frontend
werf-includes.lock
werf-includes.yaml
werf.yaml
backend:
  limits:
    cpu: 100m
    memory: 256Mi

frontend:
  limits:
    cpu: 50m
    memory: 48Mi


// dummy


const express = require('express');
const app = express();

app.get('/api/hello', (req, res) => {
  res.json({ message: 'Hello from backend!' });
});

const port = process.env.PORT || 3000;
app.listen(port, () => {
  console.log(`Backend running on port ${port}`);
});


{
  "name": "backend",
  "version": "1.0.0",
  "main": "index.js",
  "scripts": {
    "start": "node index.js"
  },
  "dependencies": {
    "express": "^4.18.2"
  }
}


<!DOCTYPE html>
<html>
<head><title>Frontend</title></head>
<body>
  <h1>Hello from Frontend</h1>
  <button onclick="callApi()">Call Backend</button>
  <p id="output"></p>

  <script>
    async function callApi() {
      const res = await fetch('$BACKEND_URL');
      const data = await res.json();
      document.getElementById('output').innerText = data.message;
    }
  </script>
</body>
</html>


includes:
    - git: https://github.com/werf/werf
      branch: main
      commit: 792c7631d2abbae8dc0acb252905f740d98c7585


includes:
  - git: https://github.com/werf/werf
    branch: main
    add: /docs/examples/includes/werf-common
    to: /
  - git: https://github.com/werf/werf
    branch: main
    add: /docs/examples/includes/common_js
    to: /


project: myapp
configVersion: 1
build:
  platform:
    - linux/amd64
  imageSpec: {{ include "imagespec" (dict "appName" "myapp") | nindent 4 }}
cleanup: {{ include "cleanup" (dict "mainBranchName" "main") | nindent 2 }}
---
{{ include "image-backend" (dict) }}
---
{{ include "image-frontend" (dict "backendUrl" "http://backend:3000") }}
  • Source with common configuration:
.werf 
{{- define "cleanup" }}
{{- $mainBranchName := default "main" .mainBranchName}}
keepImagesBuiltWithinLastNHours: 1
keepPolicies:
  - references:
      branch: /.*/
      limit:
        last: 20
        in: 168h
        operator: And
    imagesPerReference:
      last: 1
      in: 168h
      operator: And
  - references:
      branch: "{{ $mainBranchName }}"
    imagesPerReference:
      last: 1
{{- end }}


{{- define "imagespec" }}
{{- $appName := default "defaultapp" .appName}}
clearHistory: true
config:
  labels:
    example.io/app: "{{ $appName }}"
{{- end }}
  • Source with configuration specific to your project type:
.helm
.werf
.dockerignore
backend.Dockerfile
frontend.Dockerfile
**/dummy.ini

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
        - name: backend
          image: {{ $.Values.werf.image.backend }}
          ports:
            - containerPort: 3000
          resources:
            requests:
              cpu: {{ $.Values.backend.limits.cpu }}
              memory: {{ $.Values.backend.limits.memory}}
            limits:
              cpu: {{ $.Values.backend.limits.cpu }}
              memory: {{ $.Values.backend.limits.memory }}
        - name: frontend
          image: {{ $.Values.werf.image.frontend }}
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: {{ $.Values.frontend.limits.cpu }}
              memory: {{ $.Values.frontend.limits.memory}}
            limits:
              cpu: {{ $.Values.frontend.limits.cpu }}
              memory: {{ $.Values.frontend.limits.memory }}


apiVersion: v1
kind: Service
metadata:
  name: myapp
spec:
  selector:
    app: myapp
  ports:
    - name: http
      port: 80


backend:
  limits:
    cpu: 100m
    memory: 256Mi

frontend:
  limits:
    cpu: 50m
    memory: 48Mi


{{- define "image-backend" }}
image: backend
dockerfile: backend.Dockerfile
{{- end }}

{{- define "image-frontend" }}
{{- $backendUrl := default "http://example.org" .backendUrl }}
image: frontend
dockerfile: frontend.Dockerfile
args:
  BACKEND_URL: "{{ $backendUrl }}"
{{- end }}


FROM node:18-alpine
WORKDIR /app
COPY backend/ /app/
RUN npm install
COPY . .
EXPOSE 3000
CMD ["npm", "start"]


FROM nginx:alpine
ARG BACKEND_URL
ENV BACKEND_URL=${BACKEND_URL}
WORKDIR /usr/share/nginx/html
COPY frontend/index.html .
RUN envsubst < index.html > index.tmp.html && \
    mv index.tmp.html index.html

Updating includes

Deterministic

There are two ways to update include versions:

  • Use the werf includes update command. This will update all includes to the HEAD of the specified reference (branch or tag).
  • Edit the werf-includes.lock file manually or use dependency management tools like Dependabot, Renovate, etc.

If you need to use the latest HEAD versions without a lock file you can use --allow-includes-update option. The usage of this option must be enabled in werf-giterminism.yaml:

includes:
  allowIncludesUpdate: true

IMPORTANT. We do not recommend using this approach, as it may break reproducibility of builds and deployments.

Debugging

The includes mechanism has built-in commands for debugging:

Listing project files

werf includes ls-files .helm

Example output:

PATH                                             SOURCE
.helm/Chart.yaml                                 https://github.com/werf/werf
.helm/charts/backend/Chart.yaml                  https://github.com/werf/werf
.helm/charts/backend/templates/deployment.yaml   https://github.com/werf/werf
.helm/charts/backend/templates/service.yaml      https://github.com/werf/werf
.helm/charts/frontend/Chart.yaml                 https://github.com/werf/werf
.helm/charts/frontend/templates/deployment.yaml  https://github.com/werf/werf
.helm/charts/frontend/templates/service.yaml     https://github.com/werf/werf
.helm/requirements.lock                          https://github.com/werf/werf
.helm/values.yaml                                local

Retrieving file content

werf includes get-file .helm/charts/backend/templates/deployment.yaml

Example output:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: 
  annotations:
    werf.io/weight: "30"
spec:
  selector:
    matchLabels:
      app: 
  template:
    metadata:
      labels:
        app: 
    spec:
      containers:
        - name: backend
          image: 
          ports:
            - containerPort: 3000
          resources:
            requests:
              cpu: 
              memory: 
            limits:
              cpu: 
              memory:

Using local repositories

You can also use local repositories as include sources. The workflow is the same as with remote repositories.

# werf-includes.yaml
includes:
  - git: /local/repo
    branch: main
    add: local-dev
    to: /
    includePaths:
      - /.helm