Prepare the infrastructure

Requirements

Installing GitLab Runner

Install GitLab Runner on its dedicated host by following the official instructions.

Setting up the build environment with Buildah

(For Ubuntu 23.10 and later) on the GitLab Runner host run:

{ echo "kernel.apparmor_restrict_unprivileged_userns = 0" && echo "kernel.apparmor_restrict_unprivileged_unconfined = 0";} | sudo tee -a /etc/sysctl.d/20-apparmor-donotrestrict.conf && sudo sysctl -p /etc/sysctl.d/20-apparmor-donotrestrict.conf

Registering GitLab Runner

Follow the official instructions to register GitLab Runner in GitLab; specify Docker as the executor and any image as the image (e.g. alpine).

Configuring GitLab Runner

On the GitLab Runner host, open its config.toml configuration file and add the following options to the GitLab Runner you registered earlier:

[[runners]]
  name = "<name of the Runner you registered>"
  [runners.docker]
    security_opt = ["seccomp:unconfined", "apparmor:unconfined"]
    volumes = ["/home/build/.werf"]

If the GitLab Runner host runs Linux kernel version 5.12 or lower, install fuse on the host and add the following extra option to the config.toml file:

[[runners]]
  name = "<name of the Runner you registered>"
  [runners.docker]
    devices = ["/dev/fuse"]

If needed, perform additional configuration of the GitLab Runner.

Configuring the container registry

Enable garbage collection for your container registry.

Preparing the system for cross-platform building (optional)

This step is only needed to build images for platforms other than the host platform on which werf is being run.

Register emulators on your system using qemu-user-static:

docker run --restart=always --name=qemu-user-static -d --privileged --entrypoint=/bin/sh multiarch/qemu-user-static -c "/register --reset -p yes && tail -f /dev/null"

Installing Argo CD Image Updater

Install Argo CD Image Updater with the “continuous deployment of OCI Helm chart type application” patch:

kubectl apply -n argocd -f https://raw.githubusercontent.com/werf/3p-argocd-image-updater/master/manifests/install.yaml

Configure the project

Configuring Argo CD Application

  1. Apply the following Application CRD to the target cluster to deploy a bundle from the container registry:
kubectl create -f - <<EOF
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  annotations:
    argocd-image-updater.argoproj.io/chart-version: ~ 1.0
    argocd-image-updater.argoproj.io/pull-secret: pullsecret:myproject-production/myproject-regcred
  name: myproject
  namespace: argocd
  finalizers:
  - resources-finalizer.argocd.argoproj.io
spec:
  destination:
    namespace: myproject-production
    server: https://kubernetes.default.svc
  project: default
  source:
    chart: myproject
    repoURL: registry.mycompany.org/myproject
    targetRevision: 1.0.0
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
EOF

The value of argocd-image-updater.argoproj.io/chart-version="~ 1.0" means that the operator must automatically deploy the chart updated to the latest patch version in the SEMVER range 1.0.*.

  1. Create a pull secret to access the project container registry:
kubectl create -f - <<EOF
---
apiVersion: v1
kind: Secret
metadata:
  name: myproject-regcred
  namespace: myproject-production
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: BASE64_DOCKER_CONFIG_JSON
EOF

Configuring a GitLab project

  • Create and save the access token for cleaning up the no longer needed images from the container registry; use the following parameters:

    • Token name: werf-images-cleanup;

    • Role: developer;

    • Scopes: api.

  • Add the following variables to the project variables:

    • Access token to clean up the no longer needed images:

      • Key: WERF_IMAGES_CLEANUP_PASSWORD;

      • Value: <"werf-images-cleanup" access token you saved earlier>;

      • Protect variable: yes;

      • Mask variable: yes.

  • Add a scheduled nightly job to clean up the no longer needed images in the container registry and set the main/master branch as the Target branch.

Configuring CI/CD of the project

This is what the repository that uses werf for building and deploying might look like:

.helm
app
.gitlab-ci.yml
werf.yaml
stages:
  - release
  - cleanup

default:
  image:
    name: registry.werf.io/werf/werf:2-stable
    pull_policy: always
  before_script:
    - source $(werf ci-env gitlab --as-file)
  tags: ["<GitLab Runner tag>"]

Build and publish release:
  stage: release
  script:
    - werf bundle publish --tag "1.0.${CI_PIPELINE_ID}"
  only:
    - main
  except:
    - schedules

Cleanup registry:
  stage: cleanup
  script:
    - werf cr login $WERF_REPO
    - werf cleanup
  only:
    - schedules


apiVersion: apps/v1
kind: Deployment
metadata:
  name: app
spec:
  selector:
    matchLabels:
      app: app
  template:
    metadata:
      labels:
        app: app
    spec:
      containers:
      - name: app
        image: {{ .Values.werf.image.app }}


apiVersion: v1
kind: Service
metadata:
  name: app
spec:
  selector:
    app: app
  ports:
  - name: app
    port: 80


FROM node

WORKDIR /app
COPY . .
RUN npm ci

CMD ["node", "server.js"]


{
  "name": "app",
  "version": "1.0.0",
  "lockfileVersion": 2,
  "requires": true,
  "packages": {
    "": {
      "name": "app",
      "version": "1.0.0"
    }
  }
}


{
  "name": "app",
  "version": "1.0.0",
  "main": "server.js",
  "scripts": {
    "start": "node server.js"
  }
}


const http = require('http');

const hostname = '127.0.0.1';
const port = 80;

const server = http.createServer((req, res) => {
  res.statusCode = 200;
  res.setHeader('Content-Type', 'text/plain');
  res.end('Hello World');
});

server.listen(port, hostname, () => {
  console.log(`Server running at http://${hostname}:${port}/`);
});


configVersion: 1
project: myproject
---
image: app
dockerfile: Dockerfile
context: ./app

Extras:

  • Add authorization options for werf cleanup in the container registry by following the instructions.