Deployment stages

Kubernetes resources are deployed in the following stages:

  1. Deploy CustomResourceDefinitions.
  2. Deploy pre-install, pre-upgrade, pre-rollback resources, from low to high weight (werf.io/weight, helm.sh/hook-weight).
  3. Deploy install, upgrade, rollback resources, from low to high weight.
  4. Deploy post-install, post-upgrade, post-rollback resources, from low to high weight.

Resources with the same weight are grouped and deployed concurrently. Default weight is 0. Resources with the werf.io/deploy-dependency-<name> annotation are deployed as soon as their dependencies are satisfied, but within their stage (pre, main or post).

Deploying CustomResourceDefinitions

To deploy CustomResourceDefinitions, put their manifests into crds/*.yaml non-template files in any of the included charts. During deployment, CRDs are always deployed before any other resources.

Example:

# .helm/crds/crontab.yaml:
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
spec:
  names:
    kind: CronTab
# ...

# .helm/templates/crontab.yaml:
apiVersion: example.org/v1
kind: CronTab
# ...

In this case, the CRD for the CronTab resource will be deployed first, followed by the CronTab resource.

Resource ordering

Ordering via weights (werf-only)

The annotation werf.io/weight can be used to set resource ordering during deployment. Resources have weight 0 by default. Resources with a lower weight are deployed before resources with a higher weight. If resources have the same weight, they are deployed in parallel.

Take a look at the example:

# .helm/templates/example.yaml:
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: database
  annotations:
    werf.io/weight: "-1"
# ...
---
apiVersion: batch/v1
kind: Job
metadata:
  name: database-migrations
# ...
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app1
  annotations:
    werf.io/weight: "1"
# ...
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app2
  annotations:
    werf.io/weight: "1"
# ...

In this case, the database resource is deployed first, followed by database-migrations, and then app1 and app2 are deployed in parallel.

Ordering via dependencies (werf only)

The annotation werf.io/deploy-dependency-<name> can be used to set resource ordering during deployment. The resource with such an annotation will be deployed as soon as all its dependencies are satisfied. This annotation has no effect if the resource on which we depend upon is outside the stage (pre, main, post, …) of the resource with the annotation. The resource weight is ignored when this annotation is used.

For example:

# .helm/templates/example.yaml:
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: database
# ...
---
apiVersion: batch/v1
kind: Job
metadata:
  name: database-migrations
  annotations:
    werf.io/deploy-dependency-db: state=ready,kind=StatefulSet,name=database
# ...
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app1
  annotations:
    werf.io/deploy-dependency-migrations: state=ready,kind=Job,name=database-migrations
# ...
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app2
  annotations:
    werf.io/deploy-dependency-migrations: state=ready,kind=Job,name=database-migrations
# ...

In this case, the database resource is deployed first, followed by database-migrations, and then app1 and app2 are deployed in parallel.

Check out all capabilities of this annotation here.

This is a more flexible and effective way to set the order of resource deployments in comparison to werf.io/weight and other methods, as it allows you to deploy resources in a graph-like order.

Deletion ordering via dependencies (werf only)

Annotation werf.io/delete-dependency-<name> can be used to set resource ordering during deletion. The resource with such an annotation will be deleted only after all its dependencies are deleted. This annotation has no effect if the resource on which we depend upon is outside the stage (pre, main, post, …) of the resource with the annotation.

For example:

# .helm/templates/example.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app
# ...
---
apiVersion: v1
kind: Service
metadata:
  name: app
  annotations:
    werf.io/delete-dependency-ingress: state=absent,kind=Ingress,group=networking.k8s.io,version=v1,name=app
# ...
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app
  annotations:
    werf.io/delete-dependency-service: state=absent,kind=Service,version=v1,name=app
# ...

In this case, the Ingress will be deleted first, then the Service, and only then the Deployment.

Check out all capabilities of this annotation here.

Waiting for non-release (external) resources (werf only)

The werf.io/deploy-dependency-<name> and werf.io/delete-dependency-<name> annotations also support dependencies on resources that are not part of the current release — for example, resources created by a third-party operator.

By default (external=auto), if no resource matching the dependency selector is found among the current release resources, werf treats it as external and waits for it in the cluster. You can also explicitly set external=true to always treat a dependency as external regardless of what is in the release.

For example, to wait for a Secret created by a Vault operator before deploying your application:

# .helm/templates/example.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  annotations:
    werf.io/deploy-dependency-secret: state=ready,kind=Secret,version=v1,name=my-dynamic-vault-secret,external=true
# ...

The myapp deployment will start only after my-dynamic-vault-secret exists and is ready in the cluster.

To wait for multiple external resources:

# .helm/templates/example.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  annotations:
    werf.io/deploy-dependency-secret: state=ready,kind=Secret,version=v1,name=my-dynamic-vault-secret,external=true
    werf.io/deploy-dependency-db: state=ready,kind=StatefulSet,group=apps,version=v1,name=my-database,external=true
# ...

The same works for deletion ordering. To delay deletion of a release resource until an external resource is gone:

# .helm/templates/example.yaml:
apiVersion: v1
kind: ConfigMap
metadata:
  name: myapp-config
  annotations:
    werf.io/delete-dependency-lease: state=absent,kind=Lease,group=coordination.k8s.io,version=v1,name=myapp-leader-election,external=true
# ...

The myapp-config ConfigMap will be deleted only after myapp-leader-election is gone from the cluster.

When a dependency is external, name, kind, and version must all be specified. By default, werf looks for the resource in the release namespace; specify namespace to use a different one.