Importing from images and artifacts | Stapel | Build | Usage

import:
- artifact: <artifact name>
  image: <image name>
  stage: <stage name>
  before: <install  setup>
  after: <install  setup>
  add: <absolute path>
  to: <absolute path>
  owner: <owner>
  group: <group>
  includePaths:
  - <relative path or glob>
  excludePaths:
  - <relative path or glob>

The size of the final image can grow dramatically due to the assembly tools and source files eating up space. These files are generally not needed in the final image. To avoid this, the Docker community suggests installing tools, building, and removing irrelevant files in one step:

RUN “download-source && cmd && cmd2 && remove-source”

You can do the same in werf — just specify the relevant instructions for some user stage. Below is an example of specifying the shell assembly instructions for the install stage (you can do so for the ansible builder as well):
shell:
  install:
  - "download-source"
  - "cmd"
  - "cmd2"
  - "remove-source"

However, this method does not support caching. Thus, a build toolkit will be installed all over again.

Another way is to use a multi-stage build, a feature supported in Docker since version 17.05:

FROM node:latest AS storefront
WORKDIR /app
COPY react-app .
RUN npm install
RUN npm run build

FROM maven:latest AS appserver
WORKDIR /app
COPY . .
RUN mvn package

FROM java:8-jdk-alpine
COPY --from=storefront /app/react-app/build/ /static
COPY --from=appserver /app/target/AtSea-0.0.1-SNAPSHOT.jar /app/AtSea.jar

The point of this approach is to describe several auxiliary images and selectively copy artifacts from one image to another, leaving all the unnecessary data outside the final image.

werf offers the same approach, but using images and artifacts.

Why doesn’t werf use multi-stage assembly?

Historically, imports came much earlier than the Docker multi-stage mechanism, and

werf allows for greater flexibility when working with auxiliary images.

Importing resources from the images and artifacts must be described in the import directive in the destination image in the image or artifact config section. import is an array of records, where each record must contain the following:

image: <image name> or artifact: <artifact name>: source image; the name of the image copy files from.
stage: <stage name>: source image stage; the stage of the source_image to copy files from.
add: <absolute path>: source path; the absolute path to the file or directory in the source image to copy from.
to: <absolute path>: destination path; the absolute path in the destination image. If absent, the destination path defaults to the source path (as specified by the add directive).
before: <install || setup> or after: <install || setup>: destination image stage; the stage to import files. Currently, only install and setup stages are supported.

An example of the import directive:

import:
- artifact: application-assets
  add: /app/public/assets
  to: /var/www/site/assets
  after: install
- image: frontend
  add: /app/assets
  after: setup

As with the git mappings configuration, include and exclude file and directory masks are supported (includePaths: [] and excludePaths: [], respectively). Masks must be specified relative to the source path (as in the add parameter). You can also specify an owner and a group for the imported resources, owner: <owner> and group: <group>. This behavior is similar to the one used when adding code from Git repositories, and you can read more about it in the git directive section.

Note that the path of imported resources and the path specified in git mappings must not overlap.

Refer to this section to learn more about using artifacts.

What is an artifact?

An artifact is a special image used by other images and artifacts to isolate the build process and build tools resources (environments, software, data) from the application image build process. Examples of such resources include software or data used to build the image but not needed to run the application, etc.

Using artifacts, you can build an unlimited number of components. They also help you to solve the following problems:

If an application consists of a set of components, each with their own dependencies, you usually have to rebuild all the components every time. Artifacts allow you to assemble only the components that you really need.
Assembling components in different environments.

Refer to import directive to learn more about importing resources from the artifacts.

Configuration

The configuration of an artifact is similar to that of an image. Each artifact must be described in irs own artifact config section.

The instructions related to the from stage, namely, the instruction to set the base image, mounts, and imports are exactly the same as those for images.

The docker _ instructions stage and the related instructions are not supported for artifacts. An artifact is an assembly tool, and and the only thing it provides is data.

The remaining stages and instructions for defining artifacts are discussed in detail below.

Naming

artifact: string

The artifact images is declared with the artifact directive: artifact: string. In contrast to naming images, when naming artifacts, you do not have to observe the Docker naming convention, since artifacts used purely internally.

artifact: "application assets"

Adding source code from Git repositories

Unlike regular images, the artifact stage conveyor has no gitCache and gitLatestPatch stages.

In werf, there is an optional dependency on changes to Git repositories for artifacts. So, by default, werf ignores any changes to the Git repositories by caching the artifact image after the first build. But you can define file and directory dependencies that will trigger an artifact image rebuild when changed.

Refer to the documentation to learn more about working with git repositories.

Running assembly instructions

Artifacts share the same set of directives and user stages as regular images: beforeInstall, install, beforeSetup, and setup.

If you don’t specify any file dependencies in the stageDependencies directive in the Git section for a user stage, the image will be cached after the first build and will not be reassembled as long as the corresponding stage exists in the stages storage.

To rebuild the artifact whenever there are any changes in the related Git repository, specify stageDependency **/* for any user stage. Below is an example of setting it for the install stage:

git:
- to: /
  stageDependencies:
    install: "**/*"

Refer to the documentation to learn more about working with assembly instructions.

Using artifacts

Unlike the stapel image, the stapel artifact has no git latest patch stage.

This is intentional, because the git latest patch stage usually runs on every commit, applying the changes made to the files. However, it makes sense to use the Stapel artifact as a highly cacheable image that is updated infrequently (e.g., when some special files are changed).

Suppose you want to import data from Git into a Stapel artifact, and only rebuild the assets when the files that affect the assembly of the assets change. This means assets should not be rebuilt if other files in Git are changed.

Of course, sometimes you want to include changes made to any files in the Git repository in your stapel artifact (for example, if a Go app is being built in the artifact). In that case, you have to specify a dependency relative to the stage to build if there are changes in Git using git.stageDependencies and * as a pattern:

git:
- add: /
  to: /app
  stageDependencies:
    setup:
    - "*"

In this case, any changes to the files in the Git repository will result in a rebuild of the artifact image and all stapel images into which that artifact is imported.

NOTE: If you use any files in both the stapel artifact and the stapel image builds, the proper way is to use the git.add directive in each image, that is, multiple times, when adding Git files. We do not recommend adding files to the artifact and then importing them into the image using the import directive.