Install werf
for by
Install dependencies
Docker
Manage Docker as a non-root user. Create the docker group and add your user to the group:
sudo groupadd docker
sudo usermod -aG docker $USER
Git command line utility
- Minimum required version is 2.18.0.
Install werf
Installing multiwerf
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.2 alpha --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.2 alpha --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.2 alpha --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.2 beta --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.2 beta --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.2 beta --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.2 ea --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.2 ea --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.2 ea --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.2 stable --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.2 stable --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.2 stable --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.2 rock-solid --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.2 rock-solid --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.2 rock-solid --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.1 alpha --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.1 alpha --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.1 alpha --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.1 beta --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.1 beta --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.1 beta --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.1 ea --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.1 ea --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.1 ea --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.1 stable --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.1 stable --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.1 stable --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.1 rock-solid --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.1 rock-solid --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.1 rock-solid --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.2 alpha --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.2 alpha --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.2 alpha --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.2 beta --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.2 beta --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.2 beta --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.2 ea --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.2 ea --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.2 ea --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.2 stable --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.2 stable --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.2 stable --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.2 rock-solid --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.2 rock-solid --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.2 rock-solid --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.1 alpha --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.1 alpha --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.1 alpha --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.1 beta --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.1 beta --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.1 beta --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.1 ea --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.1 ea --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.1 ea --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.1 stable --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.1 stable --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.1 stable --as-file)
# We can use werf now
werf ...
# add ~/bin into PATH
export PATH=$PATH:$HOME/bin
echo 'export PATH=$PATH:$HOME/bin' >> ~/.bashrc
# install multiwerf into ~/bin directory
mkdir -p ~/bin
cd ~/bin
curl -L https://raw.githubusercontent.com/werf/multiwerf/master/get.sh | bash
Using werf in the current shell
This will create werf shell function which calls to the werf binary which multiwerf has been prepared for your session:
source $(multiwerf use 1.1 rock-solid --as-file)
werf version
...
Optional: activate werf on terminal startup
echo '. $(multiwerf use 1.1 rock-solid --as-file)' >> ~/.bashrc
How to use in the CI/CD system?
First ensure that multiwerf exists and is executable, use the type command. The command prints a message to stderr if multiwerf is not found. Thus, diagnostics in a CI/CD environment becomes simpler. Second enable werf in the CI/CD shell.
type multiwerf && . $(multiwerf use 1.1 rock-solid --as-file)
# We can use werf now
werf ...
PowerShell
$MULTIWERF_BIN_PATH = "C:\ProgramData\multiwerf\bin"
mkdir $MULTIWERF_BIN_PATH
Invoke-WebRequest -Uri https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe -OutFile $MULTIWERF_BIN_PATH\multiwerf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$MULTIWERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
Using werf in the current shell
Invoke-Expression -Command "multiwerf use 1.2 alpha --as-file --shell powershell" | Out-String -OutVariable WERF_USE_SCRIPT_PATH
. $WERF_USE_SCRIPT_PATH.Trim()
cmd.exe
Run cmd.exe as Administrator and then do the following:
set MULTIWERF_BIN_PATH="C:\ProgramData\multiwerf\bin"
mkdir %MULTIWERF_BIN_PATH%
bitsadmin.exe /transfer "multiwerf" https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe %MULTIWERF_BIN_PATH%\multiwerf.exe
setx /M PATH "%PATH%;%MULTIWERF_BIN_PATH%"
Next it is required to open new cmd.exe session to start using werf.
Using werf in the current shell
FOR /F "tokens=*" %g IN ('multiwerf use 1.2 alpha --as-file --shell cmdexe') do (SET WERF_USE_SCRIPT_PATH=%g)
%WERF_USE_SCRIPT_PATH%
PowerShell
$MULTIWERF_BIN_PATH = "C:\ProgramData\multiwerf\bin"
mkdir $MULTIWERF_BIN_PATH
Invoke-WebRequest -Uri https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe -OutFile $MULTIWERF_BIN_PATH\multiwerf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$MULTIWERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
Using werf in the current shell
Invoke-Expression -Command "multiwerf use 1.2 beta --as-file --shell powershell" | Out-String -OutVariable WERF_USE_SCRIPT_PATH
. $WERF_USE_SCRIPT_PATH.Trim()
cmd.exe
Run cmd.exe as Administrator and then do the following:
set MULTIWERF_BIN_PATH="C:\ProgramData\multiwerf\bin"
mkdir %MULTIWERF_BIN_PATH%
bitsadmin.exe /transfer "multiwerf" https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe %MULTIWERF_BIN_PATH%\multiwerf.exe
setx /M PATH "%PATH%;%MULTIWERF_BIN_PATH%"
Next it is required to open new cmd.exe session to start using werf.
Using werf in the current shell
FOR /F "tokens=*" %g IN ('multiwerf use 1.2 beta --as-file --shell cmdexe') do (SET WERF_USE_SCRIPT_PATH=%g)
%WERF_USE_SCRIPT_PATH%
PowerShell
$MULTIWERF_BIN_PATH = "C:\ProgramData\multiwerf\bin"
mkdir $MULTIWERF_BIN_PATH
Invoke-WebRequest -Uri https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe -OutFile $MULTIWERF_BIN_PATH\multiwerf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$MULTIWERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
Using werf in the current shell
Invoke-Expression -Command "multiwerf use 1.2 ea --as-file --shell powershell" | Out-String -OutVariable WERF_USE_SCRIPT_PATH
. $WERF_USE_SCRIPT_PATH.Trim()
cmd.exe
Run cmd.exe as Administrator and then do the following:
set MULTIWERF_BIN_PATH="C:\ProgramData\multiwerf\bin"
mkdir %MULTIWERF_BIN_PATH%
bitsadmin.exe /transfer "multiwerf" https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe %MULTIWERF_BIN_PATH%\multiwerf.exe
setx /M PATH "%PATH%;%MULTIWERF_BIN_PATH%"
Next it is required to open new cmd.exe session to start using werf.
Using werf in the current shell
FOR /F "tokens=*" %g IN ('multiwerf use 1.2 ea --as-file --shell cmdexe') do (SET WERF_USE_SCRIPT_PATH=%g)
%WERF_USE_SCRIPT_PATH%
PowerShell
$MULTIWERF_BIN_PATH = "C:\ProgramData\multiwerf\bin"
mkdir $MULTIWERF_BIN_PATH
Invoke-WebRequest -Uri https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe -OutFile $MULTIWERF_BIN_PATH\multiwerf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$MULTIWERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
Using werf in the current shell
Invoke-Expression -Command "multiwerf use 1.2 stable --as-file --shell powershell" | Out-String -OutVariable WERF_USE_SCRIPT_PATH
. $WERF_USE_SCRIPT_PATH.Trim()
cmd.exe
Run cmd.exe as Administrator and then do the following:
set MULTIWERF_BIN_PATH="C:\ProgramData\multiwerf\bin"
mkdir %MULTIWERF_BIN_PATH%
bitsadmin.exe /transfer "multiwerf" https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe %MULTIWERF_BIN_PATH%\multiwerf.exe
setx /M PATH "%PATH%;%MULTIWERF_BIN_PATH%"
Next it is required to open new cmd.exe session to start using werf.
Using werf in the current shell
FOR /F "tokens=*" %g IN ('multiwerf use 1.2 stable --as-file --shell cmdexe') do (SET WERF_USE_SCRIPT_PATH=%g)
%WERF_USE_SCRIPT_PATH%
PowerShell
$MULTIWERF_BIN_PATH = "C:\ProgramData\multiwerf\bin"
mkdir $MULTIWERF_BIN_PATH
Invoke-WebRequest -Uri https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe -OutFile $MULTIWERF_BIN_PATH\multiwerf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$MULTIWERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
Using werf in the current shell
Invoke-Expression -Command "multiwerf use 1.2 rock-solid --as-file --shell powershell" | Out-String -OutVariable WERF_USE_SCRIPT_PATH
. $WERF_USE_SCRIPT_PATH.Trim()
cmd.exe
Run cmd.exe as Administrator and then do the following:
set MULTIWERF_BIN_PATH="C:\ProgramData\multiwerf\bin"
mkdir %MULTIWERF_BIN_PATH%
bitsadmin.exe /transfer "multiwerf" https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe %MULTIWERF_BIN_PATH%\multiwerf.exe
setx /M PATH "%PATH%;%MULTIWERF_BIN_PATH%"
Next it is required to open new cmd.exe session to start using werf.
Using werf in the current shell
FOR /F "tokens=*" %g IN ('multiwerf use 1.2 rock-solid --as-file --shell cmdexe') do (SET WERF_USE_SCRIPT_PATH=%g)
%WERF_USE_SCRIPT_PATH%
PowerShell
$MULTIWERF_BIN_PATH = "C:\ProgramData\multiwerf\bin"
mkdir $MULTIWERF_BIN_PATH
Invoke-WebRequest -Uri https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe -OutFile $MULTIWERF_BIN_PATH\multiwerf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$MULTIWERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
Using werf in the current shell
Invoke-Expression -Command "multiwerf use 1.1 alpha --as-file --shell powershell" | Out-String -OutVariable WERF_USE_SCRIPT_PATH
. $WERF_USE_SCRIPT_PATH.Trim()
cmd.exe
Run cmd.exe as Administrator and then do the following:
set MULTIWERF_BIN_PATH="C:\ProgramData\multiwerf\bin"
mkdir %MULTIWERF_BIN_PATH%
bitsadmin.exe /transfer "multiwerf" https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe %MULTIWERF_BIN_PATH%\multiwerf.exe
setx /M PATH "%PATH%;%MULTIWERF_BIN_PATH%"
Next it is required to open new cmd.exe session to start using werf.
Using werf in the current shell
FOR /F "tokens=*" %g IN ('multiwerf use 1.1 alpha --as-file --shell cmdexe') do (SET WERF_USE_SCRIPT_PATH=%g)
%WERF_USE_SCRIPT_PATH%
PowerShell
$MULTIWERF_BIN_PATH = "C:\ProgramData\multiwerf\bin"
mkdir $MULTIWERF_BIN_PATH
Invoke-WebRequest -Uri https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe -OutFile $MULTIWERF_BIN_PATH\multiwerf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$MULTIWERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
Using werf in the current shell
Invoke-Expression -Command "multiwerf use 1.1 beta --as-file --shell powershell" | Out-String -OutVariable WERF_USE_SCRIPT_PATH
. $WERF_USE_SCRIPT_PATH.Trim()
cmd.exe
Run cmd.exe as Administrator and then do the following:
set MULTIWERF_BIN_PATH="C:\ProgramData\multiwerf\bin"
mkdir %MULTIWERF_BIN_PATH%
bitsadmin.exe /transfer "multiwerf" https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe %MULTIWERF_BIN_PATH%\multiwerf.exe
setx /M PATH "%PATH%;%MULTIWERF_BIN_PATH%"
Next it is required to open new cmd.exe session to start using werf.
Using werf in the current shell
FOR /F "tokens=*" %g IN ('multiwerf use 1.1 beta --as-file --shell cmdexe') do (SET WERF_USE_SCRIPT_PATH=%g)
%WERF_USE_SCRIPT_PATH%
PowerShell
$MULTIWERF_BIN_PATH = "C:\ProgramData\multiwerf\bin"
mkdir $MULTIWERF_BIN_PATH
Invoke-WebRequest -Uri https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe -OutFile $MULTIWERF_BIN_PATH\multiwerf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$MULTIWERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
Using werf in the current shell
Invoke-Expression -Command "multiwerf use 1.1 ea --as-file --shell powershell" | Out-String -OutVariable WERF_USE_SCRIPT_PATH
. $WERF_USE_SCRIPT_PATH.Trim()
cmd.exe
Run cmd.exe as Administrator and then do the following:
set MULTIWERF_BIN_PATH="C:\ProgramData\multiwerf\bin"
mkdir %MULTIWERF_BIN_PATH%
bitsadmin.exe /transfer "multiwerf" https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe %MULTIWERF_BIN_PATH%\multiwerf.exe
setx /M PATH "%PATH%;%MULTIWERF_BIN_PATH%"
Next it is required to open new cmd.exe session to start using werf.
Using werf in the current shell
FOR /F "tokens=*" %g IN ('multiwerf use 1.1 ea --as-file --shell cmdexe') do (SET WERF_USE_SCRIPT_PATH=%g)
%WERF_USE_SCRIPT_PATH%
PowerShell
$MULTIWERF_BIN_PATH = "C:\ProgramData\multiwerf\bin"
mkdir $MULTIWERF_BIN_PATH
Invoke-WebRequest -Uri https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe -OutFile $MULTIWERF_BIN_PATH\multiwerf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$MULTIWERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
Using werf in the current shell
Invoke-Expression -Command "multiwerf use 1.1 stable --as-file --shell powershell" | Out-String -OutVariable WERF_USE_SCRIPT_PATH
. $WERF_USE_SCRIPT_PATH.Trim()
cmd.exe
Run cmd.exe as Administrator and then do the following:
set MULTIWERF_BIN_PATH="C:\ProgramData\multiwerf\bin"
mkdir %MULTIWERF_BIN_PATH%
bitsadmin.exe /transfer "multiwerf" https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe %MULTIWERF_BIN_PATH%\multiwerf.exe
setx /M PATH "%PATH%;%MULTIWERF_BIN_PATH%"
Next it is required to open new cmd.exe session to start using werf.
Using werf in the current shell
FOR /F "tokens=*" %g IN ('multiwerf use 1.1 stable --as-file --shell cmdexe') do (SET WERF_USE_SCRIPT_PATH=%g)
%WERF_USE_SCRIPT_PATH%
PowerShell
$MULTIWERF_BIN_PATH = "C:\ProgramData\multiwerf\bin"
mkdir $MULTIWERF_BIN_PATH
Invoke-WebRequest -Uri https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe -OutFile $MULTIWERF_BIN_PATH\multiwerf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$MULTIWERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
Using werf in the current shell
Invoke-Expression -Command "multiwerf use 1.1 rock-solid --as-file --shell powershell" | Out-String -OutVariable WERF_USE_SCRIPT_PATH
. $WERF_USE_SCRIPT_PATH.Trim()
cmd.exe
Run cmd.exe as Administrator and then do the following:
set MULTIWERF_BIN_PATH="C:\ProgramData\multiwerf\bin"
mkdir %MULTIWERF_BIN_PATH%
bitsadmin.exe /transfer "multiwerf" https://flant.bintray.com/multiwerf/v1.3.0/multiwerf-windows-amd64-v1.3.0.exe %MULTIWERF_BIN_PATH%\multiwerf.exe
setx /M PATH "%PATH%;%MULTIWERF_BIN_PATH%"
Next it is required to open new cmd.exe session to start using werf.
Using werf in the current shell
FOR /F "tokens=*" %g IN ('multiwerf use 1.1 rock-solid --as-file --shell cmdexe') do (SET WERF_USE_SCRIPT_PATH=%g)
%WERF_USE_SCRIPT_PATH%
How it works?
The command multiwerf use MAJOR.MINOR CHANNEL allows using the actual werf binary and to be always up-to-date.
You can use this command both in CI/CD system and on the local machine.
The command returns a script or a path to the script file (when used with an --as-file option) that must be used as an argument to the source statement in the case of using shell. As a result, the current version of werf will be available during the shell session.
The script can be divided into two logic parts: updating and creating werf alias or the definition of the function depending on shell type. The update part performs multiwerf self-update and gets the actual werf binary for the specified MAJOR.MINOR version and CHANNEL (read more about werf versioning in the Backward Compatibility Promise section). The update is performed by the multiwerf update command.
If the script is launched for the first time or there is no suitable werf binary found locally, this script will wait until multiwerf downloads new werf binary. Otherwise, the update runs in the background, and werf alias or a function binds to the existing werf binary based on local channel mapping.
if multiwerf werf-path MAJOR.MINOR CHANNEL >~/.multiwerf/multiwerf_use_first_werf_path.log 2>&1; then
multiwerf update MAJOR.MINOR CHANNEL --in-background --output-file=~/.multiwerf/multiwerf_use_background_update.log --with-cache
else
multiwerf update MAJOR.MINOR CHANNEL
fi
WERF_PATH=$(multiwerf werf-path MAJOR.MINOR CHANNEL)
WERF_FUNC=$(cat <<EOF
werf()
{
$WERF_PATH "\$@"
}
EOF
)
eval "$WERF_FUNC"
if ((Invoke-Expression -Command "multiwerf werf-path MAJOR.MINOR CHANNEL" | Out-String -OutVariable WERF_PATH) -and ($LastExitCode -eq 0)) {
multiwerf update MAJOR.MINOR CHANNEL --in-background --output-file=~\.multiwerf\multiwerf_use_background_update.log --with-cache
} else {
multiwerf update MAJOR.MINOR CHANNEL
Invoke-Expression -Command "multiwerf werf-path MAJOR.MINOR CHANNEL" | Out-String -OutVariable WERF_PATH
}
function werf { & $WERF_PATH.Trim() $args }
FOR /F "tokens=*" %%g IN ('multiwerf werf-path MAJOR.MINOR CHANNEL') do (SET WERF_PATH=%%g)
IF %ERRORLEVEL% NEQ 0 (
multiwerf update MAJOR.MINOR CHANNEL
FOR /F "tokens=*" %%g IN ('multiwerf werf-path MAJOR.MINOR CHANNEL') do (SET WERF_PATH=%%g)
) ELSE (
multiwerf update MAJOR.MINOR CHANNEL --in-background --output-file=~\.multiwerf\multiwerf_use_background_update.log --with-cache
)
DOSKEY werf=%WERF_PATH% $*
During the update, multiwerf tries to download the desirable werf version based on a channel mapping.
The channel mapping is a special file that keeps relations between channels and werf versions.
By default, multiwerf uses the mapping file which is maintained in the werf repository: https://github.com/werf/werf/blob/multiwerf/multiwerf.json. You can always use --channel-mapping-url option and specify arbitrary url with channel mapping which you can control by yourself.
Such an approach allows a user not to worry about updates and use the same werf binary version on CI and the local machine. We create new releases with fixes and features and manage channels while you simply use a single command everywhere.
The latest release can be found at this page
curl -L https://dl.bintray.com/flant/werf/v1.2.10+fix23/werf-linux-amd64-v1.2.10+fix23 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.2.10+fix22/werf-linux-amd64-v1.2.10+fix22 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.2.10+fix22/werf-linux-amd64-v1.2.10+fix22 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf//werf-linux-amd64- -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf//werf-linux-amd64- -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.1.23+fix37/werf-linux-amd64-v1.1.23+fix37 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.1.23+fix37/werf-linux-amd64-v1.1.23+fix37 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.1.23+fix37/werf-linux-amd64-v1.1.23+fix37 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.1.23+fix36/werf-linux-amd64-v1.1.23+fix36 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.1.22+fix37/werf-linux-amd64-v1.1.22+fix37 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.2.10+fix23/werf-darwin-amd64-v1.2.10+fix23 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.2.10+fix22/werf-darwin-amd64-v1.2.10+fix22 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.2.10+fix22/werf-darwin-amd64-v1.2.10+fix22 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf//werf-darwin-amd64- -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf//werf-darwin-amd64- -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.1.23+fix37/werf-darwin-amd64-v1.1.23+fix37 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.1.23+fix37/werf-darwin-amd64-v1.1.23+fix37 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.1.23+fix37/werf-darwin-amd64-v1.1.23+fix37 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.1.23+fix36/werf-darwin-amd64-v1.1.23+fix36 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
curl -L https://dl.bintray.com/flant/werf/v1.1.22+fix37/werf-darwin-amd64-v1.1.22+fix37 -o /tmp/werf
chmod +x /tmp/werf
sudo mv /tmp/werf /usr/local/bin/werf
PowerShell
$WERF_BIN_PATH = "C:\ProgramData\werf\bin"
mkdir $WERF_BIN_PATH
Invoke-WebRequest -Uri https://dl.bintray.com/flant/werf/v1.2.10+fix23/werf-windows-amd64-v1.2.10+fix23.exe -OutFile $WERF_BIN_PATH\werf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$WERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
cmd.exe (run as Administrator)
set WERF_BIN_PATH="C:\ProgramData\werf\bin"
mkdir %WERF_BIN_PATH%
bitsadmin.exe /transfer "werf" https://dl.bintray.com/flant/werf/v1.2.10+fix23/werf-windows-amd64-v1.2.10+fix23.exe %WERF_BIN_PATH%\werf.exe
setx /M PATH "%PATH%;%WERF_BIN_PATH%"
PowerShell
$WERF_BIN_PATH = "C:\ProgramData\werf\bin"
mkdir $WERF_BIN_PATH
Invoke-WebRequest -Uri https://dl.bintray.com/flant/werf/v1.2.10+fix22/werf-windows-amd64-v1.2.10+fix22.exe -OutFile $WERF_BIN_PATH\werf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$WERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
cmd.exe (run as Administrator)
set WERF_BIN_PATH="C:\ProgramData\werf\bin"
mkdir %WERF_BIN_PATH%
bitsadmin.exe /transfer "werf" https://dl.bintray.com/flant/werf/v1.2.10+fix22/werf-windows-amd64-v1.2.10+fix22.exe %WERF_BIN_PATH%\werf.exe
setx /M PATH "%PATH%;%WERF_BIN_PATH%"
PowerShell
$WERF_BIN_PATH = "C:\ProgramData\werf\bin"
mkdir $WERF_BIN_PATH
Invoke-WebRequest -Uri https://dl.bintray.com/flant/werf/v1.2.10+fix22/werf-windows-amd64-v1.2.10+fix22.exe -OutFile $WERF_BIN_PATH\werf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$WERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
cmd.exe (run as Administrator)
set WERF_BIN_PATH="C:\ProgramData\werf\bin"
mkdir %WERF_BIN_PATH%
bitsadmin.exe /transfer "werf" https://dl.bintray.com/flant/werf/v1.2.10+fix22/werf-windows-amd64-v1.2.10+fix22.exe %WERF_BIN_PATH%\werf.exe
setx /M PATH "%PATH%;%WERF_BIN_PATH%"
PowerShell
$WERF_BIN_PATH = "C:\ProgramData\werf\bin"
mkdir $WERF_BIN_PATH
Invoke-WebRequest -Uri https://dl.bintray.com/flant/werf//werf-windows-amd64-.exe -OutFile $WERF_BIN_PATH\werf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$WERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
cmd.exe (run as Administrator)
set WERF_BIN_PATH="C:\ProgramData\werf\bin"
mkdir %WERF_BIN_PATH%
bitsadmin.exe /transfer "werf" https://dl.bintray.com/flant/werf//werf-windows-amd64-.exe %WERF_BIN_PATH%\werf.exe
setx /M PATH "%PATH%;%WERF_BIN_PATH%"
PowerShell
$WERF_BIN_PATH = "C:\ProgramData\werf\bin"
mkdir $WERF_BIN_PATH
Invoke-WebRequest -Uri https://dl.bintray.com/flant/werf//werf-windows-amd64-.exe -OutFile $WERF_BIN_PATH\werf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$WERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
cmd.exe (run as Administrator)
set WERF_BIN_PATH="C:\ProgramData\werf\bin"
mkdir %WERF_BIN_PATH%
bitsadmin.exe /transfer "werf" https://dl.bintray.com/flant/werf//werf-windows-amd64-.exe %WERF_BIN_PATH%\werf.exe
setx /M PATH "%PATH%;%WERF_BIN_PATH%"
PowerShell
$WERF_BIN_PATH = "C:\ProgramData\werf\bin"
mkdir $WERF_BIN_PATH
Invoke-WebRequest -Uri https://dl.bintray.com/flant/werf/v1.1.23+fix37/werf-windows-amd64-v1.1.23+fix37.exe -OutFile $WERF_BIN_PATH\werf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$WERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
cmd.exe (run as Administrator)
set WERF_BIN_PATH="C:\ProgramData\werf\bin"
mkdir %WERF_BIN_PATH%
bitsadmin.exe /transfer "werf" https://dl.bintray.com/flant/werf/v1.1.23+fix37/werf-windows-amd64-v1.1.23+fix37.exe %WERF_BIN_PATH%\werf.exe
setx /M PATH "%PATH%;%WERF_BIN_PATH%"
PowerShell
$WERF_BIN_PATH = "C:\ProgramData\werf\bin"
mkdir $WERF_BIN_PATH
Invoke-WebRequest -Uri https://dl.bintray.com/flant/werf/v1.1.23+fix37/werf-windows-amd64-v1.1.23+fix37.exe -OutFile $WERF_BIN_PATH\werf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$WERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
cmd.exe (run as Administrator)
set WERF_BIN_PATH="C:\ProgramData\werf\bin"
mkdir %WERF_BIN_PATH%
bitsadmin.exe /transfer "werf" https://dl.bintray.com/flant/werf/v1.1.23+fix37/werf-windows-amd64-v1.1.23+fix37.exe %WERF_BIN_PATH%\werf.exe
setx /M PATH "%PATH%;%WERF_BIN_PATH%"
PowerShell
$WERF_BIN_PATH = "C:\ProgramData\werf\bin"
mkdir $WERF_BIN_PATH
Invoke-WebRequest -Uri https://dl.bintray.com/flant/werf/v1.1.23+fix37/werf-windows-amd64-v1.1.23+fix37.exe -OutFile $WERF_BIN_PATH\werf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$WERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
cmd.exe (run as Administrator)
set WERF_BIN_PATH="C:\ProgramData\werf\bin"
mkdir %WERF_BIN_PATH%
bitsadmin.exe /transfer "werf" https://dl.bintray.com/flant/werf/v1.1.23+fix37/werf-windows-amd64-v1.1.23+fix37.exe %WERF_BIN_PATH%\werf.exe
setx /M PATH "%PATH%;%WERF_BIN_PATH%"
PowerShell
$WERF_BIN_PATH = "C:\ProgramData\werf\bin"
mkdir $WERF_BIN_PATH
Invoke-WebRequest -Uri https://dl.bintray.com/flant/werf/v1.1.23+fix36/werf-windows-amd64-v1.1.23+fix36.exe -OutFile $WERF_BIN_PATH\werf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$WERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
cmd.exe (run as Administrator)
set WERF_BIN_PATH="C:\ProgramData\werf\bin"
mkdir %WERF_BIN_PATH%
bitsadmin.exe /transfer "werf" https://dl.bintray.com/flant/werf/v1.1.23+fix36/werf-windows-amd64-v1.1.23+fix36.exe %WERF_BIN_PATH%\werf.exe
setx /M PATH "%PATH%;%WERF_BIN_PATH%"
PowerShell
$WERF_BIN_PATH = "C:\ProgramData\werf\bin"
mkdir $WERF_BIN_PATH
Invoke-WebRequest -Uri https://dl.bintray.com/flant/werf/v1.1.22+fix37/werf-windows-amd64-v1.1.22+fix37.exe -OutFile $WERF_BIN_PATH\werf.exe
[Environment]::SetEnvironmentVariable(
"Path",
[Environment]::GetEnvironmentVariable("Path", [EnvironmentVariableTarget]::Machine) + "$WERF_BIN_PATH",
[EnvironmentVariableTarget]::Machine)
$env:Path = [System.Environment]::GetEnvironmentVariable("Path","Machine") + ";" + [System.Environment]::GetEnvironmentVariable("Path","User")
cmd.exe (run as Administrator)
set WERF_BIN_PATH="C:\ProgramData\werf\bin"
mkdir %WERF_BIN_PATH%
bitsadmin.exe /transfer "werf" https://dl.bintray.com/flant/werf/v1.1.22+fix37/werf-windows-amd64-v1.1.22+fix37.exe %WERF_BIN_PATH%\werf.exe
setx /M PATH "%PATH%;%WERF_BIN_PATH%"
go get github.com/werf/werf/cmd/werf
All changes in werf
go through all stability channels
-
Alphacan bring new features
but can be unstable -
Betafor more broad testing
of new features to catch
regressions -
Early-Accessis mostly safe and can be used
in non-critical environments
or for local development -
Stableis mostly safe and we
encourage you to use
this version everywhere -
Rock-Solidthe most stable channel
and recommended for usage
in critical environments with tight SLA
When using release channels, you do not specify a version, because the version is managed automatically within the channel Stability channels and frequent releases allow receiving continuous feedback on new changes, quickly rolling problem changes back, ensuring the high stability of the software, and preserving an acceptable development speed at the same time.
The relations between channels and werf releases are described in multiwerf.json.
Backward compatibility promise
werf follows a versioning strategy called Semantic Versioning. It means that major releases (1.0, 2.0) can break backward compatibility. In the case of werf, an update to the next major release may require to do a full re-deploy of applications or to perform other non-scriptable actions.
Minor releases (1.1, 1.2, etc.) may introduce new global features, but have to do so without significant backward compatibility breaks with a major branch (1.x). In the case of werf, this means that an update to the next minor release goes smoothly most of the time. However, it may require running a provided upgrade script.
Patch releases (1.1.0, 1.1.1, 1.1.2) may introduce new features, but must do so without breaking backward compatibility within the minor branch (1.1.x). In the case of werf, this means that an update to the next patch release should be smooth and can be done automatically.
- We do not guarantee backward compatibility between:
alphareleases;betareleases;eareleases.
- We guarantee backward compatibility between:
stablereleases within the minor branch (1.1.x);rock-solidreleases within the minor branch (1.1.x).
For use in non critical environments or local development
These releases are mostly safe to use and can even be used in non critical environments or for local development. We do not guarantee backward compatibility between Early-Access releases.
Docs
[docs, ru] Advanced/Cleanup: actualize
[docs] Internal/Stages and storage: actualize
[docs] What’s new in v1.2/Changelog: cleaning commands
[docs] Internal/Build process: fix typo
[docs] Fix broken quickstart macos instructions
[docs] Fix v1.1 to v1.2 instruction: wrong images report usage
[docs] Fix v1.1 to v1.2 instruction: more correct report usage: pull images before usage
[docs] advanced/cleanup: added info about werf host cleanup GC
- Only in russian.
- Also added a helper table with all cleanup commands.
[docs] Fixed some grammar in v1.1 to v1.2 migration guide (RU)
[docs] Fix bundles translation to RU
Fixes
[dockerfile] Automatically reset the Dockerfile exception by .dockerignore
file rules
There is no way to ignore the Dockerfile due to docker limitation when building an image for a compressed context that reads fro
m STDIN.
Add an exception and warning message.
[GC] Fix werf-host-cleanup default percentage and margin values (always use 70% and 5% margin).
[helm] Werf-helm fixes
- Fix panic on werf-helm-dep-update when secret-values are used (fixes https://github.com/werf/werf/issues/3443).
- Set .Values.werf.env=”” stub werf service value in werf-helm-* commands.
Internals
- [web] Add view page source button (#3428)
[web] Add page source button - [ci] Switch to GHCR (#3441).
- Fix README werf logo path.
Installation
Fixes
[bundle] Fix werf-bundle-apply command does not use –env param
Installation
Fixes
[bundles] Fix bundle-publish command ignoring .helm/values.yaml and publishing .helm/secret-values.yaml
Installation
Fixes
[helm] Fix chart dependencies loader missing error reports
Installation
Docs
[docs] What’s new in v1.2: describe mounts rejection cases
[docs] Advanced/bundles: added supported registry implementations
Installation
Docs: refine whats_new_in_v1_2 documentation chapter and translate to russian (2)
- en: https://werf.io/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html
- ru: https://ru.werf.io/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html
(links will be available very soon)
Installation
Docs: refine whats_new_in_v1_2 documentation chapter and translate to russian
- en: https://werf.io/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html
- ru: https://ru.werf.io/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html
(links will be available very soon)
Installation
Add WERF_DISABLE_RESOURCES_WAITER=1 support
Specify env var to fully disable kubedog waiter.
Installation
Fix “Error: unable to locate chart directory: the directory “.helm” not found in the project git repository”
Installation
Docs
[docs] Basic bundles article in advanced documentation section
https://werf.io/v1.2-alpha/documentation/advanced/bundles.html
Internals
Rework ls tree and path matcher interface
[ls_tree] Change the LsTree function interface
// LsTree returns the Result with tree entries that satisfy the passed pathMatcher.
// The function works lazily and does not go through a tree directory unnecessarily.
// If the result should contain only regular files (without directories and submodules), you should use the allFiles parameter.
LsTree(ctx context.Context, repository *git.Repository, commit string, pathMatcher path_matcher.PathMatcher) (*Result, error) -> LsTree(ctx context.
Context, repository *git.Repository, commit string, pathMatcher path_matcher.PathMatcher, allFiles bool) (*Result, error)
[path_matcher] Remove the greedySearch parameter and change interface
There are the following changes in the PathMatcher interface:
- Rename the MatchPath method to IsPathMatched
- Split the ProcessDirOrSubmodulePath method into ShouldGoThrough and IsDirOrSubmodulePathMatched methods
// IsPathMatched checks for a complete matching of the path
IsPathMatched(string) bool
// ShouldGoThrough indicates that the directory or submodule path is not completely matched but may include matching files among the child files.
// The method returns false if the path is completely matched.
ShouldGoThrough(string) bool
// IsDirOrSubmodulePathMatched returns true if IsPathMatched or ShouldGoThrough.
// The method returns true if there is a possibility of containing the matching files among the child files.
IsDirOrSubmodulePathMatched(string) bool
[ls_tree] Remove the strict option in LsTree function
The option allowed to run ls tree for work tree with uninitialized submodules. In current versions, we always work with the service work tree, where the submodules must be initialized.
Installation
Automigration helm 2 to 3 fixes
[helm 2 to 3] Fix: ignore resources from helm 2 release which do not exist in kubernetes
Changed order of migration steps: first set annotations and labels, then create helm 3 release.
Installation
New giterminism documentation!
-
https://werf.io/documentation/advanced/giterminism.html
-
https://werf.io/documentation/reference/werf_giterminism_yaml.html
-
https://werf.io/documentation/reference/werf_yaml_template_engine.html
[docs] Overview: giterminism
[docs] Advanced/Configuration/Template engine: move to reference
[giterminism] Update error messages
[docs] Advanced/Giterminism
[docs] Reference/werf-giterminism.yaml
[docs] Advanced/Configuration/Template engine
[docs] Reference/werf.yaml: refactor and update
-
change the structure of werf yaml data file
-
add get_lang_field_or_raise_error filter to work effectively with new data structure
-
update directives descriptions
[ci] Fix Documentation Validation
There was an error while trying to write to `/__w/werf/werf/docs/Gemfile.lock`.
It is likely that you need to grant write permissions for that path.
[docs] Fix table line counter does not work properly
Installation
Fixes
[win][git] Fix CRLF issues and patch-does-not-apply errors
The root of the issue: werf have added file converted from LF to CRLF into built image under windows due to some local git autocrlf settings. Then werf cannot apply patch to the file with CRLF’s inside the build container.
- Fix werf to always add files into image from git without any conversions from LF to CRLF, or from CRLF to LF. Take files from the git as-is.
- Fix patches applier for files with CRLF already added into the built images for some reason (images built with an older werf version with this bug).
[docs] Fix incorrect link to Go guide
Installation
Docs
- Update guides template and overview page.
- Correct a grammar mistake.
Installation
Docs
Preparations for guides integration:
- https://werf.io/documentation/guides.html
- https://ru.werf.io/documentation/guides.html
Installation
Fixes
[follow] Fix follow mode does not reinitialize giterminism manager
Installation
Giterminism
[giterminism] Refactoring: simplify error messages and the corresponding code
the
the uncommitted configuration found in the project directory: the
[giterminism] Fix doublestar.Match matches system-dependent path
Custom project-dir related fixes
Fix dockerfile-not-found error when running werf not from the root of the git work tree
\# docs/werf.yaml
configVersion: 1
project: docs
---
image: docs
dockerfile: ./Dockerfile
When running werf from docs folder werf’s dockerfile builder will use docs dir — project dir — as build dockerfile build context root for the
ont
ext tar archive, not git work tree root.
[project-dir] Fix helm-chart-dir related errors when running werf with custom –dir.
[project-dir] Fix .git lookup should use –dir or process cwd.
Fixes and other
[doc] Fix the “Documentation” link in the site header. Fix jekyll custom filter.
Internals
[tests] Giterminism Suite: actualize tests
[giterminism] Remove unused giterminism inspector
Installation
Fixes
[deploy] werf_image and nameless image deprecation fix
Pass nameless image as .Values.werf.nameless_image instead of .Values.werf.image.
.Values.werf.image is always map[string]string and will only contain named images names.
As nameless image is being deprecated in the v1.2 there is no “normal” way to get nameless image from values, that’s why we use “special” value .Values.werf.nameless_image for now. In the v1.3 nameless images will be completely removed.
Installation
For more broad testing of new features
These releases are for more broad testing of new features to catch regressions. We do not guarantee backward compatibility between Beta releases.
Docs
[docs, ru] Advanced/Cleanup: actualize
[docs] Internal/Stages and storage: actualize
[docs] What’s new in v1.2/Changelog: cleaning commands
[docs] Internal/Build process: fix typo
[docs] Fix broken quickstart macos instructions
[docs] Fix v1.1 to v1.2 instruction: wrong images report usage
[docs] Fix v1.1 to v1.2 instruction: more correct report usage: pull images before usage
[docs] advanced/cleanup: added info about werf host cleanup GC
- Only in russian.
- Also added a helper table with all cleanup commands.
[docs] Fixed some grammar in v1.1 to v1.2 migration guide (RU)
[docs] Fix bundles translation to RU
Fixes
[dockerfile] Automatically reset the Dockerfile exception by .dockerignore
file rules
There is no way to ignore the Dockerfile due to docker limitation when building an image for a compressed context that reads fro
m STDIN.
Add an exception and warning message.
[GC] Fix werf-host-cleanup default percentage and margin values (always use 70% and 5% margin).
[helm] Werf-helm fixes
- Fix panic on werf-helm-dep-update when secret-values are used (fixes https://github.com/werf/werf/issues/3443).
- Set .Values.werf.env=”” stub werf service value in werf-helm-* commands.
Internals
- [web] Add view page source button (#3428)
[web] Add page source button - [ci] Switch to GHCR (#3441).
- Fix README werf logo path.
Installation
[dockerfile] Support dirs in contextAddFiles
- Add an ability to specify directories along with files in contextAddFile(s) directive.
Directories will be added recursively to the current Docker context. - Rename contextAddFile to contextAddFiles throughout the code and in werf.yaml, but
keep contextAddFile supported as an alias for contextAddFiles for backwards compatibility.
Cleanup improvements
[cleaning] More detailed cleaning logging
[cleaning] Remove werf-host-project commands
[cleaning] werf host purge command: add the –project-name option to remove project images
[cleaning] werf host cleanup command: add the stub –project-name option (without implementation)
[cleaning] Remove the local storage support for cleanup and purge commands
[storage] Disable handling of image metadata in the local storage
[storage] Disable handling of managed images in the local storage
Remove the local storage support for managed-images commands
[GC] Host cleanup improvements
- Cleanup service meta images: commit-metadata images, client-id images, managed-images records, import-metadata records, rejected-images records.
- Save “werf” service image label into all service images created by the werf for future usage.
- Calculate image size in cleanup GC algorithm as VIRTUAL_SIZE-SHARED_SIZE.
Builder fixes
- Fix: print rsync server docker run command on failure.
- [stapel] Fix: fetch artifact/image stage before running imports rsync server
Fetch operation will check that image exists correctly and will reset stages building conveyor to rebuild this stage if needed. - Support for rejected stages in the stages storage
Reject stage when docker-pull have failed with the “blob unknown” error.
Add special record into the stages-storage for such rejected stages. - Fix saving stages-ids into the stages-storage-cache
When some stage has appeared in the repository tags list, but stage image is invalid (due to MANIFEST_UNKNOWN or BLOB_UNKNOWN container registry err
ors), werf ignores such stage. But this stage-id of such broken stage has been saved into the stages-storage-cache anyway, which is wrong.
Docs
[docs] What’s new in v1.2/Changelog: herebyIAdmitThatFromLatestMightBreakReproducibility/herebyIAdmitThatBranchMightBreakReproducibility
Internals
[web] Add guides sitemap (#3420)
[tests] Cleanup Suite: remove the local storage tests
[tests] Cleanup Suite: werf host purge command tests
[tests] Replace werf purge after callback to werf host purge –project-name
[tests] container_registry_per_implementation set: the optional local container registry support
[tests] Cleanup Suite: code generalization
[tests] container_registry_per_implementation set: the optional local container registry support
Installation
Fixes
[bundle] Fix werf-bundle-apply command does not use –env param
Installation
Docs
[docs] Advanced/Supported container registries
[docs] Advanced/Bundles: update supported container registries section
[docs] Using the term container registry instead of docker registry implementation
- rename Advanced/Supported registry implementations to Advanced/Supported container registries
- /documentation/advanced/supported_registry_implementations.html -> /documentation/advanced/supported_container_registries.html
- correct all occurrences
[docs] Fix russian bundles doc
[docs] Exclude 404 pages from sitemaps (#3330)
Internals
[ci] More convenient job names (#3332)
Installation
Fixes
[helm] Enable secret-values in ‘werf helm *’ commands, fix lint command
- Refactored pkg/deploy/helm/chart_extender.
- Separation of chart-extenders itself and helpers.
- Common code for secret loader.
- Refactored secrets-manager: accept working dir param only when reading secret-key.
- Fixed ‘werf helm lint’ command (disabled linting of an optional .helm/Chart.yaml file).
- Refs https://github.com/werf/helm/pull/89
[helm] Fix additional –secret-values param files should redefine previously defined secret-values
Internals
[ci] Don’t deploy backend to the test env (#3327)
Installation
Fixes
[bundles] Fix bundle-publish command ignoring .helm/values.yaml and publishing .helm/secret-values.yaml
Installation
Fixes
[helm] Fix chart dependencies loader missing error reports
Installation
Docs
[docs] What’s new in v1.2: describe mounts rejection cases
[docs] Advanced/bundles: added supported registry implementations
Installation
Docs: refine whats_new_in_v1_2 documentation chapter and translate to russian (2)
- en: https://werf.io/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html
- ru: https://ru.werf.io/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html
(links will be available very soon)
Installation
Docs: refine whats_new_in_v1_2 documentation chapter and translate to russian
- en: https://werf.io/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html
- ru: https://ru.werf.io/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html
(links will be available very soon)
Installation
Add WERF_DISABLE_RESOURCES_WAITER=1 support
Specify env var to fully disable kubedog waiter.
Installation
Fix “Error: unable to locate chart directory: the directory “.helm” not found in the project git repository”
Installation
Docs
[docs] Basic bundles article in advanced documentation section
https://werf.io/v1.2-alpha/documentation/advanced/bundles.html
Internals
Rework ls tree and path matcher interface
[ls_tree] Change the LsTree function interface
// LsTree returns the Result with tree entries that satisfy the passed pathMatcher.
// The function works lazily and does not go through a tree directory unnecessarily.
// If the result should contain only regular files (without directories and submodules), you should use the allFiles parameter.
LsTree(ctx context.Context, repository *git.Repository, commit string, pathMatcher path_matcher.PathMatcher) (*Result, error) -> LsTree(ctx context.
Context, repository *git.Repository, commit string, pathMatcher path_matcher.PathMatcher, allFiles bool) (*Result, error)
[path_matcher] Remove the greedySearch parameter and change interface
There are the following changes in the PathMatcher interface:
- Rename the MatchPath method to IsPathMatched
- Split the ProcessDirOrSubmodulePath method into ShouldGoThrough and IsDirOrSubmodulePathMatched methods
// IsPathMatched checks for a complete matching of the path
IsPathMatched(string) bool
// ShouldGoThrough indicates that the directory or submodule path is not completely matched but may include matching files among the child files.
// The method returns false if the path is completely matched.
ShouldGoThrough(string) bool
// IsDirOrSubmodulePathMatched returns true if IsPathMatched or ShouldGoThrough.
// The method returns true if there is a possibility of containing the matching files among the child files.
IsDirOrSubmodulePathMatched(string) bool
[ls_tree] Remove the strict option in LsTree function
The option allowed to run ls tree for work tree with uninitialized submodules. In current versions, we always work with the service work tree, where the submodules must be initialized.
Installation
Automigration helm 2 to 3 fixes
[helm 2 to 3] Fix: ignore resources from helm 2 release which do not exist in kubernetes
Changed order of migration steps: first set annotations and labels, then create helm 3 release.
Installation
New giterminism documentation!
-
https://werf.io/documentation/advanced/giterminism.html
-
https://werf.io/documentation/reference/werf_giterminism_yaml.html
-
https://werf.io/documentation/reference/werf_yaml_template_engine.html
[docs] Overview: giterminism
[docs] Advanced/Configuration/Template engine: move to reference
[giterminism] Update error messages
[docs] Advanced/Giterminism
[docs] Reference/werf-giterminism.yaml
[docs] Advanced/Configuration/Template engine
[docs] Reference/werf.yaml: refactor and update
-
change the structure of werf yaml data file
-
add get_lang_field_or_raise_error filter to work effectively with new data structure
-
update directives descriptions
[ci] Fix Documentation Validation
There was an error while trying to write to `/__w/werf/werf/docs/Gemfile.lock`.
It is likely that you need to grant write permissions for that path.
[docs] Fix table line counter does not work properly
Installation
Fixes
[win][git] Fix CRLF issues and patch-does-not-apply errors
The root of the issue: werf have added file converted from LF to CRLF into built image under windows due to some local git autocrlf settings. Then werf cannot apply patch to the file with CRLF’s inside the build container.
- Fix werf to always add files into image from git without any conversions from LF to CRLF, or from CRLF to LF. Take files from the git as-is.
- Fix patches applier for files with CRLF already added into the built images for some reason (images built with an older werf version with this bug).
[docs] Fix incorrect link to Go guide
Installation
Fixes
[bundles] Fix “unable to create bundle: unable to write … no such file or directory”
[giterminism] Fix broken symlinks are processed incorrectly
[stapel, git] Fix adding a broken symlink does not work
Installation
Fixes
Fix interactive werf run session (–shell/–bash) works not properly
- Fix proxy stream data formatting enabled mode
Installation
Fixes
[giterminism] Fix processing of an invalid submodule state
Installation
Giterminism refinements: full git-submodules support, better custom project dir support
[giterminism] Improve feedback if a related submodule is not clean or has changes
- Ignore user work tree submodule directory if a submodule “.git” directory not found
-
Return a detailed error if a work tree submodule directory is not clean:
the submodule "<submodule path>" is not clean and must be committed. Do not forget to push the current commit to the submodule remote If this commit exists only locally Details: commit: "0000000000000000000000000000000000000000" currentWorktreeCommit: "d9243a9a45b10e6b7d985322b8108897626a25a6" expectedWorktreeCommit: "d9243a9a45b10e6b7d985322b8108897626a25a6" You might also be interested in developer mode (activated with --dev option) that allows you to work with staged changes without doing redundant commits. Just use "git add <file>..." to include the changes that should be used. -
Return a detailed error if a work tree submodule directory has changes:
the submodule "<submodule path>" has modified files and these changes must be committed (do not forget to push new changes to the submodule remote) or discarded: - file1 - file2
[giterminism] Support for configuration files outside the project directory
Internals:
- [tests] Giterminism Suite: test custom project directory
- [giterminism] Refactor symlink check code
Docs
[doc] Add Rails guide
Internals
[tests] Fix daily tests, not compiling and failing tests, refactor tests precompile
Installation
Docs
- Update guides template and overview page.
- Correct a grammar mistake.
Installation
Docs
Preparations for guides integration:
- https://werf.io/documentation/guides.html
- https://ru.werf.io/documentation/guides.html
Installation
Fixes
[follow] Fix follow mode does not reinitialize giterminism manager
Installation
Giterminism
[giterminism] Refactoring: simplify error messages and the corresponding code
the
the uncommitted configuration found in the project directory: the
[giterminism] Fix doublestar.Match matches system-dependent path
Custom project-dir related fixes
Fix dockerfile-not-found error when running werf not from the root of the git work tree
\# docs/werf.yaml
configVersion: 1
project: docs
---
image: docs
dockerfile: ./Dockerfile
When running werf from docs folder werf’s dockerfile builder will use docs dir — project dir — as build dockerfile build context root for the
ont
ext tar archive, not git work tree root.
[project-dir] Fix helm-chart-dir related errors when running werf with custom –dir.
[project-dir] Fix .git lookup should use –dir or process cwd.
Fixes and other
[doc] Fix the “Documentation” link in the site header. Fix jekyll custom filter.
Internals
[tests] Giterminism Suite: actualize tests
[giterminism] Remove unused giterminism inspector
Installation
Fixes
[deploy] werf_image and nameless image deprecation fix
Pass nameless image as .Values.werf.nameless_image instead of .Values.werf.image.
.Values.werf.image is always map[string]string and will only contain named images names.
As nameless image is being deprecated in the v1.2 there is no “normal” way to get nameless image from values, that’s why we use “special” value .Values.werf.nameless_image for now. In the v1.3 nameless images will be completely removed.
Installation
Guides has been translated to English
Fixed all links to guides: https://werf.io/applications_guide
Installation
Fixes
[cleanup] Fix goroutine stack exceeds limit
One checksum can be associated with many stages, so it is necessary to exclude the stage itself before processing linked imports
Installation
Fixes
Fix deadline-exceeded when hook job took more than 5 min:
- Fix default helm 5 min timeout => infinite.
- Fix any timeout other than default results in ‘context deadline’ error.
Installation
Fixes
- Fix Chart.lock file not unpacked when applying bundle.
Installation
Fixes
Fix “unable to create registry client” error when DOCKER_CONFIG variable is set
Installation
New bundle commands and fix bundle registry authentication
werf bundle export— creates resulting bundle chart directory without publishing this chart into the docker registr
.werf bundle download— downloads previously published chart into the directory without applying it into the kubernete
.
Use default ~/.docker/config.json authentication data when publishing and applying bundles.
Installation
Fix error: unable to resolve docker endpoint: open …/ca.pem: no such file or directory
Installation
Dev mode internal improvements and fixes
[dev] Apply a patch with staged changes also in index.
[true git] Reset worktree/staging changes when detaching commit.
[tests] Build/Stapel/Git: active developer mode tests.
[status] Check worktree and staging changes separately with options.
[status] Remove unused code.
[dev] Support staged binary files.
Dockerfile builder improvements
[dockerfile] Add project repo commit label.
Installation
Fixes
- [dockerfile] Temporarily force BuildKit deactivation due to the bug in the docker client when used as a go library
- Fix ‘werf render’ uses kubernetes based lock due to typo.
Installation
Fixes
- [dev] Fix not initialized true git package.
- [stapel, dev] Fix adding dev label to the building image.
- [giterminism] Check untracked .helm/Chart.yaml exists in the chart.
Installation
Rework dev mode for stapel and dockerfile
- Werf creates temporary commits in the dev-mode.
- Files from the git index used in the dev-mode.
- Support dev-mode in conjunction with the follow-mode.
- Commit staged files to dev branch
werf-dev-<commit>. - Add werf-dev label to git stage images.
Change default werf images storage for GitLab CI/CD and GitHub Actions
- Remove
/werfrepo suffix for GitLab CI/CD; - Remove
-werfrepo suffix for GitHub Actions.
NOTE. This is breaking change, incompatible with the previous werf versions.
Other
- Fix “list event error: context cancelled” log messages.
- [web] Update metainfo in page template.
- More fixes for werf-related ci tests.
Installation
Docs
[docs] Reform build-proces and stages-and-storage articles
- Move articles one level up.
- Rename “Images storage” article to “Stages and storage”.
- Reorganize and actualize articles.
Installation
Build images report rework
- Fix build images report not working.
- Add envfile report format (–report-format=envfile).
- Change report format structure.
Installation
Docs
[docs] Reform build-proces and stages-and-storage articles
- Move articles one level up.
- Rename “Images storage” article to “Stages and storage”.
- Reorganize and actualize articles.
Installation
Docs and fixes
- [docs] Remove outdated warning.
- [docs] Add last publications.
- [docs] Fix title generation.
- [docs] Fix breadcrumbs.
- [docs] Enable beta channel selected by default.
- [docs] Eliminate werf-deploy and werf-build-and-publish commands from docs.
- [docs] Advanced/Helm/Basics: replace werf deploy command with converge.
- [cleanup] Delete invalid import metadata.
Installation
[Local development] --follow param support for werf-run and werf-converge
werf converge --follow will loop on changes into current git-repository and restart converge command when a new commit has been made.
werf run --follow will loop on changes into current git-repository and restart docker container when a new commit has been made.
Fix ‘Error: UPGRADE FAILED: another operation (install/upgrade/rollback) is in progress’ error
https://github.com/werf/helm/pull/64
#
Installation
Fixes
- Disable kubernetes connection in the werf-render command. Werf-render command now does not need a connection to the Kubernetes cluster.
- Fix ‘No image specified for template’ when using nameless image.
Installation
Update helm 3 and k8s dependencies
- Helm 3: v3.2.4 to v3.4.0
- K8s deps: v0.18.6 -> v0.19.2
Fixes
- [stapel, imports] Fix import source checksum generation does not work on windows.
- [stapel, imports] Update debug message.
Installation
Fixes
- [build] Fix panic in the stapel builder.
- [cleanup] Fix resolve revision @name/subname@0.0.1 failed
Installation
Docs
Installation page: enable v1.2 beta switch.
Installation
These releases can bring new features, but are unstable. We do not guarantee backward compatibility between Alpha releases.
Internals
[CI] Documentation local development (#3457)
To run documentation locally, use:
cd docs/site &&
werf compose up --follow --docker-compose-command-options="-d"
Open http://localhost:80
To stop:
cd docs/site &&
werf compose down
You should commit main site files (site/*) into git to see changes.
Documentation files changes will be almost immediately available.
Docs
- [docs] cleanup.md - translate and sync to RU version.
- [docs] Advanced/Cleanup: update intro.
- [docs] Reference/werf.yaml: fix broken link.
- [docs] Reference/werf.yaml template engine: fix typo.
- [docs] Fix helm configuration templates page (#3463)
- [docs] Fix markup errors in advanced/helm/configuration/templates.html
Fixes
- [helm] Force helm kube client namespace initialization.
- [cleanup] Ignore blob-unknown errors properly.
During cleanup operation do not raise should-reset-stages-storage-cache error when get stage manifest operation have failed with blob-unknown error. - [run] Fix nonexistent option in the error.
- [follow] Fix werf run: changes cause reassembly but are not accounted for.
- [logging] Build: add waiting for background tasks log message.
Installation
Docs
[docs, ru] Advanced/Cleanup: actualize
[docs] Internal/Stages and storage: actualize
[docs] What’s new in v1.2/Changelog: cleaning commands
[docs] Internal/Build process: fix typo
[docs] Fix broken quickstart macos instructions
[docs] Fix v1.1 to v1.2 instruction: wrong images report usage
[docs] Fix v1.1 to v1.2 instruction: more correct report usage: pull images before usage
[docs] advanced/cleanup: added info about werf host cleanup GC
- Only in russian.
- Also added a helper table with all cleanup commands.
[docs] Fixed some grammar in v1.1 to v1.2 migration guide (RU)
[docs] Fix bundles translation to RU
Fixes
[dockerfile] Automatically reset the Dockerfile exception by .dockerignore
file rules
There is no way to ignore the Dockerfile due to docker limitation when building an image for a compressed context that reads fro
m STDIN.
Add an exception and warning message.
[GC] Fix werf-host-cleanup default percentage and margin values (always use 70% and 5% margin).
[helm] Werf-helm fixes
- Fix panic on werf-helm-dep-update when secret-values are used (fixes https://github.com/werf/werf/issues/3443).
- Set .Values.werf.env=”” stub werf service value in werf-helm-* commands.
Internals
- [web] Add view page source button (#3428)
[web] Add page source button - [ci] Switch to GHCR (#3441).
- Fix README werf logo path.
Installation
Helm update
Builtin helm updated to v3.5.3.
Fixes
[helm] Add support for oci registries in Chart.yaml: fix panic; updated helm
- Panic was due to uninitialized helm registry client.
Fixes https://github.com/werf/werf/issues/3415
[builder] Perform rejection of blob-unknown stages
Perform rejection of stages for which get-manifest has failed with blob-unknown error.
Internals and site
[tests] Bundle Suite: fix
Bump kramdown from 2.3.0 to 2.3.1 in /docs/documentation
Bumps kramdown from 2.3.0 to 2.3.1.
[web] Fix documentation overview assets (#3432)
[web] Use common partials for guides (#3394)
- [web] Use common partials for guides
- [web] Guides tiles fix
Installation
[dockerfile] Support dirs in contextAddFiles
- Add an ability to specify directories along with files in contextAddFile(s) directive.
Directories will be added recursively to the current Docker context. - Rename contextAddFile to contextAddFiles throughout the code and in werf.yaml, but
keep contextAddFile supported as an alias for contextAddFiles for backwards compatibility.
Cleanup improvements
[cleaning] More detailed cleaning logging
[cleaning] Remove werf-host-project commands
[cleaning] werf host purge command: add the –project-name option to remove project images
[cleaning] werf host cleanup command: add the stub –project-name option (without implementation)
[cleaning] Remove the local storage support for cleanup and purge commands
[storage] Disable handling of image metadata in the local storage
[storage] Disable handling of managed images in the local storage
Remove the local storage support for managed-images commands
[GC] Host cleanup improvements
- Cleanup service meta images: commit-metadata images, client-id images, managed-images records, import-metadata records, rejected-images records.
- Save “werf” service image label into all service images created by the werf for future usage.
- Calculate image size in cleanup GC algorithm as VIRTUAL_SIZE-SHARED_SIZE.
Builder fixes
- Fix: print rsync server docker run command on failure.
- [stapel] Fix: fetch artifact/image stage before running imports rsync server
Fetch operation will check that image exists correctly and will reset stages building conveyor to rebuild this stage if needed. - Support for rejected stages in the stages storage
Reject stage when docker-pull have failed with the “blob unknown” error.
Add special record into the stages-storage for such rejected stages. - Fix saving stages-ids into the stages-storage-cache
When some stage has appeared in the repository tags list, but stage image is invalid (due to MANIFEST_UNKNOWN or BLOB_UNKNOWN container registry err
ors), werf ignores such stage. But this stage-id of such broken stage has been saved into the stages-storage-cache anyway, which is wrong.
Docs
[docs] What’s new in v1.2/Changelog: herebyIAdmitThatFromLatestMightBreakReproducibility/herebyIAdmitThatBranchMightBreakReproducibility
Internals
[web] Add guides sitemap (#3420)
[tests] Cleanup Suite: remove the local storage tests
[tests] Cleanup Suite: werf host purge command tests
[tests] Replace werf purge after callback to werf host purge –project-name
[tests] container_registry_per_implementation set: the optional local container registry support
[tests] Cleanup Suite: code generalization
[tests] container_registry_per_implementation set: the optional local container registry support
Installation
Docs fixes
[web] Fix 404 page (#3410)
[web] Change 404 logic (#3397)
[docs] Replace Werf with werf
[docs] Fix style, grammar, and typos
Installation
[GC] Host auto cleanup refinements
- Enabled auto host cleanup by default.
- –allowed-docker-storage-volume-usage and –allowed-docker-storage-volume-usage-margin to configure docker-storage host cleanup.
- –allowed-local-cache-volume-usage and –allowed-local-cache-volume-usage-margin to configure local-storage host cleanup (including git-data).
Docs
[web] Fix installation page tab logic (#3395)
[docs] Remove 1.0 version from the installation page (#3396)
Internals
[CI] Add commit and PR tests (#3393)
[CI] Update Certificate template
Installation
[GC] Git-data repos, worktrees, archives and patches GC based on LRU algorithm
- Remove v1.1 git-data if werf v1.1 was not started for 3 days on this build-host.
- Remove *.tmp data from git_repos, which are git-clone temporary dirs accidentally left existing.
- Moved gc and git-data-manager related code into the separate package gitdata inside git_repo package (refactor).
- Fixed new bug introduced by the v1.2.10+fix14 version:
assets/setup /.werf/stapel/embedded/bin/xargs: Cannot open input file '/.werf/patch/.werf/patch/0c962ed1755d99a8937abe6fe63953bf818...
Installation
[cleanup] Fix cleaning images built on a virtual merge commit
[metadata] Add image metadata for the from virtual merge commit
Deprecations
Deprecation warning for the –repo-implementation option
DEPRECATION WARNING: The option --repo-implementation ($WERF_REPO_IMPLEMENTATION) is renamed to --repo-container-registry ($WERF_REPO_CONTAINER_REGISTRY) and will be removed in v1.3!
Internals
[docs][ci] SEO improvement and ci updates (#3373)
[ci] Tests Workflow: drop out of self-hosted runners
[ci] Documentation Validation: fix `chmod: Gemfile.lock: No such file or directory
[ci] Fix tests
```
Run # unit tests binaries
? github.com/werf/werf/cmd/werf [no test files]
go: directory integration/ci_suites/default/docs/_fixtures/cli/docs/site/backend is outside main module
Error: Process completed with exit code 1.
```
[ci] Workflows: replace always with success || failure condition
The always function returns true even when canceled, or the previous tasks skipped.
[ci] Documentation Validation: fix chmod: Gemfile.lock: No such file or directory
[ci] Use .Values.werf.env instead of .Values.global.env
[maintenance_helper] Fix error format
- Errorf format %s reads arg #2, but call has 1 arg
- Errorf format %s reads arg #3, but call has 2 args
Installation
[GC] Add support for v1.1 images into host cleanup procedure
Cleanup only those images/stages, which has been published into REPO stages-storage.
NOTE. V1.2’s werf host cleanup procedure will not cleanup –stages-storage=:local stages, because this is primary stages storage data, and it can only be cleaned by the regular per-project werf cleanup command with git-history based algorithm.
Installation
[cleanup] Fix cleaning images built on a virtual merge commit
[metadata] Add image metadata for the from virtual merge commit
Deprecations
Deprecation warning for the –repo-implementation option
DEPRECATION WARNING: The option --repo-implementation ($WERF_REPO_IMPLEMENTATION) is renamed to --repo-container-registry ($WERF_REPO_CONTAINER_REGISTRY) and will be removed in v1.3!
Internals
[docs][ci] SEO improvement and ci updates (#3373)
[ci] Tests Workflow: drop out of self-hosted runners
[ci] Documentation Validation: fix `chmod: Gemfile.lock: No such file or directory
[ci] Fix tests
```
Run # unit tests binaries
? github.com/werf/werf/cmd/werf [no test files]
go: directory integration/ci_suites/default/docs/_fixtures/cli/docs/site/backend is outside main module
Error: Process completed with exit code 1.
```
[ci] Workflows: replace always with success || failure condition
The always function returns true even when canceled, or the previous tasks skipped.
[ci] Documentation Validation: fix chmod: Gemfile.lock: No such file or directory
[ci] Use .Values.werf.env instead of .Values.global.env
[maintenance_helper] Fix error format
- Errorf format %s reads arg #2, but call has 1 arg
- Errorf format %s reads arg #3, but call has 2 args
Installation
[dockerfile] The BuildKit support with explicit user activation
- By default werf disables BuildKit when building dockerfiles.
- If the user explicitly activates BuildKit (DOCKER_BUILDKIT = 1), werf enables it but without output in the background builds due to https://github.com/docker/cli/issues/2889.
[GC] Rework host cleanup more
- Run auto host cleanup only when –disable-host-cleanup=false (WERF_DISABLE_HOST_CLEANUP=false), cleanup will be enabled by default later.
- Run auto host cleanup could run in werf-build/werf-converge and other commands as previously, BUT it will run in the end of the command.
- Big note message is printed before running auto host cleanup.
- This message contains info how to disable auto-host-cleanup.
- Big note message is printed before running auto host cleanup.
- Run /tmp/werf-config-render-* and /tmp/werf-project-data-* cleanup in the werf-host-cleanup procedure as well.
- Removed global lock for host cleanup, added individual locks for stage-images and build-containers
- Run auto host cleanup only when 2 hour has passed since first werf run on this host (try to determine that current host is persistent).
Fixes
[status] Fix duplications in the path list
Error: unable to read werf config: the following files must be committed:
- werf.yaml
- werf.yaml
Internals
[ci] Delete cleanup job (#3371)
Installation
Speeding up git-repo operations
[git_repo] Caching and optimization operations with git objects
Updated docker, buildkit and containerd go-dependencies
- github.com/docker/cli => v20.10.5
- github.com/docker/docker => v20.10.5
- github.com/containerd/containerd => v1.5.0-beta.4
- github.com/moby/buildkit => v0.8.2
Installation
[GC] Host cleaning GC refinements and fixes
[GC] Host cleaning GC refinements and fixes: correct usage of allowed-volume-usage and allowed-volume-usage-margin
Installation
Fixes
[git_repo] Invalidate archives and patches cache
Installation
[GC] Host cleanup garbage collection refinements
- Added –allowed-volume-usage-margin param which sets target volume usage after cleanup: allowed-volume-usage - allowed-volume-usage-margin.
- Delete 10 images minimal in each GC iteration to speed up cleanup.
Installation
Internals
[web] Fix documentation-version build: increment cache version
Installation
Fixes
[git_repo] Fix LsTree/Patch/Archive cache ID
[git] Invalidate cached git-archives and git-patches: changed cache version
Installation
Internals
Path matchers
- Remove irrelevant legacy methods:
TrimFileBaseFilepathandBaseFilepath - Redesign path matchers and divide them into simple entities
- base path
- include
- exclude
- dockerignore patterns
- true
- false
- complex
- multi
- Add factory constructor for easy and clear use
- Add the
IDmethod to determine the path matcher parameters unambiguously - Fix multi path matcher the
ShouldGoThroughmethod logic: the method returns true if theShouldGoThroughmethod of at least one matcher returns true and the path partially or completely matched by others (IsDirOrSubmodulePathMatchedreturns true) - Complete test coverage
Git
- Generalization of the calculating git repository files checksum
- Simplify the GetOrCreateChecksum method
- the method creates or gets a checksum from the cache
- the previous logic was specific to git mapping (it is taken out and based on the new implementation)
- The logic of stapel and dockerfile checksum calculation is reduced to one mechanism
- Simplify the GetOrCreateChecksum method
- Archive/Patch/LsTree: add the
PathScopeoption to limit paths and determine the base directory/file (similar to <pathspec> in the git commands).
Other
- [util] Fix IsSubpathOfBasePath helper
- Incorrect processing of empty (
"") and root ("/") basePath - Checking for identical paths returns true
- Incorrect processing of empty (
Installation
[GC] Basic garbage collection for build host images and containers
- Experimental feature: automatical host cleaning GC procedure will be activated only when WERF_ENABLE_HOST_STORAGE_GC=1 is set.
- Werf may perform GC procedure in main commands (werf-build, werf-converge, etc.).
- To manually call GC procedure use
werf host cleanupcommand. - Algorithm and options:
- Werf will try to maintain 80% disk usage by default and perform deletion of old images build by werf (and containers, and dangligh images as well). User may specify different threshold percentage with –allowed-volume-usage=N (WERF_ALLOWED_VOLUME_USAGE) option.
- Werf tries to detect docker storage volume to check volume usage. User may specify –docker-server-storage-path (WERF_DOCKER_SERVER_STORAGE_PATH) option manually to specify volume where docker storage directory reside.
Fixes
[dockerfile] Skip files ignored by .dockerignore file creating a context archive
[bundle] Fix werf-bundle-apply command does not use –env param
[compose] Fetch last stages of built images before running compose
[dockerfile] Fix .dockerignore does not affect on stage digest when using a custom project directory
Docs
[web] Fix some redirects (#3347)
[docs] Update the version of werf actions
werf/actions/deploy@master -> werf/actions/deploy@v1.2
[docs] Tune sitemaps (#3337, #3336)
[docs] Renaming the translation of the term “bundle”
Internals
[git_repo] Use PatchMatcher instead of FilterOptions
[path_matcher] Update multiPathMatcher string format
[ci] Delete namespace on review dismiss (#3333)
Installation
Fixes
[bundle] Fix werf-bundle-apply command does not use –env param
Installation
Docs
[docs] Advanced/Supported container registries
[docs] Advanced/Bundles: update supported container registries section
[docs] Using the term container registry instead of docker registry implementation
- rename Advanced/Supported registry implementations to Advanced/Supported container registries
- /documentation/advanced/supported_registry_implementations.html -> /documentation/advanced/supported_container_registries.html
- correct all occurrences
[docs] Fix russian bundles doc
[docs] Exclude 404 pages from sitemaps (#3330)
Internals
[ci] More convenient job names (#3332)
Installation
Fixes
[helm] Enable secret-values in ‘werf helm *’ commands, fix lint command
- Refactored pkg/deploy/helm/chart_extender.
- Separation of chart-extenders itself and helpers.
- Common code for secret loader.
- Refactored secrets-manager: accept working dir param only when reading secret-key.
- Fixed ‘werf helm lint’ command (disabled linting of an optional .helm/Chart.yaml file).
- Refs https://github.com/werf/helm/pull/89
[helm] Fix additional –secret-values param files should redefine previously defined secret-values
Internals
[ci] Don’t deploy backend to the test env (#3327)
Installation
Fixes
[bundles] Fix bundle-publish ignores .helm/values.yaml and uses .helm/secret-values.yaml
Update Helm
[helm] Update helm v3.5.0 -> v3.5.2
https://github.com/helm/helm/releases/tag/v3.5.2
Internals
[ci] Website review environments (#3315)
Installation
Giterminism
[giterminism] Separate processing for untracked and uncommitted files
Docs
[docs] What’s new in v1.2: describe mounts rejection cases
[docs] Advanced/bundles: added supported registry implementations
Other
Kubedog update: disable events informer with KUBEDOG_DISABLE_EVENTS=1
Installation
Docs: refine whats_new_in_v1_2 documentation chapter and translate to russian (2)
- en: https://werf.io/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html
- ru: https://ru.werf.io/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html
(links will be available very soon)
Installation
Docs: refine whats_new_in_v1_2 documentation chapter and translate to russian
- en: https://werf.io/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html
- ru: https://ru.werf.io/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html
(links will be available very soon)
Installation
Developer mode change for follow mode and docs
[docs] Actualize the developer mode references
[dev] Update the –dev option description
[follow] New following logic in development mode
The mode allows restarting the command on a new commit.
In development mode (–dev), it additionally tracks changes in the index state of the git repository, regardless of whether simple or strict development mode (–dev-mode) is used.
[dev] Change the cache logic in simple development mode
Create a single commit for changes in the worktree instead of two separate commits for index and worktree changes.
Installation
Support for modified/tracked files for the dev mode
- Introduced
--dev-mode simple|strictoption. --dev-mode simple— default mode when--devoption has been specified, to use strict mode specify `–dev-mode strict
.- In simple dev mode werf will use only files which are staged for commit (files which has been added with the
git addcommand) and also modified files changes, which has not been staged for commit. - In strict mode werf will use only files which are staged for commit (files which has been added with the
git addcommand).
Fix glob DIR/** not working in stage-dependencies
[path_matcher] Fix Simple/GitMappingPathMatcher.IsMatched method logic
Several sets of globes are used when comparing paths:
- The globes as they are.
- The globes without asterisks on the right (path//dir//*, path/*/dir/, path//dir///** -> path/*/dir).
- The globes from the previous set with the universal part
**/*(path//dir/**/).
Installation
Fix “Error: unable to locate chart directory: the directory “.helm” not found in the project git repository”
Installation
Docs
[docs] “What’s new in v1.2 / How to migrate from v1.1 to v1.2” guide
- English version only, Russian coming soon.
- https://werf.io/v1.2-alpha/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html (link will be available very soon).
[docs] Advanced/Stapel/Mount: add giterminism note
[docs] Reference/werf.yaml: contextAddFile
[docs] Reference: rearrange menu items
Internal fixes
Fixes for new CI and new site infra.
[web] Fix topnav
[web] Fix RSS feeds and installation page
[ci] Fix web converge
[web] Fix version menu
Installation
Docs
[docs] Translate to russian and refine advanced/helm chapter
https://ru.werf.io/v1.2-alpha/documentation/advanced/helm/overview.html (ссылка начнёт работать в ближайшее время)
[docs] Introduction: add a reference to giterminism
[docs] Advanced/Giterminism: add dev mode note
[docs] Internals/Build Process
- More details about dockerfile image
- Parallel builds
[docs] Internals/Stages and storage: sync
[docs] Reference/werf.yaml template engine: the .Env variable
[docs] Advanced/Giterminism: small improvement
[docs] Reference/werf.yaml template engine: the template directory
[docs] Reference/werf.yaml template engine: update the tpl function
Fix stageDependencies dir/** does to work
[path_matcher] Update Simple/GitMappingPathMatcher.IsMatched method logic
Several sets of globes are used when comparing paths:
- The universal part */ is added to all globes (as before).
- From all globes all asterisks to the right are cut off.
Internals
[ci] Daily Tests: fix “Prepare coverage file” step
[ci] Daily Tests: disable self-hosted runners
[ci] Fix directory integration/ci_suites/default/docs/_fixtures/cli/docs/backend is outside main module
Installation
Docs
[docs] “What’s new in v1.2 / How to migrate from v1.1 to v1.2” guide
- English version only, Russian coming soon.
- https://werf.io/v1.2-alpha/documentation/whats_new_in_v1_2/how_to_migrate_from_v1_1_to_v1_2.html (link will be available very soon).
[docs] Advanced/Stapel/Mount: add giterminism note
[docs] Reference/werf.yaml: contextAddFile
[docs] Reference: rearrange menu items
Internal fixes
Fixes for new CI and new site infra.
[web] Fix topnav
[web] Fix RSS feeds and installation page
[ci] Fix web converge
[web] Fix version menu
Installation
Docs
[docs] Added new animation for intro scheme
[docs] Better releache channel switcher
[docs] “What’s new in v1.2” documentation chapter (part 1/2)
Chapter consists of 2 articles:
- Changelog — full descriptive list of key changes since v1.1, explaining what and wh
. - How to migrate from v1.1 to v1.2 — fast guide to migrate your project from v1.1 to v1.2 without excess explanations
‘
This change is part 1 of the following changes:
- english version of changelog;
- english and russian version of migration guide and russian version of changelog (will be available in next releases).
Installation
Docs
[docs] Basic bundles article in advanced documentation section
https://werf.io/v1.2-alpha/documentation/advanced/bundles.html
Internals
Rework ls tree and path matcher interface
[ls_tree] Change the LsTree function interface
// LsTree returns the Result with tree entries that satisfy the passed pathMatcher.
// The function works lazily and does not go through a tree directory unnecessarily.
// If the result should contain only regular files (without directories and submodules), you should use the allFiles parameter.
LsTree(ctx context.Context, repository *git.Repository, commit string, pathMatcher path_matcher.PathMatcher) (*Result, error) -> LsTree(ctx context.
Context, repository *git.Repository, commit string, pathMatcher path_matcher.PathMatcher, allFiles bool) (*Result, error)
[path_matcher] Remove the greedySearch parameter and change interface
There are the following changes in the PathMatcher interface:
- Rename the MatchPath method to IsPathMatched
- Split the ProcessDirOrSubmodulePath method into ShouldGoThrough and IsDirOrSubmodulePathMatched methods
// IsPathMatched checks for a complete matching of the path
IsPathMatched(string) bool
// ShouldGoThrough indicates that the directory or submodule path is not completely matched but may include matching files among the child files.
// The method returns false if the path is completely matched.
ShouldGoThrough(string) bool
// IsDirOrSubmodulePathMatched returns true if IsPathMatched or ShouldGoThrough.
// The method returns true if there is a possibility of containing the matching files among the child files.
IsDirOrSubmodulePathMatched(string) bool
[ls_tree] Remove the strict option in LsTree function
The option allowed to run ls tree for work tree with uninitialized submodules. In current versions, we always work with the service work tree, where the submodules must be initialized.
Installation
Docs: rework advanced/helm chapter (part 1/3)
This change include only part 1 of the following steps:
- English version of advanced/helm chapter.
- Refinements for advanced/helm and new reference/helm chapter.
- Russian version of advanced/helm and reference/helm.
Advanced/helm chapter contains multiple sections:
- configuration;
- deploy process;
- releases;
— each of which contains multiple articles, structured in sideba
.
Added giterminism article into configuration section.
Installation
Fixes: giterminism and color output in CI/CD
[giterminism] Fix loose giterminism mode
Error:
unable to locate chart directory: the directory ".helm" not found in the project git repository
[tests] Giterminism Suite: actualize tests
[go.mod] github.com/werf/logboek v0.5.3
- Fix color output in CI jobs
- Update github.com/gookit/color v1.3.5 -> v1.3.7
Installation
Automigration helm 2 to 3 fixes
[helm 2 to 3] Fix: ignore resources from helm 2 release which do not exist in kubernetes
Changed order of migration steps: first set annotations and labels, then create helm 3 release.
Installation
Automigration helm 2 to 3 fixes
[helm 2 to 3] Change migration mechanics from PATCH to REPLACE
Do not patch resources to add annotations and labels, because not all resources kinds supports strategic-merge-patch.
Use simple GET, client-side merge, then REPLACE operation (as kubectl edit use).
[migrate 2 to 3] Print warning when helm 2 and helm 3 releases exists at the same time
If for some unknown and unexpected by werf migration workflow reason there is helm 2 and helm 3 releases at the moment,
then ignore helm 2 release and continue deploy process into helm 3 release, but print a WARNING about existing helm 2 release.
To disable this warning user should possible cleanup helm 2 metadata by kubectl -n kube-system delete cm RELEASE.VERSION.
Installation
New giterminism documentation!
-
https://werf.io/documentation/advanced/giterminism.html
-
https://werf.io/documentation/reference/werf_giterminism_yaml.html
-
https://werf.io/documentation/reference/werf_yaml_template_engine.html
[docs] Overview: giterminism
[docs] Advanced/Configuration/Template engine: move to reference
[giterminism] Update error messages
[docs] Advanced/Giterminism
[docs] Reference/werf-giterminism.yaml
[docs] Advanced/Configuration/Template engine
[docs] Reference/werf.yaml: refactor and update
-
change the structure of werf yaml data file
-
add get_lang_field_or_raise_error filter to work effectively with new data structure
-
update directives descriptions
[ci] Fix Documentation Validation
There was an error while trying to write to `/__w/werf/werf/docs/Gemfile.lock`.
It is likely that you need to grant write permissions for that path.
[docs] Fix table line counter does not work properly
Installation
Added support for –kube-config-base64
Ability to pass kube config in base64 encoded form using cli –kube-config-base64 or $WERF_KUBE_CONFIG_BASE64, or $WERF_KUBECONFIG_BASE64 or $KUBE_CONFIG_BASE64 or $KUBECONFIG_BASE64.
Installation
Docs
[docs] Advanced/Configuration: shell examples take precedence over ansible
Fixes and other
[ci_env] Ignore the non-existent –tagging-strategy option
Ignore only if the old form of the command call is used: source <(werf ci-env gitlab --tagging-strategy=...)
[ci_env] Fix unable to create tmp docker config based on the user’s one
Error: unable to create tmp docker config: unable to copy /home/user/.docker to /tmp/werf-docker-config-780142139: open /home/user/.docker/run/docker-cli-api.sock: no such device or address
[bundles] Do not render published chart in werf-publish and bundle-export commands
Render cmd was used only to initiate loading of chart configuration.
Eliminated render cmd, use loader directly instead.
[render] Add –validate and include-crds opts for werf-render cmd
Installation
Automigration to helm 3 in werf converge command
When werf-converge command detects existing helm 2 release with the same name it will try to render current release using helm 3 with full validation,
Then if render has been succeeded werf will migrate helm 2 release to a newly created helm 3 release. Old helm 2 release will be removed.
CAUTION!
This operation cannot be undone.
Once werf v1.2 has converted your project to helm 3, it cannot be deployed by werf v1.1 anymore. Werf v1.1 will refuse to deploy your project once helm 3 release has been created.
[config] Add fromYaml function
Available in the werf.yaml and .werf/**/*.tmpl configs.
- fromYAML(str string) (map[string]interface{}, error)
- return error if unmarshalling failed
werf.yaml
...
{{- $values := .Files.Get "werf_values.yaml" | fromYaml -}} // or fromYaml (.Files.Get "werf_values.yaml")
from: {{- $values.image.from }}
werf_values.yaml
image:
from: alpine
Internals and fixes
[git_repo, status] Fix no error handling.
[git_repo, status] Fix the use of service work tree when the repository contains submodules
Status should always be taken for the user’s work tree
Installation
Giterminism internals improvements and fixes
Added support for .gitignore into giterminism subsystem.
[giterminism] Optimize listing fs files
Skip file if:
- not accepted by giterminism config
- not changed locally
- ignored by .gitignore
- not inside an unclean submodule repository
[giterminism] Use pathMatcher for matching paths
[path_matcher] Implement StubPathMatcher that always returns false
[git_repo] ListCommitFilesWithGlob: small improvement
The file path does not get into the result if the glob is the file path without patterns
Installation
Fixes
[win][git] Fix CRLF issues and patch-does-not-apply errors
The root of the issue: werf have added file converted from LF to CRLF into built image under windows due to some local git autocrlf settings. Then werf cannot apply patch to the file with CRLF’s inside the build container.
- Fix werf to always add files into image from git without any conversions from LF to CRLF, or from CRLF to LF. Take files from the git as-is.
- Fix patches applier for files with CRLF already added into the built images for some reason (images built with an older werf version with this bug).
Installation
Fixes
[helm 2 to 3] Fix werf helm migrate2to3 command: create helm 3 release in the first place
- Migration command should create helm 3 release object because it contains managed fields of all resources.
- It is better not lost this information about managed fields, because werf-converge will not delete fields, which has been deleted for some resource in the current repo .helm/templates in such case.
[bundles] Fix “unable to create bundle: unable to write … no such file or directory”
[giterminism] Fix broken symlinks are processed incorrectly
[stapel, git] Fix adding a broken symlink does not work
[docs] Fix incorrect link to Go guide
[docs] Updating Telegram links
[docs] Fix incorrect Liquid syntax
[werf.io landing] Index page update: new intro slides
[helm][giterminism] Cache chart dependencies by hashsum of the whole .helm/Chart.lock, not digest
Other
- Temporarily revert custom docker images tags.
- Added global .Values.global.werf.name value.
-
[config] .Files.Glob fails with an error instead of a warning message if no matches found
Error: unable to load werf config: template: werfConfig:8:9: executing "werfConfig" at <.Files.Glob>: error calling Glob: {{ .Files.Glob "path" }}: no matches found - [giterminism] Optimize listing files with glob
- Check each file path for the possibility of matching child files
- Get glob prefix without patterns and use it as start directory
Internals
[tests] Giterminism Suite: disable log color
[giterminism] Refactor: WalkConfigurationFilesWithGlob returns paths relative to dir
Installation
Fixes
[bundles] Fix “unable to create bundle: unable to write … no such file or directory”
[giterminism] Fix broken symlinks are processed incorrectly
[stapel, git] Fix adding a broken symlink does not work
Installation
Giterminism and helm fixes
Cache chart dependencies by hashsum of the whole .helm/Chart.lock, not digest. This allows usage of update chart dependency when digest has not been changed.
Installation
Fixes
Fix interactive werf run session (–shell/–bash) works not properly
- Fix proxy stream data formatting enabled mode
Installation
Introduce custom build tags and migration from helm2 to helm3
-
Custom build tags.
- Add
--add-custom-tagthat can be used multiple times for build command. - Add
--use-custom-tagfor helm commands. - It is necessary to use the image name shortcut
%image%or%image_slug%in the tag format if there is more than one image in the werf config. - [giterminism] Inspect custom tags: the tags should be accepted by
cli.AllowCustomTagsdirective. - Should be built check: an image for a specified custom tag must exist and be the same as an image for related content-based tag.
- Add
-
Migration from helm 2 to helm 3.
- werf converge command will fail if it detects existing helm 2 command;
- start a migration process with the “werf helm migrate2to3 –release HELM2_RELEASE –target-namespace TARGET_HELM3_NAMESPACE” command;
- finish a migration process with “werf converge” command into specified release and namespace;
- more docs are coming soon.
Installation
Fixes
Fix interactive werf run session (–shell/–bash) works not properly
- Fix proxy stream data formatting enabled mode
Installation
Fixes
Ignore BLOB_UNKNOWN registry errors. Ignore silently without warnings until werf-cleanup does not delete these broken images.
Installation
Fixes
[git repo] Move cache and change checksum calculation lock logic
[status] Do not cache repository and submodules repositories in the result
[docs] Fix custom helm release name template docs example
[refactor] Container runtime pkg: remove unused code and small fixes
[ls_tree] Do not cache repository and submodule repositories in the result
[docs] Fix a single quote in flag description breaks styles on cli command page
- [cli] Automatic replacement of single quotes (‘) with accent graves (`) in a flag description
Installation
Fixes
[giterminism] Fix processing of an invalid submodule state
Installation
Giterminism refinements: full git-submodules support, better custom project dir support
[giterminism] Improve feedback if a related submodule is not clean or has changes
- Ignore user work tree submodule directory if a submodule “.git” directory not found
-
Return a detailed error if a work tree submodule directory is not clean:
the submodule "<submodule path>" is not clean and must be committed. Do not forget to push the current commit to the submodule remote If this commit exists only locally Details: commit: "0000000000000000000000000000000000000000" currentWorktreeCommit: "d9243a9a45b10e6b7d985322b8108897626a25a6" expectedWorktreeCommit: "d9243a9a45b10e6b7d985322b8108897626a25a6" You might also be interested in developer mode (activated with --dev option) that allows you to work with staged changes without doing redundant commits. Just use "git add <file>..." to include the changes that should be used. -
Return a detailed error if a work tree submodule directory has changes:
the submodule "<submodule path>" has modified files and these changes must be committed (do not forget to push new changes to the submodule remote) or discarded: - file1 - file2
[giterminism] Support for configuration files outside the project directory
Internals:
- [tests] Giterminism Suite: test custom project directory
- [giterminism] Refactor symlink check code
Docs
[doc] Add Rails guide
Internals
[tests] Fix daily tests, not compiling and failing tests, refactor tests precompile
Installation
Support setting initial number of replicas when HPA is active for Deployment
Set "werf.io/replicas-on-creation": NUM annotation, do not set spec.replicas field in templates explicitly.
Integration test and docs included.
Configuration go-templates changes
[config] The required function to declare a variable as required
The required function gives developers the ability to declare a value entry as required for config rendering. If the value is empty, the config will not render and will return an error message supplied by the developer.
{{ required "A valid <anything> value required!" <anything> }}
[config] the env function requires the set environment variable
- The used environment variable can be empty but must be set (ENV_NAME=””).
- Otherwise, an error
executing "werfConfig" at <env "test">: error calling env: the environment variable "<ENV_NAME>" must be setwill be returned
Fixes and other
[logging] logboek v0.5.0
- Migrate to gookit/color
- Support for colors in WIN cmd.exe
[win] Fix swith-work-tree occurs on every run due to backslash-slash issues
Internals
[tests] Cleanup old three_way_merge_patches_creator_* fixtures
[tests] Docs Suite: fix and add to default set
Installation
Giterminism refinements: full git-submodules support, better custom project dir support
[giterminism] Improve feedback if a related submodule is not clean or has changes
- Ignore user work tree submodule directory if a submodule “.git” directory not found
-
Return a detailed error if a work tree submodule directory is not clean:
the submodule "<submodule path>" is not clean and must be committed. Do not forget to push the current commit to the submodule remote If this commit exists only locally Details: commit: "0000000000000000000000000000000000000000" currentWorktreeCommit: "d9243a9a45b10e6b7d985322b8108897626a25a6" expectedWorktreeCommit: "d9243a9a45b10e6b7d985322b8108897626a25a6" You might also be interested in developer mode (activated with --dev option) that allows you to work with staged changes without doing redundant commits. Just use "git add <file>..." to include the changes that should be used. -
Return a detailed error if a work tree submodule directory has changes:
the submodule "<submodule path>" has modified files and these changes must be committed (do not forget to push new changes to the submodule remote) or discarded: - file1 - file2
[giterminism] Support for configuration files outside the project directory
Internals:
- [tests] Giterminism Suite: test custom project directory
- [giterminism] Refactor symlink check code
Docs
[doc] Add Rails guide
Internals
[tests] Fix daily tests, not compiling and failing tests, refactor tests precompile
Installation
Giterminism refinements: full support for symlinks
[giterminism] Improve feedback messages and manager logic.
- Read the configuration file from fs if not resolved path accepted by giterminism config:
- the file must exist in the project git work tree directory;
- the path must be fully accepted by giterminism config (each resolved symlink target must be accepted).
- Read the configuration file from commit:
- the path must not have any uncommitted changes locally (each symlink target).
- the file must exist in the current commit;
Improve resolve symlink failed error messages:
- fs symlink:
- `unable to read <config type>: accepted symlink "<symlink path>" check failed: too many levels of symbolic links`
- `unable to read <config type>: accepted symlink "<symlink path>" check failed: the link target "<target path>" should be also accepted by giterminism config`
- commit symlink:
- `unable to read <config type>: symlink "<symlink path>" check failed: commit tree entry "<target path>" not found in the repository`
- `unable to read <config type>: symlink "<symlink path>" check failed: the file "<target path>" must be committed`
[giterminism] Check uncommitted/untracked files by git status
[giterminism] Debug with $WERF_DEBUG_GITERMINISM_MANAGER
[giterminism] Full support for symlinks
Installation
Docs
- Update guides template and overview page.
- Correct a grammar mistake.
Installation
Docs
Preparations for guides integration:
- https://werf.io/documentation/guides.html
- https://ru.werf.io/documentation/guides.html
Installation
Fixes
[follow] Fix follow mode does not reinitialize giterminism manager
Installation
Giterminism
[giterminism] Refactoring: simplify error messages and the corresponding code
the
the uncommitted configuration found in the project directory: the
[giterminism] Fix doublestar.Match matches system-dependent path
Custom project-dir related fixes
Fix dockerfile-not-found error when running werf not from the root of the git work tree
\# docs/werf.yaml
configVersion: 1
project: docs
---
image: docs
dockerfile: ./Dockerfile
When running werf from docs folder werf’s dockerfile builder will use docs dir — project dir — as build dockerfile build context root for the
ont
ext tar archive, not git work tree root.
[project-dir] Fix helm-chart-dir related errors when running werf with custom –dir.
[project-dir] Fix .git lookup should use –dir or process cwd.
Fixes and other
[doc] Fix the “Documentation” link in the site header. Fix jekyll custom filter.
Internals
[tests] Giterminism Suite: actualize tests
[giterminism] Remove unused giterminism inspector
Installation
Fixes
[deploy] werf_image and nameless image deprecation fix
Pass nameless image as .Values.werf.nameless_image instead of .Values.werf.image.
.Values.werf.image is always map[string]string and will only contain named images names.
As nameless image is being deprecated in the v1.2 there is no “normal” way to get nameless image from values, that’s why we use “special” value .Values.werf.nameless_image for now. In the v1.3 nameless images will be completely removed.
Installation
Giterminism
werf-giterminism.yaml giterminsimConfigVersion could be number 1 or string “1”.
Helm update
Update helm to v3.5.0.
Guides has been translated to English
https://werf.io/guides/
Fixes
Fix WERF_SET_* arguments are not active when –set has been specified issue. Add ability to specify WERF_SET_* order of processing by sorting environment variables alphanumerically by the name.
https://github.com/werf/werf/issues/3128
https://github.com/werf/werf/issues/3137
Complete list of refactored list arguments:
- –set + WERF_SET_*;
- –set-string + WERF_SET_STRING_*;
- –set-file + WERF_SET_FILE_*;
- –add-label + WERF_ADD_LABEL_*;
- –add-annotation + WERF_ADD_ANNOTATION_*;
- –ssh-key + WERF_SSH_KEY_*;
- –secondary-repo + WERF_SECONDARY_REPO_*;
Process params as follows:
- Sort WERF__* environment variables alphanumerically.
- Merge all params passed by environment variables with CLI-params (CLI-params have more priority).
DEPRECATION WARNINGS
- werf_image will be removed in the v1.3;
- –loose-giterminism will be removed in the v1.2.
Installation
Giterminism
This release contains complete support for werf-giterminism.yaml.
- Read the giterminism config from git.
- Remove –non-strict-giterminism-inspection option.
- Lookup for the .git directory in the current working dir or parent directories, or use –git-work-tree option to specify work tree manually (or WERF_GIT_WORK_TREE env var).
- Inspect uncommitted build context files
- implement multi path matcher
- inspect uncommitted build context files with giterminism manager:
- dockerfile context
- stapel git mappings
- status: fix untracked files not detected
- cache git status result in git repo instance
Bundles
Add –set-docker-config-json-value option. Set .Values.dockerconfigjson value to be used in the registry Secret to pull images from the repo (https://kubernetes.io/docs/tasks/configure-pod-
container/pull-image-private-registry/#registry-secret-existing-credentials).
Other
- [deploy] Deprecate
werf_imagetemplate in favor of .Values.werf.image.NAME. - [dismiss] Do not lock release on dismiss when –with-namespace option has been specified for now.
Internals
[tests][ci] Disable container_registry_per_implementation matrix for implementations
Enable all implementations at the same time for each test
[tests] Fix bundles test: use –set-docker-config-json-value for registry authentication
[tests] Fix git-repo-not-found related errors for integration suites
Installation
More werf-giterminism.yaml options
- Refactoring and fixes:
- Update common error message
- Update not found the werf config error message
- Generalize code: readConfigurationFile, checkConfigurationFileExistence, isConfigurationFileExist
- Change configurationFilesGlob method args
- Fix error handling
- Fix and refactor optional werf.yaml handling
- Support for the config dockerfile allowUncommitted / allowUncommittedDockerignore.
- Support for the helm chart files.
- Do not wrap globes in / (/*//).
Bundles
[bundle] Add support for automatic dependencies building on bundle-apply
Internals
[tests] Giterminism Suite: config dockerfile / dockerignore tests
[tests] Giterminism Suite: helm chart files tests
Installation
Docs fixes
- Fix links to install binary package on install page according to selected channel
- Fix docs tests suite.
- Fix links in topnav.
Installation
New v1.2 deprecations
[config] Deprecate support for the nameless image
[config] Deprecate the fromArtifact directive
Fixes
[config] Fix werf config templates reading (.werf/*/.tmpl)
All files in the werf config templates directory (.werf by default) were treated as templates.
The template file must have .tmpl extension.
[cleanup] Cast a user regexp value to full match regexp
[giterminism] Cast a user regexp value to full match regexp
Small features
Added .Values.global.werf.version service value with werf cli util version.
Installation
Fixes
Fix remote git for stapel builder error: error writing … meta.json: no such file or directory
Rework ~/.werf/local_cache/git_* caches files structure.
Installation
Giterminism improvements
- Added minimal reference of
werf-giterminism.yamlconfiguration file at https://werf.io/v1.2-alpha/documentation/advanced/configuration/giterminism.html. - Added deprecation of
--loose-giterminismoption usage.
Installation
Giterminism config extended
Giterminism config reside in werf-giterminism.yaml.
-
Allow
fromLatestandbranch:giterminismConfigVersion: "1" config: stapel: allowFromLatest: true git: allowBranch: true -
Allow uncommitted config templates:
giterminismConfigVersion: "1" config: allowUncommittedTemplates: - /*/**/ - .werf/template.tmpl
Other:
- Do not work without project git repository.
- Custom config path (–config) must be in project directory.
Fixes
[cleanup] Fix goroutine stack exceeds limit
One checksum can be associated with many stages, so it is necessary to exclude the stage itself before processing linked imports
Installation
Fixes
Fix deadline-exceeded when hook job took more than 5 min:
- Fix default helm 5 min timeout => infinite.
- Fix any timeout other than default results in ‘context deadline’ error.
Installation
Introduce werf-giterminism.yaml config
Implement fine-grained giterminism config (part 1), werf-giterminism.yaml:
giterminismConfigVersion: 1
config:
stapel:
mount:
allowBuildDir:
allowFromPaths:
- PATH1
- PATH2
goTemplateRendering:
allowEnvVariables:
- VARIABLE_X
- VARIABLE_Y
- /CI_*/
dockerfile:
allowContextAddFile:
- aaa
- bbb
allowUncommitted:
- **/*
- myapp/Dockerfile
allowUncommittedDockerignoreFiles:
- **/*
- myapp/.dockerignore
WERF_LOOSE_GITERMINISM=1 still working but will be removed soon.
Fixes
Fix werf-helm-template and werf-helm-lint commands try to load kube config; fix werf-render command excess options.
Installation
Fixes
- Fix Chart.lock file not unpacked when applying bundle.
Installation
Giterminism fixes
Extended support for git-submodules for the giterminism mode, added ability to have werf.yaml (or any other configuration files) which is symlink to another file, which reside in a submodule.
Installation
Fixes
Fix “unable to create registry client” error when DOCKER_CONFIG variable is set
Installation
Fixes
Fix panic when running ‘werf helm dependency update’ in the project with secret files
Installation
New bundle commands and fix bundle registry authentication
werf bundle export— creates resulting bundle chart directory without publishing this chart into the docker registr
.werf bundle download— downloads previously published chart into the directory without applying it into the kubernete
.
Use default ~/.docker/config.json authentication data when publishing and applying bundles.
Installation
Introduce initial werf bundle apply command
Publish (no kubernetes connection needed):
cd werf-project
werf bundle publish --repo REPO --tag TAG
Apply (Kubernetes connection required, no werf project needed):
werf bundle apply --repo REPO --tag TAG
Installation
Fix error: unable to resolve docker endpoint: open …/ca.pem: no such file or directory
Installation
Support docker client compatible tls docker server settings
Support following environment variables:
- DOCKER_TLS
- DOCKER_TLS_VERIFY
- DOCKER_CERT_PATH
Installation
Introduce ‘werf bundle publish’ command
werf bundle publish --repo REPO --tag TAG
Will publish current chart of the werf project with together with built images and all needed service-values as an image into the provided REPO by the provided TAG. This command does not need a connection to the Kubernetes cluster.
Published bundle can be later installed using werf bundle apply ... command, which is not implemented yet.
Dev mode improvements
[true git] Fix SyncDevBranchWithStagedFiles handles submodules changes improperly
SyncDevBranchWithStagedFiles function creates commit but does not prepare worktree for it. Staged files may include submodule changes that require special handling
[dockerfile] Remove excess git ignored files check
Installation
Fixes and tests
[dockerfile] Fix contextAddFile checksum calculation
- Fix empty checksum
- Fix os-sensitive contextAddFile path in the checksum
[logging] Show nameless image correctly
[tests] Build/Stapel/Git: developer mode test
[tests] Dockerfile Suite: fix project name is not set
[tests] Dockerfile Suite: upd contextAddFile test
[ci] Daily Tests: fix temporary docker config directory
Installation
Fixes
- [dockerfile] Temporarily force BuildKit deactivation due to the bug in the docker client when used as a go library
- Fix ‘werf render’ uses kubernetes based lock due to typo.
Installation
Fixes
- [dev] Fix not initialized true git package.
- [stapel, dev] Fix adding dev label to the building image.
- [giterminism] Check untracked .helm/Chart.yaml exists in the chart.
Installation
Rework dev mode for stapel and dockerfile
- Werf creates temporary commits in the dev-mode.
- Files from the git index used in the dev-mode.
- Support dev-mode in conjunction with the follow-mode.
- Commit staged files to dev branch
werf-dev-<commit>. - Add werf-dev label to git stage images.
Change default werf images storage for GitLab CI/CD and GitHub Actions
- Remove
/werfrepo suffix for GitLab CI/CD; - Remove
-werfrepo suffix for GitHub Actions.
NOTE. This is breaking change, incompatible with the previous werf versions.
Other
- Fix “list event error: context cancelled” log messages.
- [web] Update metainfo in page template.
- More fixes for werf-related ci tests.
Installation
Support subcharts in the giterminism mode
Support subcharts loading from the local filesystem in the giterminism mode:
- Read committed dependencies lock file (
requirements.lockorChart.lock). - Scan local filesystem for the charts archives specified in the lock-file and use these charts in the giterminism mode.
No auto werf helm dependency build yet.
Fixes
[deploy] Fix max releases history: use default limit = 10.
[deploy] Fix dismiss –with-namespace not waiting until namespace terminated.
Other changes
[build] Remove stages_and_images read-only lock during build process
- stages_and_images read only lock is preventive measure against running cleanup and build commands at the same time;
- this lock creates unnecessary load on the synchronization server, because this lock working all the time the build process is active for each build process;
- it is mostly safe to just omit this lock completely;
- also removed unused “image” lock from the lock manager (legacy from v1.1).
- also renamed kubernetes-related stage and stage-cache locks to include project name (just for more correctness, incompatible with v1.1 change).
Installation
Determinism to giterminism
Renamed determinism to giterminism: git + determinism or “determinated by the git”.
--disable-determinism param renamed to --loose-giterminism (WERF_DISABLE_DETERMINSM to WERF_LOOSE_GITERMINISM).
Helm related changes
- Update helm to v3.4.1.
- Fix auto generation of Chart.yaml:
- Take Chart.yaml from the repository if exists.
- Override metadata.name field with werf project name from the werf.yaml.
- Set metadata.version = 1.0.0 if not set.
- Make Chart.yaml optional for all
werf helm *commands. - Fix enable –with-namespace and –with-hooks werf-dismiss options not working.
Installation
Docs
- Fix quickstart minikube setup guides for MacOS and Windows.
- Make a separate instructions list for Windows, MacOS and Linux OS in the quickstart guide.
- Require for werf dependencies installation explicitly.
Werf CI and tests
[ci] Disable ECR and GitHub Packages cleanup tests more visibly.
[ci] Enable windows self hosted runners.
[ci] K8s versions: 1.16, 1.17, 1.18, 1.19.
[tests] Config Suite: fix line breaks.
[tests] Helm Suite
Fixes
[render, windows] Fix helm does not recognize chart templates.
[windows] Fix unable to read file from local git repo.
[render] Fix comment with template source path.
Fix incorrect usage of println/printf-like functions.
Clean kubedog log from klog warnings: silence klog/v2 and klog.
Fix checking existence of secret files in chart.
[logging, debug] Disable debug docker messages.
Other
[stapel] Remove the ability to cache each instruction separately with asLayers directive.
Remove unused code.
Remove samples.
Installation
Docs
- [docs] Main page: 2100 stars by default.
- [determinism] Added basic determinism page to the advanced documentation: https://werf.io/v1.2-alpha/documentation/advanced/configuration/determinism.html (+ ru version).
Fixes
- [windows] Fix unable to rename archive (file already use by another process).
- [helm] Fix disable-determinism flag is not added to commands which lead to panics.
- Fix werf helm 3 does not respect
WERF_KUBE_*settings.
Installation
Determinism
Determinism is enabled by default since v1.2.1 version.
- All configuration and application files will be taken from the current git commit in this mode.
- Usage and existance of uncommitted files in the git work tree is restricted.
What’s new in this version:
- Pass Dockerfile builder context from the current git commit arhive, not from the current git work tree with uncommitted files.
- Use contextAddFile directive for dockerfile builder to override and add specified uncommitted or untracked files to the context from the current git work tree:
image: X
dockerfile: Dockerfile
context: vote
contextAddFile:
- app.py
- some-generated-file.tar.gz
- contextAddFile will affect checksum calculation of the dockerfile stage during building phase.
- Load all helm-related configuration (.helm/templates, –values, –secret-values, –set-file from the local git repo commit).
Other
- Refactor git patches and archives generation.
- store tmp patches and archives in the
~/.werf/local_cache/git_data/1/tmp/instead of/tmp; - store patches cache in the
~/.werf/local_cache/git_data/1/patches/; - store archives cache in the
~/.werf/local_cache/git_data/1/archives/.
- store tmp patches and archives in the
- [docs] Refactor directive summary block for stapel.
- [ci] Documentation Validation Workflow.
Fixes
- Fix get-namespace, get-release and rotate-secret-key commands panics.
- [report] Fix envfile report format
- name:
- use template
WERF_<FORMATTED_WERF_IMAGE_NAME>_DOCKER_IMAGE_NAMEinstead ofWERF_IMAGE_<FORMATTED_WERF_IMAGE_NAME>_NAME - replace charset
-/in werf image name with_
- use template
- value: set docker image name instead of werf image name
- name:
- Update kubedog: fix Job hanging case (https://github.com/werf/kubedog/pull/184), fix color disabled in CI/CD systems.
Installation
Fixes
- Fix –docker-options option cannot contain multiple spaces
- Fix getting of parentID from image manifest.
- ..
Cleanup
- Add option to keep images that were built within period. The cleaning process skips stages (and relatives) that were built within specified period. Option –keep-stages-built-within-last-n-hours (2 hours by default).
- Git-history based cleanup improvement. The keep policy imagesPerReference.last counts several stages that were built on the same commit as one.
Thus, the policy may cover more tags than expected, and this behavior is correct. - Update log levels and messages.
Determinism mode
- Enabled by default, to disable use option –disable-determinism or WERF_DISABLE_DETERMINISM env var;
mount,git.branchandfromLatestdirectives are available only when determinism is disabled;.Files.Getand.Files.Globread data only from the local git repo by default.-
Define –helm-chart-dir in the werf.yaml:
configVersion: 1 deploy: helmChartDir: .helmThere is no more
--helm-char-diroption. - Add –env support to werf.yaml config render. Use
{{ .Env }}to get an environment value which passed to the werf by –env option or WERF_ENV env var. -
Move –allow-git-shallow-clone, –git-unshallow and –git-history-synchronization options into werf.yaml. Added new gitWorktree meta section:
configVersion: 1 project: X gitWorktree: forceShallowClone: true|false (default false) allowUnshallow: true|false (default true) allowFetchOriginBranchesAndTags: true|false (default true)Disabled ci-env generation of old options. Always unshallow git clone by default.
Installation
Basic docker compose support
| Added compose config | up | down commands. |
Basic determinism mode (disabled by default for now): werf.yaml from git, helm templates from git, disable env go-templates functions
- Refactor code to use local git repo instead of filesystem to read some configs.
- Determinism mode is disabled by default for now, because this is experimental feature which is in progress now.
Refs https://github.com/werf/werf/issues/2874
Other
- [docs] Added seo meta tags.
- [build] Add envfile report format (–report-format=envfile).
Fixes
- Disable manifests cache
~/.werf/local_cache/manifestsfor local stages storage. Manifests cache for local stages storage causes problems when built image has been deleted accidentally or unintentionally by docker rmi or docker compose. - Automatically invalidate corrupted manifests cache json records.
- [cleanup] Fix invalid import metadata found.
Installation
Docs
[docs] Reform build-proces and stages-and-storage articles
- Move articles one level up.
- Rename “Images storage” article to “Stages and storage”.
- Reorganize and actualize articles.
Installation
Docs and fixes
- [docs] Remove outdated warning.
- [docs] Add last publications.
- [docs] Fix title generation.
- [docs] Fix breadcrumbs.
- [docs] Enable beta channel selected by default.
- [docs] Eliminate werf-deploy and werf-build-and-publish commands from docs.
- [docs] Advanced/Helm/Basics: replace werf deploy command with converge.
- [cleanup] Delete invalid import metadata.
Installation
[Local development] --follow param support for werf-run and werf-converge
werf converge --follow will loop on changes into current git-repository and restart converge command when a new commit has been made.
werf run --follow will loop on changes into current git-repository and restart docker container when a new commit has been made.
Fix ‘Error: UPGRADE FAILED: another operation (install/upgrade/rollback) is in progress’ error
https://github.com/werf/helm/pull/64
#
Installation
Fixes
- Disable kubernetes connection in the werf-render command. Werf-render command now does not need a connection to the Kubernetes cluster.
- Fix ‘No image specified for template’ when using nameless image.
Installation
Update helm 3 and k8s dependencies
- Helm 3: v3.2.4 to v3.4.0
- K8s deps: v0.18.6 -> v0.19.2
Fixes
- [stapel, imports] Fix import source checksum generation does not work on windows.
- [stapel, imports] Update debug message.
Installation
Fixes
- [build] Fix panic in the stapel builder.
- [cleanup] Fix resolve revision @name/subname@0.0.1 failed
Installation
Docs
Installation page: enable v1.2 beta switch.
Installation
Docs
- Added lang switch to topnav.
- Better reference werf.yaml table.
- Add publications and extra links.
- Publication: What GitOps actually is and what it is not.
- Publication: Delivering cloud-native apps to Kubernetes using werf.
- Added extra links.
- Support for custom service urls.
Fixes
- [dockerfile] Fix unable to build from scratch.
- [dockerfile, digest] Fix ENV/ARG replacement for dockefile instructions.
- [dockerfile] Fix calculation without git does not support globs with **.
Installation
Documentation improvements
- updated details widget & quickstart architecture scheme;
- index page image fixes;
- updated index presentation;
- rename signature to digest;
- fix Installation link in the top navigation menu.
[stapel] Сaching of imports by files checksum
Reimport files from artifacts only when files checksum itself has changed.
Installation
Docs
- Reworked introduction presentation: https://werf.io/introduction.html
- Updated reference werf.yaml.
Installation
Documentation
- Redesigned documentation overview page: https://werf.io/documentation/index.html
Other
- New ‘werf render’ command:
- render Kubernetes templates;
- repo param is optional: calculate stages digests when passed, build images if not exists;
- render image names stubs if no –repo has been passed.
Installation
More documentation rework
- Internals/How CI/CD integrations works: add GitHub Actions article.
- Reference/werf.yaml: small improvements
- Fix all broken links.
- Actualized README.
- Small fixes for introduction, using-with-ci-cd-systems, quickstart and documentation/overview articles.
Fixes
- [dockerfile] Fix FROM with composite ARG variable value is not resolved.
- [ci-env] Fix extra annotation with workflow run url for GitHub Actions.
Installation
More documentation rework
- Translate introduction, quickstart, installation, using-with-ci-cd-systems, reference/werf-yaml.
- Reworked sidebar for reference/cli.
Installation
Documentation refinements
- Add what-is-gitops video to the werf main page.
- Fix installation page: show v1.2 + alpha channel only.
- Partially translate to Russian installation page.
- Small changes for landing page.
Installation
Documentation improvements
- Add new publications.
- Reorganize ru.werf.io site for v1.2.
- Fix some broken links.
- Fix landing page formulations for werf.io and ru.werf.io.
Installation
Docs
Introduction and using-with-ci-cd-system articles refinements.
Installation
Fix “Object ‘Kind’ is missing”
Fix “… Object ‘Kind’ is missing …” error which occurs when templates contain empty yaml block.
Installation
Fix zero stage size log message when using --repo images storage
Installation
Docs minor fixes
- Fix documentation overview page links and remove setup-minikube article.
- Synchronize sidebar articles names and real articles names.
Installation
Docs
- the new cool introduction page with presentation;
- small fixes for English index page;
- updated Russian index page for v1.2.
Change service values format
- do not use .Values.global.werf section, use .Values.werf instead (subcharts does not need to have an access for image-related info);
- change image-related values format:
.Values.werf.image.NAMEor.Values.werf.image(for nameless images); - use
werf_imageas a main way to generate full image name.
Installation
Documentation rework release: small fixes
- installation page;
- fix broken links.
Installation
The big documentation rework release
- new landing page;
- new installation page;
- new quickstart page;
- new using with ci/cd systems page;
- new sidebar structure;
- new documentation structure and logic “from easy to complex”.
The work on documentation is still in progress. No russian version available.
Installation
- Signature to digest rename.
- Fix syncrhonization server panics.
Installation
Fixes
- Fix GitLab Container Registry delete method selection.
- Fix system ssh agent forwarding on macOS.
Installation
For usage in critical environments with tight SLA
This is the most stable channel and recommended for usage in critical environments with tight SLA. We guarantee backward compatibility between Rock-Solid releases within minor branch (1.1.x). We guarantee that Stable release should become a Rock-Solid release not earlier than 2 weeks of broad testing.
Fixes
Update kubedog: fix Job hanging case (https://github.com/werf/kubedog/pull/184), fix color disabled in CI/CD systems.
Installation
Fix documentation broken links
Also:
- [ci] Documentation Validation Workflow.
- [docs] Refactor directive summary block for stapel.
Installation
Backport fix for cli-docs broken sidebar navigation
Installation
Fixes
- [dockerfile] Fix “panic: interface conversion” while discarding newly built image.
Installation
Docs fixes
[readme] Hide code coverage badge
[docs] Configuration/Cleanup: add directives overview
[docs] Configuration/Introduction: fix adding file with shell example
Installation
Fix ‘patch does not apply’ error in github actions
"virtual-merge-commit": "true" label in the docker image of the stage was not reset to false when building a new image for regular commit (not virtual merge commit).
Installation
Added ability to use slash “/” and plus “+” signs in werf.yaml image names
Installation
Fix “@ERROR: invalid gid nogroup” on imports stage
The error occurs when artifact image does not contain “nogroup” group, which was used by default when running rsync server. For example when using “maven:3.6.3-jdk-13” as artifact base image.
Fixed by running rsync server using root group.
Installation
Fix git-related stages selection when multiple git-mappings by the same url used
https://github.com/flant/werf/issues/2407
The problem only reproduces when multiple git-mappings by the same url is used and these git-mappings use different commit.
Installation
Fix dockerInstructions stage not isolated when used after gitCache
After this release dockerInstructions signature will change if used after gitCache. Stage-not-exists in stages-storage errors may occur in this case.
Installation
Fixes
Fix gitLatestPatch signature not stable against index-line changes in git patches.
Installation
Stable channel releases are mostly safe to use and we encourage to use this channel everywhere. We guarantee backward compatibility between Early-Access releases within minor branch (1.1.x). We guarantee that Early-Access release should become Stable not earlier than 1 week after internal tests
Fixes
[v1.1] Skip blob-unknown images in cleanup properly
Installation
Fixes
Fix “Error: parsing file … failed: yaml: unmarshal errors … cannot unmarshal !!map into string”
Installation
[1.1] Helm 3 release existence check
werf-converge and werf-deploy commands will refuse to work when helm 3 release with the same name as target helm 2 release exists in the current namespace.
This check protects existing helm 3 release when your project has already been migrated from helm 2 to helm 3, but for some reason werf v1.1 tries to redeploy this release.
Installation
Fixes
Fix “Error: parsing file … failed: yaml: unmarshal errors … cannot unmarshal !!map into string”
Installation
Fixes
Update kubedog: fix Job hanging case (https://github.com/werf/kubedog/pull/184), fix color disabled in CI/CD systems.
Installation
Fix documentation broken links
Also:
- [ci] Documentation Validation Workflow.
- [docs] Refactor directive summary block for stapel.
Installation
Fix incompatible change for Dockerfile-builder introduced in v1.1.22+fix34
- [dockerfile, digest] Fix ENV/ARG replacement for dockefile instructions, fix incompatible diges change.
- [dockerfile] Fix error handling typo.
Docs
- Fix cli documentation structure for ru-version.
Installation
Fixes
- [dockerfile] Fix FROM with composite ARG variable value is not resolved.
- [ci-env] Fix extra annotation with workflow run url for GitHub Actions.
Installation
Fix artifact import error ‘@ERROR: invalid uid root’
Use 0:0 instead of root:root symbolic names in rsyncd which runs inside artifact.
Issue reproduces when artifact image does not have root line in the /etc/passwd file.
Refs https://github.com/werf/werf/issues/2762
Installation
Fixes
commit c4d8ccd9dcd8335493dbabd692d8d115841cab5e (HEAD -> master, origin/master, origin/HEAD)
Merge: d74ae250f a72139b32
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Wed Sep 23 14:37:54 2020 +0300
Merge pull request #2698 from werf/dismiss
[helm-v3] Helm-v3 support for werf-dismiss command
commit a72139b3231926b9bf0e21fec15b7b4745ddb533 (origin/dismiss, dismiss)
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Mon Sep 21 20:19:12 2020 +0300
[helm-v3] Helm-v3 support for werf-dismiss command
commit d74ae250f81f9687439c54301206cb9f47ab234c
Merge: 80ba58ce9 c8a62aef9
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:33 2020 +0300
Merge pull request #2697 from werf/cleanup_fix_runtime_error
[cleanup] Fix runtime error when MANIFEST_UNKNOWN occurs
commit 80ba58ce9a0223adec34aaf9bc964fb18a4fa717
Merge: eb77f0bff ec0bde19c
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:00 2020 +0300
Merge pull request #2696 from werf/cleanup_tests_disable_parallel_for_quay
[tests] Cleanup Suite: disable parallel for quay tests
commit eb77f0bffdb982a71ec7358a39ce77141eb95599
Merge: 5615d25b9 75bd2adf5
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:39:34 2020 +0300
Merge pull request #2695 from werf/cleanup_default_parallel_tasks_limit
[cleanup] Increase default parallel tasks limit to 10
commit ec0bde19cafb751b91b5b0bd02e7083c83f844b9 (origin/cleanup_tests_disable_parallel_for_quay)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:22:00 2020 +0100
[tests] Cleanup Suite: disable parallel for quay tests
commit c4d8ccd9dcd8335493dbabd692d8d115841cab5e (HEAD -> master, origin/master, origin/HEAD)
Merge: d74ae250f a72139b32
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Wed Sep 23 14:37:54 2020 +0300
Merge pull request #2698 from werf/dismiss
[helm-v3] Helm-v3 support for werf-dismiss command
commit a72139b3231926b9bf0e21fec15b7b4745ddb533 (origin/dismiss, dismiss)
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Mon Sep 21 20:19:12 2020 +0300
[helm-v3] Helm-v3 support for werf-dismiss command
commit d74ae250f81f9687439c54301206cb9f47ab234c
Merge: 80ba58ce9 c8a62aef9
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:33 2020 +0300
Merge pull request #2697 from werf/cleanup_fix_runtime_error
[cleanup] Fix runtime error when MANIFEST_UNKNOWN occurs
commit 80ba58ce9a0223adec34aaf9bc964fb18a4fa717
Merge: eb77f0bff ec0bde19c
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:00 2020 +0300
Merge pull request #2696 from werf/cleanup_tests_disable_parallel_for_quay
[tests] Cleanup Suite: disable parallel for quay tests
commit eb77f0bffdb982a71ec7358a39ce77141eb95599
Merge: 5615d25b9 75bd2adf5
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:39:34 2020 +0300
Merge pull request #2695 from werf/cleanup_default_parallel_tasks_limit
[cleanup] Increase default parallel tasks limit to 10
commit ec0bde19cafb751b91b5b0bd02e7083c83f844b9 (origin/cleanup_tests_disable_parallel_for_quay)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:22:00 2020 +0100
[tests] Cleanup Suite: disable parallel for quay tests
commit c8a62aef9e817b1b1904d0d22d917a4dd5c2c1ab (origin/cleanup_fix_runtime_error)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:30:34 2020 +0100
[cleanup] Fix runtime error when MANIFEST_UNKNOWN occurs
commit b4b4d3e2d01fffb0e89f81a2274800e5393d41d4
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:19:20 2020 +0100
[cleanup] Fix typo
commit 75bd2adf588ec972428d747f5a6ea6f395010a7a (origin/cleanup_default_parallel_tasks_limit)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:15:46 2020 +0100
[cleanup] Increase default parallel tasks limit to 10
Installation
Fixes
- [dockerfile] Fix “panic: interface conversion” while discarding newly built image.
Installation
Fixes
- [harbor] Fix NOT_FOUND error during synchronization client id saving.
Installation
Documentation
- Reference/Deploy into Kubernetes: reviewed documentation, rephrased
- Reference/Working with docker registries: upd
Installation
Support for Jfrog Artifactory implementation
- [docker registry] JFrog Artifactory: fix MANIFEST_INVALID: manifest
invalid error - [docs] Working with Docker registries: JFrog Artifactory
[docker registry] Handle non-existent repository for harbor implementation
[ls tree] Fix submodule processing fails when its name and path are not equal
[cleanup] Fix the certain kube context is not supported
[dockerfile] Fix handling specific status of a local uncommitted file
Fix cleanup panic “image … metadata was not found”
Documentation
- [docs] Update werf and multiwerf versions
- [docs] Hide absolute keyring path in CLI reference
Installation
Fix helm release name not slugged by default anymore
Fix deploy meta config templates defaults.
Installation
Fix helm release name not slugged by default anymore
Fix deploy meta config templates defaults.
Installation
Fix helm release name not slugged by default anymore
Fix deploy meta config templates defaults.
Installation
Fix helm release name not slugged by default anymore
Fix deploy meta config templates defaults.
Installation
Fix ‘patch does not apply’ error in github actions
"virtual-merge-commit": "true" label in the docker image of the stage was not reset to false when building a new image for regular commit (not virtual merge commit).
Installation
Fix ‘patch does not apply’ error in github actions
"virtual-merge-commit": "true" label in the docker image of the stage was not reset to false when building a new image for regular commit (not virtual merge commit).
Installation
Doc CI/CD integration fixes
- Updating supported CI systems + a few corrections.
- Fixed intro for unsupported CI systems.
- Renaming generic CI/CD integration doc.
Installation
Cleanup fixes
[cleanup] Fix UNAUTHORIZED: authentication required error with gitlab.com
[docker_registry] Autodetect gitlab.com implementation
Installation
Docs
[docs] Fixed github stats api usage, requests will be sent only once an hour
Installation
Changed https://werf.io and https://ru.werf.io landing pages!
Backport change for v1.1.16.
- [doc] Added animated presentation, introduction to werf became more comprehensible
- [doc] Added three new publications
Installation
Added ability to use slash “/” and plus “+” signs in werf.yaml image names
Installation
Fix “@ERROR: invalid gid nogroup” on imports stage
The error occurs when artifact image does not contain “nogroup” group, which was used by default when running rsync server. For example when using “maven:3.6.3-jdk-13” as artifact base image.
Fixed by running rsync server using root group.
Installation
Fix git-related stages selection when multiple git-mappings by the same url used
https://github.com/flant/werf/issues/2407
The problem only reproduces when multiple git-mappings by the same url is used and these git-mappings use different commit.
Installation
Fixes
- Fix kubernetes namespace slug: should not allow capital letters.
- Fix “@ERROR: invalid gid nogroup” on imports stage. The error occurs when artifact image does not contain “nogroup” group, which was used by default when running rsync server. For example when using “maven:3.6.3-jdk-13” as artifact base image. Fixed by running rsync server using root group.
- Fix git-related stages selection when multiple git-mappings by the same url used.
https://github.com/flant/werf/issues/2407
Installation
Fix git-related stages selection when multiple git-mappings by the same url used
https://github.com/flant/werf/issues/2407
The problem only reproduces when multiple git-mappings by the same url is used and these git-mappings use different commit.
Installation
Fix dockerInstructions stage not isolated when used after gitCache
After this release dockerInstructions signature will change if used after gitCache. Stage-not-exists in stages-storage errors may occur in this case.
Installation
Fixes
Fix gitLatestPatch signature not stable against index-line changes in git patches.
Installation
Fix build panics and errors when newly built image has been discarded
Example of error:
delete stage image fdf27403-2c05-4966-850f-dbb0c79ff577 manually and retry the build
Installation
Fix dockerfile builder
[dockerfile] Fix git status checksum algorithm does not count symlinks properly
[dockerfile] Skip outside the build context wildcard till fail on docker build
[dockerfile] Take into account ignored files by .gitignore files
[dockerfile] Change checksum calculation without git logic
- Add relative file path and mode to checksum
- Add symlink link to checksum instead of linked data
[dockerfile] Fix git status with not initialized submodules
Installation
Fix release
Fix problems related to changes introduced in v1.1.8+fix13: signatures has been accidentally changed in v1.1.8+fix13.
Fixes that were made in v1.1.8+fix13 are also available in this v1.1.8+fix14 release:
[build][git] Handle “commits not present” line in git diffs
[build][git] Fix gitLatestPatch not built when previous stage has been discarded during build
- remove git-building-stage map from Conveyor;
- remove AfterSignatureCalculated Stage hook.
[dockerfile] Fix ONBUILD instruction processing
- dockerfile stage signature should take into account ONBUILD value;
- dockerfile stage that uses another one with ONBUILD instructions should
calculate files checksum for COPY/ADD trigger instruction in addition to ONBUILD value.
Installation
Fix dockerfile builder
[dockerfile] Fix git status checksum algorithm does not count symlinks properly
[dockerfile] Skip outside the build context wildcard till fail on docker build
[dockerfile] Take into account ignored files by .gitignore files
[dockerfile] Change checksum calculation without git logic
- Add relative file path and mode to checksum
- Add symlink link to checksum instead of linked data
[dockerfile] Fix git status with not initialized submodules
Installation
Fix release
- Use 1.1 stable in README installation and docs.
- Change v1.1.8 stages-storage-cache dir to prevent bugs related to update to new v1.1.9.
Installation
Fix werf dockerfile image not rebuilds when COPY/ADD source path files changed
Fix COPY/ADD source path with trailing backslash is not counted in the signature.
Installation
For use in non critical environments or local development
These releases are mostly safe to use and can even be used in non critical environments or for local development. We do not guarantee backward compatibility between Early-Access releases.
Fixes
[logging] Build: add waiting for background tasks log message.
[cleanup] Ignore blob-unknown errors properly.
During cleanup operation do not raise should-reset-stages-storage-cache error when get stage manifest operation have failed with
blob-unknown error.
Installation
Fixes
[v1.1] Skip blob-unknown images in cleanup properly
Installation
Fixes
Automatically invalidate corrupted manifests cache json records.
Installation
Fixes
[buidler] Fix panic docker_registry.IsNameUnknownError
Installation
[v1.1] Fix saving stages-ids into the stages-storage-cache
When some stage has appeared in the repository tags list, but stage image is invalid (due to MANIFEST_UNKNOWN or BLOB_UNKNOWN container registry errors), werf ignores such stage. But this stage-id of such broken stage has been saved into the stages-storage-cache anyway, which is wrong.
Installation
Fixes
[builder] Proper rejection of blob-unknown stages
Do not reject stage when manifest-unknown or name-unknown errors detected. Only reject images with blob-unknown errors.
Installation
[v1.1] Fix saving stages-ids into the stages-storage-cache
When some stage has appeared in the repository tags list, but stage image is invalid (due to MANIFEST_UNKNOWN or BLOB_UNKNOWN container registry errors), werf ignores such stage. But this stage-id of such broken stage has been saved into the stages-storage-cache anyway, which is wrong.
Installation
[v1.1][stapel] Support for rejected stages in the stages storage
Reject stage when docker-pull have failed with the “blob unknown” error.
Add special record into the stages-storage for such rejected stages.
Other
[v1.1] Add ability to set werf-stapel image name with WERF_STAPEL_IMAGE_NAME environment variable.
Installation
[stapel] Fix: fetch artifact/image stage before running imports rsync server
Fetch operation will check that image exists correctly and will reset stages building conveyor to rebuild this stage if needed.
Installation
Fixes
[v1.1][GC] Fix StagesStorage ShouldFetch method: perform fresh docker-inspect, do not use cached inspection result from the previous inspect.
This fix particularly needed for the v1.2 host cleanup procedure which can remove freshly built and saved into the stages storage images. In such case should-fetch and stages-storage-manager fetch method will lock an image and perform standard fetch procedure.
[v1.1] Added stage image locking into stages-storage-manager to prevent v1.2 host cleanup procedure from deleting stages which are in use.
Installation
Fixes
Completely remove redudant stages_and_images read-only shared lock usage to reduce the number of requests to lock-server.
Installation
Debug release
Kubedog update: disable events informer with KUBEDOG_DISABLE_EVENTS=1
Installation
[v1.1] Helm 3 release existence checker
Add support for custom kube-context and kube-config-base64.
Installation
[config] Add fromYaml function
- fromYAML(str string) (map[string]interface{}, error)
- return error if unmarshalling failed
werf.yaml
...
{{- $values := .Files.Get "werf_values.yaml" | fromYaml -}} // or fromYaml (.Files.Get "werf_values.yaml")
from: {{- $values.image.from }}
werf_values.yaml
image:
from: alpine
Installation
[1.1] Helm 3 release existence check
werf-converge and werf-deploy commands will refuse to work when helm 3 release with the same name as target helm 2 release exists in the current namespace.
This check protects existing helm 3 release when your project has already been migrated from helm 2 to helm 3, but for some reason werf v1.1 tries to redeploy this release.
Installation
[config] Add fromYaml function
- fromYAML(str string) (map[string]interface{}, error)
- return error if unmarshalling failed
werf.yaml
...
{{- $values := .Files.Get "werf_values.yaml" | fromYaml -}} // or fromYaml (.Files.Get "werf_values.yaml")
from: {{- $values.image.from }}
werf_values.yaml
image:
from: alpine
Installation
Fixes
Do not print warnings for MANIFEST_UNKNOWN and BLOB_UNKNOWN registry errors until werf-cleanup does not delete these images automatically.
Installation
Fixes
[BLOB_UNKNOWN] Completely ignore images with BLOB_UNKNOWN errors from the standard docker registry
- Consider image is not exists in that case.
- Log warnings on BLOB_UNKNOWN and MANIFEST_UNKNOWN for default docker registry impl.
- Removed experimental WERF_EXPERIMENTAL_IGNORE_IMAGE_METADATA_BY_COMMIT_BLOB_UNKNOWN env variable.
Installation
DEPRECATION WARNING
Add deprecation warning for the fromImageArtifact directive.
Fixes
Fix WERF_SET_* arguments are not active when –set has been specified issue. Add ability to specify WERF_SET_* order of processing by sorting environment variables alphanumerically by the name.
https://github.com/werf/werf/issues/3128
https://github.com/werf/werf/issues/3137
Complete list of refactored list arguments:
- –set + WERF_SET_*;
- –set-string + WERF_SET_STRING_*;
- –set-file + WERF_SET_FILE_*;
- –add-label + WERF_ADD_LABEL_*;
- –add-annotation + WERF_ADD_ANNOTATION_*;
- –ssh-key + WERF_SSH_KEY_*;
- –secondary-repo + WERF_SECONDARY_REPO_*;
Process params as follows:
- Sort WERF__* environment variables alphanumerically.
- Merge all params passed by environment variables with CLI-params (CLI-params have more priority).
Installation
Fixes
Fix “Error: parsing file … failed: yaml: unmarshal errors … cannot unmarshal !!map into string”
Installation
Fixes
Update kubedog: fix Job hanging case (https://github.com/werf/kubedog/pull/184), fix color disabled in CI/CD systems.
Installation
Fix documentation broken links
Also:
- [ci] Documentation Validation Workflow.
- [docs] Refactor directive summary block for stapel.
Installation
Fix “docker: invalid reference format: repository name must be lowercase”
Fix –docker-options option cannot contain multiple spaces.
Installation
Fix incompatible change for Dockerfile-builder introduced in v1.1.22+fix34
- [dockerfile, digest] Fix ENV/ARG replacement for dockefile instructions, fix incompatible diges change.
- [dockerfile] Fix error handling typo.
Docs
- Fix cli documentation structure for ru-version.
Installation
Fixes
- [dockerfile] Fix FROM with composite ARG variable value is not resolved.
- [ci-env] Fix extra annotation with workflow run url for GitHub Actions.
Installation
Fixes
- [dockerfile] Fix unable to build from scratch.
- [dockerfile, digest] Fix ENV/ARG replacement for dockefile instructions.
Installation
Fixes
- [dockerfile] Fix FROM with composite ARG variable value is not resolved.
- [ci-env] Fix extra annotation with workflow run url for GitHub Actions.
Installation
Fix artifact import error ‘@ERROR: invalid uid root’
Use 0:0 instead of root:root symbolic names in rsyncd which runs inside artifact.
Issue reproduces when artifact image does not have root line in the /etc/passwd file.
Refs https://github.com/werf/werf/issues/2762
Installation
Fixes
- Fix system ssh agent forwarding on macOS.
- Fix GitLab Container Registry delete method selection.
Installation
Fixes
commit c4d8ccd9dcd8335493dbabd692d8d115841cab5e (HEAD -> master, origin/master, origin/HEAD)
Merge: d74ae250f a72139b32
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Wed Sep 23 14:37:54 2020 +0300
Merge pull request #2698 from werf/dismiss
[helm-v3] Helm-v3 support for werf-dismiss command
commit a72139b3231926b9bf0e21fec15b7b4745ddb533 (origin/dismiss, dismiss)
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Mon Sep 21 20:19:12 2020 +0300
[helm-v3] Helm-v3 support for werf-dismiss command
commit d74ae250f81f9687439c54301206cb9f47ab234c
Merge: 80ba58ce9 c8a62aef9
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:33 2020 +0300
Merge pull request #2697 from werf/cleanup_fix_runtime_error
[cleanup] Fix runtime error when MANIFEST_UNKNOWN occurs
commit 80ba58ce9a0223adec34aaf9bc964fb18a4fa717
Merge: eb77f0bff ec0bde19c
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:00 2020 +0300
Merge pull request #2696 from werf/cleanup_tests_disable_parallel_for_quay
[tests] Cleanup Suite: disable parallel for quay tests
commit eb77f0bffdb982a71ec7358a39ce77141eb95599
Merge: 5615d25b9 75bd2adf5
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:39:34 2020 +0300
Merge pull request #2695 from werf/cleanup_default_parallel_tasks_limit
[cleanup] Increase default parallel tasks limit to 10
commit ec0bde19cafb751b91b5b0bd02e7083c83f844b9 (origin/cleanup_tests_disable_parallel_for_quay)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:22:00 2020 +0100
[tests] Cleanup Suite: disable parallel for quay tests
commit c4d8ccd9dcd8335493dbabd692d8d115841cab5e (HEAD -> master, origin/master, origin/HEAD)
Merge: d74ae250f a72139b32
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Wed Sep 23 14:37:54 2020 +0300
Merge pull request #2698 from werf/dismiss
[helm-v3] Helm-v3 support for werf-dismiss command
commit a72139b3231926b9bf0e21fec15b7b4745ddb533 (origin/dismiss, dismiss)
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Mon Sep 21 20:19:12 2020 +0300
[helm-v3] Helm-v3 support for werf-dismiss command
commit d74ae250f81f9687439c54301206cb9f47ab234c
Merge: 80ba58ce9 c8a62aef9
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:33 2020 +0300
Merge pull request #2697 from werf/cleanup_fix_runtime_error
[cleanup] Fix runtime error when MANIFEST_UNKNOWN occurs
commit 80ba58ce9a0223adec34aaf9bc964fb18a4fa717
Merge: eb77f0bff ec0bde19c
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:00 2020 +0300
Merge pull request #2696 from werf/cleanup_tests_disable_parallel_for_quay
[tests] Cleanup Suite: disable parallel for quay tests
commit eb77f0bffdb982a71ec7358a39ce77141eb95599
Merge: 5615d25b9 75bd2adf5
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:39:34 2020 +0300
Merge pull request #2695 from werf/cleanup_default_parallel_tasks_limit
[cleanup] Increase default parallel tasks limit to 10
commit ec0bde19cafb751b91b5b0bd02e7083c83f844b9 (origin/cleanup_tests_disable_parallel_for_quay)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:22:00 2020 +0100
[tests] Cleanup Suite: disable parallel for quay tests
commit c8a62aef9e817b1b1904d0d22d917a4dd5c2c1ab (origin/cleanup_fix_runtime_error)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:30:34 2020 +0100
[cleanup] Fix runtime error when MANIFEST_UNKNOWN occurs
commit b4b4d3e2d01fffb0e89f81a2274800e5393d41d4
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:19:20 2020 +0100
[cleanup] Fix typo
commit 75bd2adf588ec972428d747f5a6ea6f395010a7a (origin/cleanup_default_parallel_tasks_limit)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:15:46 2020 +0100
[cleanup] Increase default parallel tasks limit to 10
Installation
Fixes and docs
- [dockerfile] Fix “panic: interface conversion” while discarding newly built image.
- [docs] a hyphen for low-level.
Git-history based cleanup
- [cleanup] Git history-based algorithm: fix keep policies with many references might work too slow.
- [logging] Stages cleanup: print skipped stages by ignore period policy.
- [cleanup] Git history-based algorithm: skip content signatures without related images.
Helm-v3 based deploy
- Use helm-v3 in
werf deploycommand whenWERF_DEPLOY_HELM_V3=1is set.- Supported features:
- extra labels/annotations;
- distributed release locking using kubernetes to prevent simultaneous deploy of the same release;
- kubedog-based deploy tracking;
- autogeneration of Chart.yaml based on the werf.yaml when Chart.yaml file is not present in
.helmdir; - secret values; werf does not write secret values into the release data, which will be accessible by
werf helm-v3 get all RELEASE.
- Supported features:
- Lock release by name in werf helm-v3 install/upgrade and werf deploy commands.
- Unsupported yet features:
- adoption of resources by werf.io annotation;
- auto migration of existing helm-v2 release to helm-v3;
werf dismisscommand should use helm-v3 and wait until resources deleted;- extra values and template-helpers to generate full werf-images names in the chart templates.
Installation
More helm-v3 support
- Secret values support for
werf helm-v3subcommands.
Cleanup
- [cleanup] Cleanup meta images
- Reject lazy scanning: check each policy references even it is not needed
- Keep a minimum number of commit hashes in stages storage preserving only the latest commit for each content signature in each policy references
- [tests] Cleanup Suite: init stages storage docker registry options.
- [tests] Cleanup Suite: meta images cleanup
Docs
- [docs] Add new publication: “Werf: Fully customizable GitOps”
Installation
Fixes
- [logging] Ignore klog reflector ERROR messages.
- [logging] More details in concurrent builds plan.
- [harbor] Fix NOT_FOUND error during synchronization client id saving.
- [build] Fix getting inspect for built stage.
Installation
Documentation
- Reference/Deploy into Kubernetes: reviewed documentation, rephrased
- Reference/Working with docker registries: upd
Installation
Support for Jfrog Artifactory implementation
- [docker registry] JFrog Artifactory: fix MANIFEST_INVALID: manifest
invalid error - [docs] Working with Docker registries: JFrog Artifactory
[docker registry] Handle non-existent repository for harbor implementation
[ls tree] Fix submodule processing fails when its name and path are not equal
[cleanup] Fix the certain kube context is not supported
[dockerfile] Fix handling specific status of a local uncommitted file
Fix cleanup panic “image … metadata was not found”
Documentation
- [docs] Update werf and multiwerf versions
- [docs] Hide absolute keyring path in CLI reference
Installation
Docs fixes
[readme] Hide code coverage badge
[docs] Configuration/Cleanup: add directives overview
[docs] Configuration/Introduction: fix adding file with shell example
Installation
Fix helm release name not slugged by default anymore
Fix deploy meta config templates defaults.
Installation
Fix ‘patch does not apply’ error in github actions
"virtual-merge-commit": "true" label in the docker image of the stage was not reset to false when building a new image for regular commit (not virtual merge commit).
Installation
Fix ‘patch does not apply’ error in github actions
"virtual-merge-commit": "true" label in the docker image of the stage was not reset to false when building a new image for regular commit (not virtual merge commit).
Installation
Fix werf cleanup slow deletion of images
[cleanup] Gitlab registry implementation: use deleting a tag method instead of an image
More details in gitlab-org/container-registry documentation:
- https://gitlab.com/gitlab-org/container-registry/-/blob/release/2.9-gitlab/docs/spec/api.md#deleting-a-tag
- https://gitlab.com/gitlab-org/container-registry/-/blob/release/2.9-gitlab/docs/spec/api.md#deleting-an-image
Installation
Docs translation and fixes
Git history based cleanup related doc articles has been translated.
Installation
Doc CI/CD integration fixes
- Updating supported CI systems + a few corrections.
- Fixed intro for unsupported CI systems.
- Renaming generic CI/CD integration doc.
Installation
Shallow clone related fixes
Do origin fetch only when git shallow clone is detected. The complete commit history is enough for all commands except images cleanup.
Installation
Cleanup fixes
[cleanup] Fix UNAUTHORIZED: authentication required error with gitlab.com
[docker_registry] Autodetect gitlab.com implementation
Installation
Fix distributed werf hangs sometimes
The hanging case was in the lockgate library, described in the https://github.com/werf/lockgate/pull/24.
Installation
[git] Do not use ‘git worktree add –no-checkout’ option
This option is not avaiable in the v2.17.1 version of git, which is the latest avaiable version for 18.04 lts ubuntu.
Specify target commit instead (which is by the way a faster and more precise approach).
Installation
Docs
[docs] Fixed github stats api usage, requests will be sent only once an hour
Installation
Changed https://werf.io and https://ru.werf.io landing pages!
Backport change for v1.1.16.
- [doc] Added animated presentation, introduction to werf became more comprehensible
- [doc] Added three new publications
Installation
[config] Add .Files.Glob Go template function
{{ range $path, $content := .Files.Glob ".werf/files/**" }}
{{ $path }}: |
{{ $content | indent 2 }}
{{ end }}
-
Process symlinks with .Files.Glob
.Files.Glob returns the hash of regular files and their contents for the paths that are matched pattern. This function follows only symlinks pointed to a regular file (not to a directory). -
.Files.Glob: has an ability to merge results
The function supports shell pattern matching +**. Results can be merged withmergesprig function (e.g{{ $filesDict := merge (.Files.Glob "*/*.txt") (.Files.Glob "app/ **/*.txt") }})
Installation
Fixes
[ci_env] GitHub: fix typo os.Getenv(“IMAGES REPO”)
[cli] Fix –release and –namespace params: add WERF_RELEASE and WERF_NAMESPACE options
Installation
Fix hard-coded images repo/stages storage implementation for ci-env
Automatically set gitlab docker registry implementation for stages storage and images repo.
Installation
Added ability to use slash “/” and plus “+” signs in werf.yaml image names
Installation
Docs
- Guides/GitHub CI/CD integration.
- Add distributed-werf reference info and switch to distributed guide.
Installation
Fixes
- Fix kubernetes namespace slug: should not allow capital letters.
- Fix git-related stages selection when multiple git-mappings by the same url used.
https://github.com/flant/werf/issues/2407
Installation
Fix v1.1 stage selection algorithm
Fix for a case when two images of the same stage for two consecutive commits are building at the same time. This can lead to the bug, because werf uses oldest stage by creation timestamp and not oldest by save-into-stages-storage timestamp.
Use selection based on saving-into-stages-storage-timestamp instead of docker-image creation timestamp.
After this change stages-storage-cache will be invalidated automatically.
Installation
Fixes
- Fix kubernetes namespace slug: should not allow capital letters.
- Fix git-related stages selection when multiple git-mappings by the same url used.
https://github.com/flant/werf/issues/2407
Installation
Fix v1.1 stage selection algorithm
Fix for a case when two images of the same stage for two consecutive commits are building at the same time. This can lead to the bug, because werf uses oldest stage by creation timestamp and not oldest by save-into-stages-storage timestamp.
Use selection based on saving-into-stages-storage-timestamp instead of docker-image creation timestamp.
After this change stages-storage-cache will be invalidated automatically.
Installation
Fix “@ERROR: invalid gid nogroup” on imports stage
The error occurs when artifact image does not contain “nogroup” group, which was used by default when running rsync server. For example when using “maven:3.6.3-jdk-13” as artifact base image.
Fixed by running rsync server using root group.
Retry docker pull/push on internal server error
Werf automatically retries pull/push operations on internal server errors.
Installation
Fix dockerInstructions stage not isolated when used after gitCache
After this release dockerInstructions signature will change if used after gitCache. Stage-not-exists in stages-storage errors may occur in this case.
Installation
Fix cleanup deletes stages that are related to deployed images
Installation
Remote storages support
Auto reset stages-storage-cache and retry action when docker-image has been deleted from stages-storage, but exists in the stages-storage-cache.
Fixes
Fix gitLatestPatch signature not stable to index line in patches.
Installation
Fixes
Fix gitLatestPatch signature not stable against index-line changes in git patches.
Installation
Fix build panics and errors when newly built image has been discarded
Example of error:
delete stage image fdf27403-2c05-4966-850f-dbb0c79ff577 manually and retry the build
Installation
Fix dockerfile builder
[dockerfile] Fix git status checksum algorithm does not count symlinks properly
[dockerfile] Skip outside the build context wildcard till fail on docker build
[dockerfile] Take into account ignored files by .gitignore files
[dockerfile] Change checksum calculation without git logic
- Add relative file path and mode to checksum
- Add symlink link to checksum instead of linked data
[dockerfile] Fix git status with not initialized submodules
Installation
Fix release
Fix problems related to changes introduced in v1.1.8+fix13: signatures has been accidentally changed in v1.1.8+fix13.
Fixes that were made in v1.1.8+fix13 are also available in this v1.1.8+fix14 release:
[build][git] Handle “commits not present” line in git diffs
[build][git] Fix gitLatestPatch not built when previous stage has been discarded during build
- remove git-building-stage map from Conveyor;
- remove AfterSignatureCalculated Stage hook.
[dockerfile] Fix ONBUILD instruction processing
- dockerfile stage signature should take into account ONBUILD value;
- dockerfile stage that uses another one with ONBUILD instructions should
calculate files checksum for COPY/ADD trigger instruction in addition to ONBUILD value.
Installation
Fix dockerfile builder
[dockerfile] Fix git status checksum algorithm does not count symlinks properly
[dockerfile] Skip outside the build context wildcard till fail on docker build
[dockerfile] Take into account ignored files by .gitignore files
[dockerfile] Change checksum calculation without git logic
- Add relative file path and mode to checksum
- Add symlink link to checksum instead of linked data
[dockerfile] Fix git status with not initialized submodules
Installation
Fix release
- Use 1.1 stable in README installation and docs.
- Change v1.1.8 stages-storage-cache dir to prevent bugs related to update to new v1.1.9.
Installation
Fix werf dockerfile image not rebuilds when COPY/ADD source path files changed
Fix COPY/ADD source path with trailing backslash is not counted in the signature.
Installation
INCOMPATIBLE CHANGE Fix stages and images isolation problem for git-latest-patch and stages-signature
-
gitLatestPatch stage was not isolated between different branches when the same changes
has been made in two different commits: added commits ancestry check when selecting
suitable stage from cache for gitLatestPatch. The signature of gitLatestPatch stage
has not been changed. -
Stages-signature was not isolated between different branches when the same
changes has been maed in two different commits. IMPORTANT stages-signature
calculation algorithm has been changed, stages-signature will change for all projects
that use stages-signature tagging strategy (content-based).
Installation
[dockerfile] Fix dockerfile stage name from “COPY –from” is not detected
Docker parser coverts dockerfile stage name (AS) to lowercase but in places of the name usage (COPY –from) does not.
This affected dockerfile stage signature calculation and occurred false warning messages.
Installation
For more broad testing of new features
These releases are for more broad testing of new features to catch regressions. We do not guarantee backward compatibility between Beta releases.
Fixes
[logging] Build: add waiting for background tasks log message.
[cleanup] Ignore blob-unknown errors properly.
During cleanup operation do not raise should-reset-stages-storage-cache error when get stage manifest operation have failed with
blob-unknown error.
Installation
Fixes
[v1.1] Skip blob-unknown images in cleanup properly
Installation
Fixes
Automatically invalidate corrupted manifests cache json records.
Installation
Fixes
[buidler] Fix panic docker_registry.IsNameUnknownError
Installation
[v1.1] Fix saving stages-ids into the stages-storage-cache
When some stage has appeared in the repository tags list, but stage image is invalid (due to MANIFEST_UNKNOWN or BLOB_UNKNOWN container registry errors), werf ignores such stage. But this stage-id of such broken stage has been saved into the stages-storage-cache anyway, which is wrong.
Installation
Fixes
[builder] Proper rejection of blob-unknown stages
Do not reject stage when manifest-unknown or name-unknown errors detected. Only reject images with blob-unknown errors.
Installation
[v1.1] Fix saving stages-ids into the stages-storage-cache
When some stage has appeared in the repository tags list, but stage image is invalid (due to MANIFEST_UNKNOWN or BLOB_UNKNOWN container registry errors), werf ignores such stage. But this stage-id of such broken stage has been saved into the stages-storage-cache anyway, which is wrong.
Installation
[v1.1][stapel] Support for rejected stages in the stages storage
Reject stage when docker-pull have failed with the “blob unknown” error.
Add special record into the stages-storage for such rejected stages.
Other
[v1.1] Add ability to set werf-stapel image name with WERF_STAPEL_IMAGE_NAME environment variable.
Installation
[stapel] Fix: fetch artifact/image stage before running imports rsync server
Fetch operation will check that image exists correctly and will reset stages building conveyor to rebuild this stage if needed.
Installation
Fixes
[v1.1][GC] Fix StagesStorage ShouldFetch method: perform fresh docker-inspect, do not use cached inspection result from the previous inspect.
This fix particularly needed for the v1.2 host cleanup procedure which can remove freshly built and saved into the stages storage images. In such case should-fetch and stages-storage-manager fetch method will lock an image and perform standard fetch procedure.
[v1.1] Added stage image locking into stages-storage-manager to prevent v1.2 host cleanup procedure from deleting stages which are in use.
Installation
Fixes
Completely remove redudant stages_and_images read-only shared lock usage to reduce the number of requests to lock-server.
Installation
Debug release
Kubedog update: disable events informer with KUBEDOG_DISABLE_EVENTS=1
Installation
[v1.1] Helm 3 release existence checker
Add support for custom kube-context and kube-config-base64.
Installation
[config] Add fromYaml function
- fromYAML(str string) (map[string]interface{}, error)
- return error if unmarshalling failed
werf.yaml
...
{{- $values := .Files.Get "werf_values.yaml" | fromYaml -}} // or fromYaml (.Files.Get "werf_values.yaml")
from: {{- $values.image.from }}
werf_values.yaml
image:
from: alpine
Installation
[1.1] Helm 3 release existence check
werf-converge and werf-deploy commands will refuse to work when helm 3 release with the same name as target helm 2 release exists in the current namespace.
This check protects existing helm 3 release when your project has already been migrated from helm 2 to helm 3, but for some reason werf v1.1 tries to redeploy this release.
Installation
[config] Add fromYaml function
- fromYAML(str string) (map[string]interface{}, error)
- return error if unmarshalling failed
werf.yaml
...
{{- $values := .Files.Get "werf_values.yaml" | fromYaml -}} // or fromYaml (.Files.Get "werf_values.yaml")
from: {{- $values.image.from }}
werf_values.yaml
image:
from: alpine
Installation
Fixes
Do not print warnings for MANIFEST_UNKNOWN and BLOB_UNKNOWN registry errors until werf-cleanup does not delete these images automatically.
Installation
Fixes
[BLOB_UNKNOWN] Completely ignore images with BLOB_UNKNOWN errors from the standard docker registry
- Consider image is not exists in that case.
- Log warnings on BLOB_UNKNOWN and MANIFEST_UNKNOWN for default docker registry impl.
- Removed experimental WERF_EXPERIMENTAL_IGNORE_IMAGE_METADATA_BY_COMMIT_BLOB_UNKNOWN env variable.
Installation
DEPRECATION WARNING
Add deprecation warning for the fromImageArtifact directive.
Fixes
Fix WERF_SET_* arguments are not active when –set has been specified issue. Add ability to specify WERF_SET_* order of processing by sorting environment variables alphanumerically by the name.
https://github.com/werf/werf/issues/3128
https://github.com/werf/werf/issues/3137
Complete list of refactored list arguments:
- –set + WERF_SET_*;
- –set-string + WERF_SET_STRING_*;
- –set-file + WERF_SET_FILE_*;
- –add-label + WERF_ADD_LABEL_*;
- –add-annotation + WERF_ADD_ANNOTATION_*;
- –ssh-key + WERF_SSH_KEY_*;
- –secondary-repo + WERF_SECONDARY_REPO_*;
Process params as follows:
- Sort WERF__* environment variables alphanumerically.
- Merge all params passed by environment variables with CLI-params (CLI-params have more priority).
Installation
Small features
Added .Values.global.werf.version service value with werf cli util version.
Installation
Fixes
Fix redundant stapel container initialization for dockerfile image
Installation
Fixes
Fix “Error: parsing file … failed: yaml: unmarshal errors … cannot unmarshal !!map into string”
Installation
Fixes
Update kubedog: fix Job hanging case (https://github.com/werf/kubedog/pull/184), fix color disabled in CI/CD systems.
Installation
Fix documentation broken links
Also:
- [ci] Documentation Validation Workflow.
- [docs] Refactor directive summary block for stapel.
Installation
Fix “docker: invalid reference format: repository name must be lowercase”
Fix –docker-options option cannot contain multiple spaces.
Installation
Fix ‘Error: UPGRADE FAILED: another operation (install/upgrade/rollback) is in progress’ error
https://github.com/werf/helm/pull/61
Installation
Fix incompatible change for Dockerfile-builder introduced in v1.1.22+fix34
- [dockerfile, digest] Fix ENV/ARG replacement for dockefile instructions, fix incompatible diges change.
- [dockerfile] Fix error handling typo.
Docs
- Fix cli documentation structure for ru-version.
Installation
Fixes
- [dockerfile] Fix FROM with composite ARG variable value is not resolved.
- [ci-env] Fix extra annotation with workflow run url for GitHub Actions.
Installation
Fixes
- [dockerfile] Fix unable to build from scratch.
- [dockerfile, digest] Fix ENV/ARG replacement for dockefile instructions.
Installation
Fixes
- [dockerfile] Fix FROM with composite ARG variable value is not resolved.
- [ci-env] Fix extra annotation with workflow run url for GitHub Actions.
Installation
Fix publish report related panic ‘fatal error: concurrent map writes’
Installation
Fix artifact import error ‘@ERROR: invalid uid root’
Use 0:0 instead of root:root symbolic names in rsyncd which runs inside artifact.
Issue reproduces when artifact image does not have root line in the /etc/passwd file.
Refs https://github.com/werf/werf/issues/2762
Installation
Fixes
- Fix system ssh agent forwarding on macOS.
- Fix GitLab Container Registry delete method selection.
Installation
Fixes
commit c4d8ccd9dcd8335493dbabd692d8d115841cab5e (HEAD -> master, origin/master, origin/HEAD)
Merge: d74ae250f a72139b32
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Wed Sep 23 14:37:54 2020 +0300
Merge pull request #2698 from werf/dismiss
[helm-v3] Helm-v3 support for werf-dismiss command
commit a72139b3231926b9bf0e21fec15b7b4745ddb533 (origin/dismiss, dismiss)
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Mon Sep 21 20:19:12 2020 +0300
[helm-v3] Helm-v3 support for werf-dismiss command
commit d74ae250f81f9687439c54301206cb9f47ab234c
Merge: 80ba58ce9 c8a62aef9
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:33 2020 +0300
Merge pull request #2697 from werf/cleanup_fix_runtime_error
[cleanup] Fix runtime error when MANIFEST_UNKNOWN occurs
commit 80ba58ce9a0223adec34aaf9bc964fb18a4fa717
Merge: eb77f0bff ec0bde19c
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:00 2020 +0300
Merge pull request #2696 from werf/cleanup_tests_disable_parallel_for_quay
[tests] Cleanup Suite: disable parallel for quay tests
commit eb77f0bffdb982a71ec7358a39ce77141eb95599
Merge: 5615d25b9 75bd2adf5
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:39:34 2020 +0300
Merge pull request #2695 from werf/cleanup_default_parallel_tasks_limit
[cleanup] Increase default parallel tasks limit to 10
commit ec0bde19cafb751b91b5b0bd02e7083c83f844b9 (origin/cleanup_tests_disable_parallel_for_quay)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:22:00 2020 +0100
[tests] Cleanup Suite: disable parallel for quay tests
commit c4d8ccd9dcd8335493dbabd692d8d115841cab5e (HEAD -> master, origin/master, origin/HEAD)
Merge: d74ae250f a72139b32
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Wed Sep 23 14:37:54 2020 +0300
Merge pull request #2698 from werf/dismiss
[helm-v3] Helm-v3 support for werf-dismiss command
commit a72139b3231926b9bf0e21fec15b7b4745ddb533 (origin/dismiss, dismiss)
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Mon Sep 21 20:19:12 2020 +0300
[helm-v3] Helm-v3 support for werf-dismiss command
commit d74ae250f81f9687439c54301206cb9f47ab234c
Merge: 80ba58ce9 c8a62aef9
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:33 2020 +0300
Merge pull request #2697 from werf/cleanup_fix_runtime_error
[cleanup] Fix runtime error when MANIFEST_UNKNOWN occurs
commit 80ba58ce9a0223adec34aaf9bc964fb18a4fa717
Merge: eb77f0bff ec0bde19c
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:00 2020 +0300
Merge pull request #2696 from werf/cleanup_tests_disable_parallel_for_quay
[tests] Cleanup Suite: disable parallel for quay tests
commit eb77f0bffdb982a71ec7358a39ce77141eb95599
Merge: 5615d25b9 75bd2adf5
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:39:34 2020 +0300
Merge pull request #2695 from werf/cleanup_default_parallel_tasks_limit
[cleanup] Increase default parallel tasks limit to 10
commit ec0bde19cafb751b91b5b0bd02e7083c83f844b9 (origin/cleanup_tests_disable_parallel_for_quay)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:22:00 2020 +0100
[tests] Cleanup Suite: disable parallel for quay tests
commit c8a62aef9e817b1b1904d0d22d917a4dd5c2c1ab (origin/cleanup_fix_runtime_error)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:30:34 2020 +0100
[cleanup] Fix runtime error when MANIFEST_UNKNOWN occurs
commit b4b4d3e2d01fffb0e89f81a2274800e5393d41d4
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:19:20 2020 +0100
[cleanup] Fix typo
commit 75bd2adf588ec972428d747f5a6ea6f395010a7a (origin/cleanup_default_parallel_tasks_limit)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:15:46 2020 +0100
[cleanup] Increase default parallel tasks limit to 10
Installation
Parallelize operations with docker registry
- parallel read/delete stages from stages-storage operations;
- parallel read/delete images info from images-repo operations;
- image metadata cache;
- add –parallel and –parallel-tasks-limit options for cleanup (by default, parallelization is enabled and tasks limit is 5).
Other fixes
- Do not lock cleanup process by images-repo.
- Push image with retries when specific errors occur.
- http2: server sent GOAWAY and closed the connection
- http2: Transport received Server’s graceful shutdown GOAWAY
- REDACTED: UNKNOWN
Helm-v3 support
Add support for werf helpers templates (werf_container_image, werf_container_env) and service values for werf-deploy command when WERF_DEPLOY_HELM_V3=1 is set.
Installation
Fixes and docs
- [dockerfile] Fix “panic: interface conversion” while discarding newly built image.
- [docs] a hyphen for low-level.
Git-history based cleanup
- [cleanup] Git history-based algorithm: fix keep policies with many references might work too slow.
- [logging] Stages cleanup: print skipped stages by ignore period policy.
- [cleanup] Git history-based algorithm: skip content signatures without related images.
Helm-v3 based deploy
- Use helm-v3 in
werf deploycommand whenWERF_DEPLOY_HELM_V3=1is set.- Supported features:
- extra labels/annotations;
- distributed release locking using kubernetes to prevent simultaneous deploy of the same release;
- kubedog-based deploy tracking;
- autogeneration of Chart.yaml based on the werf.yaml when Chart.yaml file is not present in
.helmdir; - secret values; werf does not write secret values into the release data, which will be accessible by
werf helm-v3 get all RELEASE.
- Supported features:
- Lock release by name in werf helm-v3 install/upgrade and werf deploy commands.
- Unsupported yet features:
- adoption of resources by werf.io annotation;
- auto migration of existing helm-v2 release to helm-v3;
werf dismisscommand should use helm-v3 and wait until resources deleted;- extra values and template-helpers to generate full werf-images names in the chart templates.
Installation
More helm-v3 support
- Secret values support for
werf helm-v3subcommands.
Cleanup
- [cleanup] Cleanup meta images
- Reject lazy scanning: check each policy references even it is not needed
- Keep a minimum number of commit hashes in stages storage preserving only the latest commit for each content signature in each policy references
- [tests] Cleanup Suite: init stages storage docker registry options.
- [tests] Cleanup Suite: meta images cleanup
Docs
- [docs] Add new publication: “Werf: Fully customizable GitOps”
Installation
Fixes
- [logging] Ignore klog reflector ERROR messages.
- [logging] More details in concurrent builds plan.
- [harbor] Fix NOT_FOUND error during synchronization client id saving.
- [build] Fix getting inspect for built stage.
Installation
Fixes
- [dockerfile] Handle unsupported Media Type (especially v1 manifest version).
- Image metadata by commit: long image name support.
Installation
Logging fixes
- Fix klog warning messages: suppress klog info/warning log levels, print error level through logboek.
- Fix empty size info message for built stage.
- Cleanup logging: hide empty block.
- Fix collapsible GitLab log section:
- apply only for top-level processes;
- disable for blocks.
Installation
Fixes
- [dockerfile] Handle unsupported Media Type (especially v1 manifest version).
- Image metadata by commit: long image name support.
Installation
Helmfile support and fixes
Werf now can be used as helm-3 replacement with helmfile as follows:
-
Install latest helmfile.
-
Install helm-diff plugin (which is needed for the helmfile):
source $(multiwerf use 1.1 alpha --as-file)
werf helm-v3 plugin install https://github.com/databus23/helm-diff
- Prepare helmfile.yaml:
repositories:
- name: stable
url: https://kubernetes-charts.storage.googleapis.com
releases:
- name: mytraefik
namespace: aaa
createNamespace: true
wait: true
chart: stable/traefik
labels:
name: traefik-public
version: 1.86.2
- Set environment variable
WERF_HELM3_MODE=1— this will make werf binary CLI interface helm-compatible, then runhelmfilewith `–helm-binary=werf
.
source $(multiwerf use 1.1 alpha --as-file)
export WERF_HELM3_MODE=1
helmfile -b werf apply
- Check installed release with werf:
werf list --namespace aaa
NOTE Helmfile does not support live output from helm binary, so werf rich live output will be available only after werf command is done. However werf output with --wait flag is more useful than helm’s output even in that condition and werf will “fail fast” when something goes wrong during deploy process.
Refs https://github.com/werf/werf/issues/1606
Refs https://github.com/werf/helm/pull/50
Installation
Parallel builds fixes
- Maximum 5 parallel tasks by default.
- Fix index out of range [0] with length 0.
Installation
Documentation
- Reference/Deploy into Kubernetes: reviewed documentation, rephrased
- Reference/Working with docker registries: upd
Installation
[build] Fix makeslice len out of range (without images in werf.yaml)
Installation
Support for Jfrog Artifactory implementation
- [docker registry] JFrog Artifactory: fix MANIFEST_INVALID: manifest
invalid error - [docs] Working with Docker registries: JFrog Artifactory
[docker registry] Handle non-existent repository for harbor implementation
[ls tree] Fix submodule processing fails when its name and path are not equal
[cleanup] Fix the certain kube context is not supported
[dockerfile] Fix handling specific status of a local uncommitted file
Fix cleanup panic “image … metadata was not found”
Documentation
- [docs] Update werf and multiwerf versions
- [docs] Hide absolute keyring path in CLI reference
Installation
Docs fixes
[readme] Hide code coverage badge
[docs] Configuration/Cleanup: add directives overview
[docs] Configuration/Introduction: fix adding file with shell example
Installation
Fix managed images records with capital letters
Ignore image names with capital letters for managed-images records.
Installation
Fix helm release name not slugged by default anymore
Fix deploy meta config templates defaults.
Installation
Fix ‘patch does not apply’ error in github actions
"virtual-merge-commit": "true" label in the docker image of the stage was not reset to false when building a new image for regular commit (not virtual merge commit).
Installation
Fix werf cleanup slow deletion of images
[cleanup] Gitlab registry implementation: use deleting a tag method instead of an image
More details in gitlab-org/container-registry documentation:
- https://gitlab.com/gitlab-org/container-registry/-/blob/release/2.9-gitlab/docs/spec/api.md#deleting-a-tag
- https://gitlab.com/gitlab-org/container-registry/-/blob/release/2.9-gitlab/docs/spec/api.md#deleting-an-image
Installation
Synchronization fixes
- Http server main page landing with some info.
- Added http synchronization server docs.
Installation
Public http synchronization server by default in distributed mode
Use https://synchronization.werf.io by default when stages-storage not :local.
Fixes
Fix manifest-cache not filled by manifests of newly built stages. Store manifest of newly built stage into manifest stage immediately after building, before storing this stage into stages storage.
Installation
Synchronization fixes
- Http server main page landing with some info.
- Added http synchronization server docs.
Installation
Synchronization server basic health check procedure
- GET /health gives
{"err":"","echo":"","status":"OK"}. - POST /heath + body
{"echo": "HEY!"}gives:{"err":"","echo":"HEY!","status":"OK"}.
Installation
Doc CI/CD integration fixes
- Updating supported CI systems + a few corrections.
- Fixed intro for unsupported CI systems.
- Renaming generic CI/CD integration doc.
Installation
New implementation of distributed locking http server: use new version of lockgate
https://github.com/werf/lockgate/pull/26
Installation
Shallow clone related fixes
Do origin fetch only when git shallow clone is detected. The complete commit history is enough for all commands except images cleanup.
Installation
Optimistic git shallow clone handling
- Ignore git shallow clone when it is possible
- Force using git shallow clone despite restrictions with option –allow-git-shallow-clone
The new cleanup algorithm: config syntax updates and fixes
[cleanup] Change configuration syntax: replace createdIn/publishedIn with in
[docs] Configuration/Cleanup: replace createdIn/publishedIn with in
[tests] Cleanup Suite: update syntax
[tests] Cleanup Suite: fix checks are not compatible with monorepo implementations
[cleanup] UNAUTHORIZED error should not be ignored
[cleanup] Fail purge commands if deletion failed due to using by containers without the force option
Installation
Docs and small fixes
[docs] Reference/Cleanup: more details for git history based cleanup
[managed images] Ignore artifact names
[docs] Adding href rels to topnav.html
[docs] Adding rels to slack_popup.html links for security
Installation
Fix dockerfile ONBUILD
[dockerfile] Take into account ONBUILD instructions from base image
The new cleanup algorithm updates
[tests] Cleanup Suite: fix commit commiter when overriding
[cleanup] Improve logging: shortify commit and content signature
[cleanup] Disable meta images fetching by default
Installation
Fix AWS ECR build and publish
Fix AWS ECR build/publish problem ‘Image has to have at least 1 layer’
Create empty zero-bytes layer for manifest-only images: managed images records and image metadata by commit id records.
The new cleanup algorithm backports for v1.1.20
[cleanup] Update default policies
[cleanup] Fix parsing of cleanup policy tag/branch parameter
[cleanup] Use creation date for annotated tags instead of head commit date
[docs] Reference/Git history based cleanup
[cleanup] Git history based cleanup: improve algorithm
- Add opportunity to keep images by publishing period.
- Update default policies
- Update cleanup section configuration:
cleanup:
keepPolicies:
- references:
branch: /.*/
tag: /.*/
limit:
last: 10
createdIn: 168h
operator: And
imagesPerReference:
last: 10
publishedIn: 168h
operator: And
[tests] Cleanup Suite: git history based tests
[logging] Optimize verbose output
Installation
Fix dockerfile ONBUILD
[dockerfile] Take into account ONBUILD instructions from base image
The new cleanup algorithm updates
[tests] Cleanup Suite: fix commit commiter when overriding
[cleanup] Improve logging: shortify commit and content signature
[cleanup] Disable meta images fetching by default
Installation
Fix AWS ECR build and publish
Fix AWS ECR build/publish problem ‘Image has to have at least 1 layer’
Create empty zero-bytes layer for manifest-only images: managed images records and image metadata by commit id records.
The new cleanup algorithm backports for v1.1.20
[cleanup] Update default policies
[cleanup] Fix parsing of cleanup policy tag/branch parameter
[cleanup] Use creation date for annotated tags instead of head commit date
[docs] Reference/Git history based cleanup
[cleanup] Git history based cleanup: improve algorithm
- Add opportunity to keep images by publishing period.
- Update default policies
- Update cleanup section configuration:
cleanup:
keepPolicies:
- references:
branch: /.*/
tag: /.*/
limit:
last: 10
createdIn: 168h
operator: And
imagesPerReference:
last: 10
publishedIn: 168h
operator: And
[tests] Cleanup Suite: git history based tests
[logging] Optimize verbose output
Installation
Cleanup fixes
[cleanup] Fix UNAUTHORIZED: authentication required error with gitlab.com
[docker_registry] Autodetect gitlab.com implementation
Installation
Cleanup fixes
[cleanup] Fix UNAUTHORIZED: authentication required error with gitlab.com
[docker_registry] Autodetect gitlab.com implementation
Installation
Cleanup fixes
[cleanup] Fix UNAUTHORIZED: authentication required error with gitlab.com
[docker_registry] Autodetect gitlab.com implementation
Installation
Fix distributed werf hangs sometimes
The hanging case was in the lockgate library, described in the https://github.com/werf/lockgate/pull/24.
Installation
[git] Do not use ‘git worktree add –no-checkout’ option
This option is not avaiable in the v2.17.1 version of git, which is the latest avaiable version for 18.04 lts ubuntu.
Specify target commit instead (which is by the way a faster and more precise approach).
Installation
Enable ‘Waiting for locked …’ log messages for kubernetes locker in distributed werf mode
Installation
Docs
[docs] Fixed github stats api usage, requests will be sent only once an hour
Installation
Distributed mode fix
- Use locker-with-retry for all kubernetes locks in werf (10 attempts for acquire/release operations are configured).
- Log all failed attempts to the stderr.
More fixes for werf.io landing
- Fix grammar and rephrase sentences.
- Added extra step to introduction presentation.
Installation
Changed https://werf.io and https://ru.werf.io landing pages!
Backport change for v1.1.16.
- [doc] Added animated presentation, introduction to werf became more comprehensible
- [doc] Added three new publications
Installation
[config] Add .Files.Glob Go template function
{{ range $path, $content := .Files.Glob ".werf/files/**" }}
{{ $path }}: |
{{ $content | indent 2 }}
{{ end }}
-
Process symlinks with .Files.Glob
.Files.Glob returns the hash of regular files and their contents for the paths that are matched pattern. This function follows only symlinks pointed to a regular file (not to a directory). -
.Files.Glob: has an ability to merge results
The function supports shell pattern matching +**. Results can be merged withmergesprig function (e.g{{ $filesDict := merge (.Files.Glob "*/*.txt") (.Files.Glob "app/ **/*.txt") }})
Installation
Fixes
[ci_env] GitHub: fix typo os.Getenv(“IMAGES REPO”)
[cli] Fix –release and –namespace params: add WERF_RELEASE and WERF_NAMESPACE options
Installation
Fix hard-coded images repo/stages storage implementation for ci-env
Automatically set gitlab docker registry implementation for stages storage and images repo.
Installation
Added ability to use slash “/” and plus “+” signs in werf.yaml image names
Installation
Docs
- Guides/GitHub CI/CD integration.
- Add distributed-werf reference info and switch to distributed guide.
Installation
Fixes
[dockerfile] Fix COPY –from arbitrary image (not only dockerfile stage)
Docs
Guides/GitLab CI/CD integration: added overview
Installation
Fixes
- Fix kubernetes namespace slug: should not allow capital letters.
- Fix git-related stages selection when multiple git-mappings by the same url used.
https://github.com/flant/werf/issues/2407
Installation
Fix v1.1 stage selection algorithm
Fix for a case when two images of the same stage for two consecutive commits are building at the same time. This can lead to the bug, because werf uses oldest stage by creation timestamp and not oldest by save-into-stages-storage timestamp.
Use selection based on saving-into-stages-storage-timestamp instead of docker-image creation timestamp.
After this change stages-storage-cache will be invalidated automatically.
Installation
Fixes
- Fix kubernetes namespace slug: should not allow capital letters.
- Fix git-related stages selection when multiple git-mappings by the same url used.
https://github.com/flant/werf/issues/2407
Installation
Fix v1.1 stage selection algorithm
Fix for a case when two images of the same stage for two consecutive commits are building at the same time. This can lead to the bug, because werf uses oldest stage by creation timestamp and not oldest by save-into-stages-storage timestamp.
Use selection based on saving-into-stages-storage-timestamp instead of docker-image creation timestamp.
After this change stages-storage-cache will be invalidated automatically.
Installation
Fix “@ERROR: invalid gid nogroup” on imports stage
The error occurs when artifact image does not contain “nogroup” group, which was used by default when running rsync server. For example when using “maven:3.6.3-jdk-13” as artifact base image.
Fixed by running rsync server using root group.
Retry docker pull/push on internal server error
Werf automatically retries pull/push operations on internal server errors.
Installation
Custom helm chart directory (.helm) and werf configuration file (werf.yaml) options
- Add –helm-chart-dir option.
- Add –config / –config-templates-dir options to override default paths.
Installation
Docs
[docs] Guides/GitLab CI/CD integration
[docs] Support hiding headers in TOC
Installation
Fix dockerInstructions stage not isolated when used after gitCache
After this release dockerInstructions signature will change if used after gitCache. Stage-not-exists in stages-storage errors may occur in this case.
Other
Detailed error messages for git submodules when submodule exists in the repo in the incorrect state.
[test] Test “Check broken links” hase been updated.
Installation
Fix cleanup deletes stages that are related to deployed images
Installation
Introducing distributed builds
Werf now can store stages in the docker registry by specifying --stages-storage REPO instead of --stages-storage :local.
There is no more restriction that a single host should be used to run all werf commands for a project:
- All werf commands that need an access to the stages should specify the same stages storage.
- It is irrelevant on which host werf command is running as long as the same stages storage is used for the commands like: build, publish, cleanup and deploy.
- Also all hosts which run werf should have a connection to the Kubernetes cluster, because werf uses Kubernetes to synchronize multiple build/publish/deploy processes running from different machines.
“werf ci-env gitlab” default mode now is to store stages by path CI_REGISTRY_IMAGE/stages. All existing werf installations will continue to use :local stages storage because --stages-storage :local param has explicitly been specified in all CI-configs. So to use distributed builds user needs to remove this param, see guided steps below about migrating to the distributed stages storage.
Synchronization, caches and cleanup
Introduced a new parameter named --synchronization, which controls how multiple werf processes are synchronized between each other. There is a stages-storage cache and locks components that werf needs to run. Stages storage cache is an internal component used to boost werf performance for idle builds when calculating stages. Locks are needed to organize correct publishing of new stages into stages-storage, publishing images into images-repo and for deploy process also.
When --synchronization :local used (which is factually by default in the previous versions < v1.1.10), then stages-storage cache reside in the local host file system (~/.werf/shared_context/storage/stages_storage_cache/1/PROJECT) and local file locks are used.
When --synchronization kubernetes://NAMESPACE is used, then werf creates a ConfigMap for project in this namespace, stores stages storage cache in this ConfigMap and uses distributed locking over this ConfigMap to synchronize multiple werf processes running from multiple hosts.
By default, when non local stages-storage is used werf automatically use kubernetes://werf-synchronization namespace and create a ConfigMap named werf-PROJECT in this namespace. User can specify arbitrary non-standard namespace on own needs.
What about local docker images cache on hosts where werf is running? — There is such a cache, which is cleared either by the werf itself or can freely be removed by other tools (docker rmi
.
New commands to sync and switch stages
werf stages sync --from=:local|REPO --to=:local|REPO.- Command will copy stages from one stages-storage to another.
- Command will copy only difference of stages from one stages-storage to another.
- Furthermore command will copy multiple stages in parallel.
- Command run result is idempotent: sync can be called multiple times, interrupted, then called again — the result will be the sam
. - There are delete options:
--remove-sourceand--cleanup-local-cache, which controls whether werf will delete synced stages from source stages-storage and wheter werf will cleanup localhost from temporary docker images created during sync process. - This command can be used to download project stages-storage to the localhost for development purpose as well as backuping and migrating purposes.
werf stages switch-from-local --to=REPO- Command will automatically sync existing stages from :local stages storage to the specified REPO.
- Command will block project from being used with :local stages-storage.
- This means after werf stages switch-from-local is done, any werf command that specifies :local stages-storage for the project will fail preventing storing and using build results from different stages-storages.
- Note that project is blocked after all existing stages has been synced.
How to migrate to distributed mode
Let’s say you have some project in Gitlab CI, that uses werf. Steps:
- Go to the directory where your project reside on the runner host.
- Create an MR in the project, that removes
--stages-storage :localparam from all werf invocations. There is no need to set this param explicitly anymore, becausewerf ci-envwill automatically set it toCI_REGISTRY_IMAGE/stages. - Run
werf stages switch-from-local --to=CI_REGISTRY_IMAGE/stageswhere CI_REGISTRY_IMAGE is the address of container registry for your gitlab project. After this step all werf invocations from different git branches of the project that uses--stages-storage :localwill refuse to work, because switch-from-local command sets a block to use :local. - Merge your MR with correct stages-storage param.
- All git branches that need to run werf should rebase then onto the new changes to use new stages-storage, otherwise werf will refuse to use
:localstages storage.
Other changes since last release v1.1.10-alpha.7
[docs] Remove –stages-storage :local from Gitlab CI docs
[ci] Cache GO modules
[tests] New tests
[tests] K8S/Cleanup Suite: test cleanup with deployed image
[tests] K8S/Guides Suite: actualize after each
Error: –stages-storage=ADDRESS param required
[tests] Helm/Render Suite: add extra annotations and labels check
[tests] Cleanup Suite: test stages storage repo
Installation
Remote storages support
Auto reset stages-storage-cache and retry action when docker-image has been deleted from stages-storage, but exists in the stages-storage-cache.
Fixes
Fix gitLatestPatch signature not stable to index line in patches.
Installation
Fix build panics and errors when newly built image has been discarded
Example of error:
delete stage image fdf27403-2c05-4966-850f-dbb0c79ff577 manually and retry the build
Installation
Fix dockerfile builder
[dockerfile] Fix git status checksum algorithm does not count symlinks properly
[dockerfile] Skip outside the build context wildcard till fail on docker build
[dockerfile] Take into account ignored files by .gitignore files
[dockerfile] Change checksum calculation without git logic
- Add relative file path and mode to checksum
- Add symlink link to checksum instead of linked data
[dockerfile] Fix git status with not initialized submodules
Installation
Fix release
Fix problems related to changes introduced in v1.1.8+fix13: signatures has been accidentally changed in v1.1.8+fix13.
Fixes that were made in v1.1.8+fix13 are also available in this v1.1.8+fix14 release:
[build][git] Handle “commits not present” line in git diffs
[build][git] Fix gitLatestPatch not built when previous stage has been discarded during build
- remove git-building-stage map from Conveyor;
- remove AfterSignatureCalculated Stage hook.
[dockerfile] Fix ONBUILD instruction processing
- dockerfile stage signature should take into account ONBUILD value;
- dockerfile stage that uses another one with ONBUILD instructions should
calculate files checksum for COPY/ADD trigger instruction in addition to ONBUILD value.
Installation
Fix dockerfile builder
[dockerfile] Fix git status checksum algorithm does not count symlinks properly
[dockerfile] Skip outside the build context wildcard till fail on docker build
[dockerfile] Take into account ignored files by .gitignore files
[dockerfile] Change checksum calculation without git logic
- Add relative file path and mode to checksum
- Add symlink link to checksum instead of linked data
[dockerfile] Fix git status with not initialized submodules
Installation
Fix dockerfile builder
[dockerfile] Fix git status checksum algorithm does not count symlinks properly
[dockerfile] Skip outside the build context wildcard till fail on docker build
[dockerfile] Take into account ignored files by .gitignore files
[dockerfile] Change checksum calculation without git logic
- Add relative file path and mode to checksum
- Add symlink link to checksum instead of linked data
[dockerfile] Fix git status with not initialized submodules
Installation
Fixes
[cleanup] Skip images with MANIFEST_UNKNOWN error
[build][git] Handle “commits not present” line in git diffs
[build][git] Fix gitLatestPatch not built when previous stage has been discarded during build
- remove git-building-stage map from Conveyor;
- remove AfterSignatureCalculated Stage hook.
[dockerfile] Fix ONBUILD instruction processing
- dockerfile stage signature should take into account ONBUILD value;
- dockerfile stage that uses another one with ONBUILD instructions should calculate files checksum for COPY/ADD trigger instruction in addition to ONBUILD value.
Installation
Fix dockerfile builder
[dockerfile] Fix git status checksum algorithm does not count symlinks properly
[dockerfile] Skip outside the build context wildcard till fail on docker build
[dockerfile] Take into account ignored files by .gitignore files
[dockerfile] Change checksum calculation without git logic
- Add relative file path and mode to checksum
- Add symlink link to checksum instead of linked data
[dockerfile] Fix git status with not initialized submodules
Installation
Remote storages support: safe switch between alpha and beta werf versions
Invalidate stages-storage-cache directory when old cache directory exists. Remove old and current stages-storage-cache directories in this case.
Installation
Fix release
- Use 1.1 stable in README installation and docs.
- Change v1.1.8 stages-storage-cache dir to prevent bugs related to update to new v1.1.9.
Installation
Fix werf dockerfile image not rebuilds when COPY/ADD source path files changed
Fix COPY/ADD source path with trailing backslash is not counted in the signature.
Installation
Support non-default docker registries implementations and dockerhub
- AWS ECR, Azure CR, GCR, GitHub Packages, GitLab Registry, Harbor, Quay.
- Documentation: https://werf.io/v1.1-beta/documentation/reference/working_with_docker_registries.html
Support stages-storage in docker registry
For now only when building from single host. Building from multiple hosts will be available soon in v1.1.
Installation
Fix werf dockerfile image not rebuilds when COPY/ADD source path files changed
Fix COPY/ADD source path with trailing backslash is not counted in the signature.
Installation
Fix build panic when running ‘werf build ARTIFACT’ and artifact depends on another artifact
Installation
Support import from certain artifact/image stage
Added ability to specify stage directive in the import directive to perform import from certain artifact or image stage:
import:
- add: /file
to: /file
artifact: myartifact
before: setup
stage: install
Note that image which imports some files from another image stage will be rebuild only if this specified stage has been rebuilt.
This feature allows to avoid unnecessary imports of unchanged files.
Implementation details
- image stages-signature has been renamed to image content-signature;
- added content-signature to each stage, which differs from regular signature and represents content of the stage.
Installation
Support building only artifacts specified for ‘werf build’ command
werf build -s :local ARTIFACT_NAME1 ARTIFACT_NAME2 ...
Installation
INCOMPATIBLE CHANGE Fix stages and images isolation problem for git-latest-patch and stages-signature
-
gitLatestPatch stage was not isolated between different branches when the same changes
has been made in two different commits: added commits ancestry check when selecting
suitable stage from cache for gitLatestPatch. The signature of gitLatestPatch stage
has not been changed. -
Stages-signature was not isolated between different branches when the same
changes has been maed in two different commits. IMPORTANT stages-signature
calculation algorithm has been changed, stages-signature will change for all projects
that use stages-signature tagging strategy (content-based).
Installation
Werf kubernetes manifests yaml generation fixes
-
Fix werf extra labels and annotations not added to deployed resources.
-
Unfortunately revert fix to the problem with ‘|’ rendered as ‘|-‘ in kubernetes templates,
which was in the https://github.com/flant/werf/releases/tag/v1.1.7 release.The fix for this problem is being investigated and will be released soon.
Installation
Fixes
Fix “UNSUPPORTED: Invalid parameter at ‘maxResults’” when using ECR. Fixed by updating github.com/google/go-containerregistry, the issue: https://github.com/google/go-containerregistry/issues/681.
Docs
Added publications about werf 1.1.
Installation
Fix failed to prepare update: unable to decode “”: Object ‘Kind’ is missing
Installation
Fixes
Fix “UNSUPPORTED: Invalid parameter at ‘maxResults’” when using ECR. Fixed by updating github.com/google/go-containerregistry, the issue: https://github.com/google/go-containerregistry/issues/681.
Docs
Added publications about werf 1.1.
Installation
Fix extra annotations/labels are not added to k8s resources
Installation
Fixes
Fix “UNSUPPORTED: Invalid parameter at ‘maxResults’” when using ECR. Fixed by updating github.com/google/go-containerregistry, the issue: https://github.com/google/go-containerregistry/issues/681.
Docs
Added publications about werf 1.1.
Installation
An option to enable cleanup for stages-signature tagging strategy
When running werf in ci with werf ci-env command, then either:
-
create cleanup config for the host:
mkdir ~/.werf/config echo "stagesSignatureStrategyExpiryDays: 30" >> ~/.werf/config/cleanup.yaml echo "stagesSignatureStrategyLimit: 50" >> ~/.werf/config/cleanup.yaml -
export environment variables in your CI/CD system:
WERF_STAGES_SIGNATURE_STRATEGY_LIMIT=50andWERF_STAGES_SIGNATURE_STRATEGY_EXPIRY_DAYS=30.
By default werf will not cleanup images published with stages-signature tagging strategy and related stages.
Installation
Support non-default docker registries implementations and dockerhub
- AWS ECR, Azure CR, GCR, GitHub Packages, GitLab Registry, Harbor, Quay.
- Documentation: https://werf.io/v1.1-beta/documentation/reference/working_with_docker_registries.html
Support stages-storage in docker registry
For now only when building from single host. Building from multiple hosts will be available soon in v1.1.
Installation
An option to enable cleanup for stages-signature tagging strategy
When running werf in ci with werf ci-env command, then either:
-
create cleanup config for the host:
mkdir ~/.werf/config echo "stagesSignatureStrategyExpiryDays: 30" >> ~/.werf/config/cleanup.yaml echo "stagesSignatureStrategyLimit: 50" >> ~/.werf/config/cleanup.yaml -
export environment variables in your CI/CD system:
WERF_STAGES_SIGNATURE_STRATEGY_LIMIT=50andWERF_STAGES_SIGNATURE_STRATEGY_EXPIRY_DAYS=30.
By default werf will not cleanup images published with stages-signature tagging strategy and related stages.
Installation
[deploy] Hotfix werf helm render: fix problem with ‘|’ rendered as ‘|-‘
Change gopkg.in/yaml.v2 to github.com/ghodss/yaml in custom werf templates engine,
as github.com/ghodss/yaml is used everywhere in the helm 2.
In the helm 3 sigs.k8s.io/yaml is used, but it is not possible for now to switch to sigs.k8s.io/yaml before switching to helm 3 codebase.
https://github.com/helm/helm/blob/master/pkg/engine/funcs.go#L27
Installation
Docs
Added rss feeds for werf releases.
Fixes
[dockerfile] Fail build if dockerfile COPY instruction refers to nonexistent stage.
Installation
[dockerfile] Fix dockerfile stage name from “COPY –from” is not detected
Docker parser coverts dockerfile stage name (AS) to lowercase but in places of the name usage (COPY –from) does not.
This affected dockerfile stage signature calculation and occurred false warning messages.
Installation
Fix release
[dockerfile] Fix calculating context files checksum when submodule is used
Installation
Fix git problems release
Fix creating an archive fails when nested submodules is used
Installation
Fix git problem release
Fix creating an archive fails when submodule dir/file is specified in git.add
Installation
Official v1.1 beta
Implemented content based tagging and new stages storage architecture as a big step toward distributed and concurrent builds.
New stages format and cache selection algorithm
New stage name generation rule. Every build of the stage generates uniq stage-cache image name, which consists of 2 parts: signature (as in v1.0) plus unique identifier.
For example, full stage image name will look like:
werf-stages-storage/myproject:d2c5ad3d2c9fcd9e57b50edd9cb26c32d156165eb355318cebc3412b-1582656767835
werf-stages-storage/PROJECT:SIGNATURE-TIMESTAMP_MILLISEC
Signature identifier of the stage represents content of the stage and depends on git history which lead to this content.
Werf stage selection algorithm is based on the git commits ancestry detection:
-
Werf calculates a stage signature for some stage.
-
There may be multiple stages in the stages storage by this signature, werf selects all suitable stages by the signature.
-
If current stage is related to git (git-archive, user stage with git patch or git latest patch), then werf selects only
those stages which are relaed to the commit that is ancestor of current git commit.
-
Select from the remaining stages the oldest by the creation timestamp.
There may be multiple built images for a single signature.
Stage for different git branches can have the same signature, but werf will prevent cache of different git branches from
being reused for totally different branch.
For more info see documentation: https://werf.io/v1.1-beta/documentation/reference/stages_and_images.html#stage-naming.
New stage building and saving algorithm
If suitable stage has not been found during stage selection, werf starts building a new image for stage.
Note that multiple processes (on a single or multiple hosts) may start building the same stage at the same time. Werf uses optimistic locking when saving newly built image into the stages storage: when a new stage has been built werf locks stages storage and saves newly built stage image into storage stages cache only if there are no suitable already existing stages exists. Newly saved image will have a guaranteed unique identifier
by TIMESTAMP_MILLISEC. In the case when already existing stage has been found in the stages storage werf will discard newly built image and
use already existing one as a cache.
In other words: the first process which finishes the build (the fastest one) will have a chance to save newly built stage into the stages storage. The slow build process will not block faster processes from saving build results and building next stages.
For more info see documentation: https://werf.io/v1.1-beta/documentation/reference/stages_and_images.html#stage-building-and-saving.
Benefits of new stages storage architecture
This new stages storage architecture opens up an opportunities to build stages concurrently and even distributely
from multiple build machines. For now there are no theoretical limitations of current werf architecture to implement
distributed or concurrent builds. So we consider this release as a big step towards distributed and concurrent builds.
Werf have an --stages-storage and --synchronization parameters which specify addresses of the stages storage and
synchronization lock manager. For now there is only :local implementations of both primitives. By changing these implementations to docker-registry for the stages storage and Redis or Kubernetes server based synchronization lock manager, werf will implement distributed builds out of the box.
Content based tagging
Werf v1.1 supports so called content based tagging. Tags of resulting docker images depend on the content of these images.
When using werf publish --tags-by-stages-signature or werf ci-env --tagging-strategy=stages-signature werf will tag result images by so called image stages signature. Each image tagged by own stages signature which calculated by the same rules as regular signature of image stage.
Image stages signature depends on content of the image and depends on the git history which lead to this content.
There may be dummy commits into the git repo that do not change resulting images. For example empty commits, merge commits or commits which change files that are not imported into the resulting image.
When using tagging by git-commits these dummy commits will cause werf to create new images names even if content of these images is the same. New images names in turn will cause restarts of application Pods in Kubernetes which is totally not a desired behaviour. All in all this is the reason preventing storing multiple application services in the single git repo.
Stages signature on the countrary will not change on dummy commits, so these commits will not cause restarts of application Pods in kubernetes, yet it similarly to commit-id relates to the git history of edits and depends on the content of the files.
Also tagging by stages signatures is more realiable tagging method than tagging by a git-branch for example, because resulting images content does not depend on order of pipelines execution. Stages signature leads to stable immutable images names which represent the address of the certain image content.
NOTE From now and on stages-signature is the default tagging strategy and the only recommended one for usage.
For more info see documentation: https://werf.io/v1.1-beta/documentation/reference/publish_process.html#content-based-tagging.
Improved performance of stages selection and verification
Idle builds when all stages are exist in the stages storage are really fast now (in the most cases build retry will run under 1sec).
Stages verification performance during werf deploy and werf run commands also improved a lot.
Improved performance of Dockerfile builder
Werf Dockerfile builder signature calculation performance is improved due to using git ls-tree checksums
for docker context files. Signature calculation does not depend on the docker context size.
Improved performance of artifact/image imports
Werf uses rsync server to import files from artifacts and images.
MacOS imports performance does not depend on docker implementation of volumes now. MacOS imports performance is the same as in Linux and Windows.
Other changes
-
Added
--log-debug,--log-verboseand--log-quietmodes. -
Removed legacy params and code (dapp secret key legacy support removed;
--insecure-repolegacy option removed, only –skip-tls-verify-registry is available). -
Eliminated
werf.io/recreateannotation, werf useshelm.sh/hook-delete-policy=before-hook-creationmode by default (as in helm 3). -
Improved werf cleanup procedure not to clean images from werf.yaml defined in the different git branches,
those are not exist in the werf.yaml config of the master branch. Added new commands to control managed images:
werf managed-images ls|add|rm.
Installation
Docs
-
New stages storage architecture documented.
-
Content based tagging documented.
Other fixes
-
fromLatest/git.Branch params could be used only with
herebyIAdmitThatBranchMightBreakReproducibilityandherebyIAdmitThatFromLatestMightBreakReproducibilityparams; -
werf images managedsubcommand renamed towerf managed-images; -
tiny logging improvements:
-
hide docker container output in silent mode;
-
initialize stapel container before preparing of stage run args.
-
Installation
Fix logs in default mode
Remove ‘Switch work tree to commit …’ messages in default mode (will be printed in –log-verbose or –log-debug).
Installation
WARNING: incompatible release
Stages signatures was changed, pipelines that were already build will use new signatures on retry, so rebuild needed.
Dockerfile builder support for v1.1!
Calculate docker build context files checksum with git ls-tree and status.
Fixed imports: allow merging of directories
import:
- artifact: myartifact
add: /mydir
to: /my/existing/dir
after: install
Werf will automatically merge content of /mydir into /my/existing/dir.
This case was broken in v1.1.0-beta.4 due to imports optimization rework: the specified config will create my/existing/dir/mydir instead of merging directories.
v1.1.0-beta.3 was not affected by this bug.
Fixed logging performance
Fixed logboek library logging performance when processing large output, e.g. from make build command. Logging performance affects build time.
Stapel image updated to version 0.6.1.
Logging improvement
-
Disabled debug logging by default.
-
Default logging includes minimum viable info for commands.
-
--log-debugand--log-verboseoptions are avaiable. -
More info in
--log-debugmode.
Store managed config images in stages storage
Images that were built by werf will be remained in stages storage. For each such image managed image record created as werf-managed-images/PROJECT:IMAGE_NAME.
During cleanup procedure werf will preserve stages cache of images that are defined in the current werf.yaml config and images from managed images list. This prevents images defined only in custom git branches from being cleaned up, when running werf cleanup procedure main (master) branch.
Werf automatically creates records for managed images during build procedure.
Managed images list could be manually viewed and changed using commands:
-
werf images managed -s :local ls -
werf images managed -s :local add IMAGE_NAME -
werf images managed -s :local rm [IMAGE_NAME, ...]
Small fixes and changes
-
Fix publishing of only specified in cli images by names.
-
Renamed
--stages-storage-lockparam to--synchronization: address of synchronizer for multiple werf processes to work with a single stages storage (:local by default). -
Update kubedog: load more kubernetes client-go auth plugins: azure, exec, oidc, openstack.
Installation
Fixes release (INCOMPATIBLE — more info belo
)
Small fixes and additions
- Add –log-verbose, –log-quiet options, use –log-debug=true by default for werf v1.1 for now.
- Fixed and enabled cleanup procedure.
- Do not publish artifacts fix.
- Fix: cleanup dockerfile builder tmp img on conveyor termination.
- Fix helm release name slugify: should be a valid DNS-1123 subdomain and be maximum 53 chars.
Fix import stage logic and stages cleanup
The import stage image has the specific label value that is used during stages cleanup procedure. In this commit, related image/artifact signature which stored in the label is replaced on image ID.
The signature does not identify the stage since we started to use not only signature in an image tag. From v1.1, werf is building stages consistently so related image/artifact image ID is available when werf is preparing import stage.
INCOMPATIBLE change, because signature of import stage has been changed. Already built images should be rebuilt with this version of werf.
Also prevented possible bug, which may occur when artifacts are used and there are empty stages.
Installation
Fixes related to the “Resource already exists” deploy error
- fix “resource already exists” error on redeploy when new resource with an error has been added to the chart,
refs https://github.com/flant/helm/pull/40 - fix “resource already exists” error message when creating a helm.sh/hook that already exists,
refs https://github.com/flant/helm/pull/41 - do not allow resource adoption during initial release installation, fix error messages,
refs https://github.com/flant/helm/pull/42
Installation
Fixes release (INCOMPATIBLE — more info belo
)
Small fixes and additions
- Add –log-verbose, –log-quiet options, use –log-debug=true by default for werf v1.1 for now.
- Fixed and enabled cleanup procedure.
- Do not publish artifacts fix.
- Fix: cleanup dockerfile builder tmp img on conveyor termination.
- Fix helm release name slugify: should be a valid DNS-1123 subdomain and be maximum 53 chars.
Fix import stage logic and stages cleanup
The import stage image has the specific label value that is used during stages cleanup procedure. In this commit, related image/artifact signature which stored in the label is replaced on image ID.
The signature does not identify the stage since we started to use not only signature in an image tag. From v1.1, werf is building stages consistently so related image/artifact image ID is available when werf is preparing import stage.
INCOMPATIBLE change, because signature of import stage has been changed. Already built images should be rebuilt with this version of werf.
Also prevented possible bug, which may occur when artifacts are used and there are empty stages.
Installation
Fixes related to the “Resource already exists” deploy error
- fix “resource already exists” error on redeploy when new resource with an error has been added to the chart,
refs https://github.com/flant/helm/pull/40 - fix “resource already exists” error message when creating a helm.sh/hook that already exists,
refs https://github.com/flant/helm/pull/41 - do not allow resource adoption during initial release installation, fix error messages,
refs https://github.com/flant/helm/pull/42
Installation
First Werf v1.1 beta release!
Main changes since v1.0:
- Fully reworked build process, new incompatible stages storage format (new build cache).
-
Improved performance of idle builds (from cache) nearly by one order of magnitude when git repo of the project is big.
-
Build cache isolation based on the git history. Build cache of different branches without same base is not overlapped. No more “rebase” related problems.
-
Also improved performance of stage-dependencies calculation (using git ls-tree mechanix) when large folders are specified as stage dependencies.
-
Content based tagging using
werf publish --tag-by-stages-signatureoption orwerf ci-env --tagging-strategy=stages-signaturetagging strategy.The default tagging strategy is
stages-signature, werf ci-env parameter--tagging-strategymay be omitted to use default strategy.
Not supported in this version
IMPORTANT Following feature are not supported in the current beta version and will be repaired in the following releases very soon!
-
Building Dockerfile-based images. This version of werf v1.1 beta should not be used to build dockerfile-based images.
-
werf cleanupcommand will not actually clean stages, only images, due to the bug in the cleaning procedure.werf cleanupcommand can and should be used already now and will be fixed soon by automatic releases updater for 1.1 beta channel.
Fully reworked build process
-
Stage image name format changed: werf-stages-storage/PROJECT_NAME:SIGNATURE-UNIX_TIMESTAMP_MILLISECS.
-
UNIX_TIMESTAMP_MILLISECS is the unique part of stage image name.
-
There may be multiple stages with the same SIGNATURE part. Signature is based on the content of the image, while timestamp is added to keep separately built images separate.
-
IMPORTANT! Multiple processes may build the same stage by the same signature simultaneously at the same time. The first builder that builds the stage will store the stage in the stages storage. Other builders will discard build results when saving newly built image into stages storage if stages storage already contains suitable image built by the other process.
-
There may be multiple stages with the same SIGNATURE for different git branches. Stage selection algorithm will make sure that stage is suitable for your branch.
Compatibility notes
Incompatibilities with v1.0:
-
Werf v1.1 will generate new stages for stages storage.
-
Removed legacy ruby dapp secret-key format support.
-
Removed –insecure-repo option, use –skip-tls-verify-registry option instead.
-
“helm.sh/hook-delete-policy=before-hook-creation” by default for all kinds of hooks (not only Jobs as in v1.0).
-
Strict yaml parsing which does not allow duplicate keys in yaml docs (affects werf.yaml, .helm/templates/**, .helm/values.yaml etc.).
It is OK to use v1.0 and v1.1 simultaneously:
-
v1.1 and v1.0 may exists on the same host system, without any restrictions;
-
v1.1 and v1.0 may be used in the different branches of the same project without any restrictions;
-
cleanup procedure of v1.0 will not affect v1.1 and vice versa;
-
build cache (stages in the stages-storage) of v1.0 and v1.0 is fully separated.
How to enable v1.1
-
Change multiwerf use command:
type multiwerf && source <(multiwerf use 1.1 beta) -
(Optional) To use content based tagging change ci-env command:
type werf && source <(werf ci-env gitlab)
Installation
These releases can bring new features, but are unstable. We do not guarantee backward compatibility between Alpha releases.
Fixes
[logging] Build: add waiting for background tasks log message.
[cleanup] Ignore blob-unknown errors properly.
During cleanup operation do not raise should-reset-stages-storage-cache error when get stage manifest operation have failed with
blob-unknown error.
Installation
Fixes
[v1.1] Skip blob-unknown images in cleanup properly
Installation
Fixes
[v1.1] Skip blob-unknown images in werf-cleanup procedure
Installation
Fixes
Automatically invalidate corrupted manifests cache json records.
Installation
Fixes
[buidler] Fix panic docker_registry.IsNameUnknownError
Installation
[v1.1] Fix saving stages-ids into the stages-storage-cache
When some stage has appeared in the repository tags list, but stage image is invalid (due to MANIFEST_UNKNOWN or BLOB_UNKNOWN container registry errors), werf ignores such stage. But this stage-id of such broken stage has been saved into the stages-storage-cache anyway, which is wrong.
Installation
Fixes
[builder] Proper rejection of blob-unknown stages
Do not reject stage when manifest-unknown or name-unknown errors detected. Only reject images with blob-unknown errors.
Installation
[v1.1] Fix saving stages-ids into the stages-storage-cache
When some stage has appeared in the repository tags list, but stage image is invalid (due to MANIFEST_UNKNOWN or BLOB_UNKNOWN container registry errors), werf ignores such stage. But this stage-id of such broken stage has been saved into the stages-storage-cache anyway, which is wrong.
Installation
[v1.1][stapel] Support for rejected stages in the stages storage
Reject stage when docker-pull have failed with the “blob unknown” error.
Add special record into the stages-storage for such rejected stages.
Other
[v1.1] Add ability to set werf-stapel image name with WERF_STAPEL_IMAGE_NAME environment variable.
Installation
[stapel] Fix: fetch artifact/image stage before running imports rsync server
Fetch operation will check that image exists correctly and will reset stages building conveyor to rebuild this stage if needed.
Installation
Fixes
Fix: print rsync server docker run command on imports server run failure
Installation
Fixes
[v1.1][GC] Fix StagesStorage ShouldFetch method: perform fresh docker-inspect, do not use cached inspection result from the previous inspect.
This fix particularly needed for the v1.2 host cleanup procedure which can remove freshly built and saved into the stages storage images. In such case should-fetch and stages-storage-manager fetch method will lock an image and perform standard fetch procedure.
[v1.1] Added stage image locking into stages-storage-manager to prevent v1.2 host cleanup procedure from deleting stages which are in use.
Installation
Fixes
Completely remove redudant stages_and_images read-only shared lock usage to reduce the number of requests to lock-server.
Installation
[cleanup][v1.1] Enable v1.2 werf-cleanup mode for v1.1 by default
–git-history-based-cleanup-v1.2 option has been enabled by default: use git history based cleanup and delete images tags without related image metadata (default $WERF_GIT_HISTORY_BASED_CLEANUP_v1_2).
Installation
[GC][v1.1] Save last/first werf run timestamps for v1.1
This info needed for v1.2’s host cleanup procedure.
Installation
Debug release
Kubedog update: disable events informer with KUBEDOG_DISABLE_EVENTS=1
Installation
[v1.1] Helm 3 release existence checker
Add support for custom kube-context and kube-config-base64.
Installation
[config] Add fromYaml function
- fromYAML(str string) (map[string]interface{}, error)
- return error if unmarshalling failed
werf.yaml
...
{{- $values := .Files.Get "werf_values.yaml" | fromYaml -}} // or fromYaml (.Files.Get "werf_values.yaml")
from: {{- $values.image.from }}
werf_values.yaml
image:
from: alpine
Installation
Debug release
Disable resources waiter debug env WERF_DISABLE_RESOURCES_WAITER=1
Installation
[1.1] Helm 3 release existence check
werf-converge and werf-deploy commands will refuse to work when helm 3 release with the same name as target helm 2 release exists in the current namespace.
This check protects existing helm 3 release when your project has already been migrated from helm 2 to helm 3, but for some reason werf v1.1 tries to redeploy this release.
Installation
[config] Add fromYaml function
- fromYAML(str string) (map[string]interface{}, error)
- return error if unmarshalling failed
werf.yaml
...
{{- $values := .Files.Get "werf_values.yaml" | fromYaml -}} // or fromYaml (.Files.Get "werf_values.yaml")
from: {{- $values.image.from }}
werf_values.yaml
image:
from: alpine
Installation
[werf.yaml] Mix Sprig V2 and V3 functions for werf.yaml templating
Introduce the following list of sprig v3 functions which was not previously avaialable in the sprig v2:
add1f
addf
all
any
bcrypt
chunk
concat
decryptAES
deepCopy
deepEqual
dig
divf
duration
durationRound
encryptAES
fromJson
genCAWithKey
genSelfSignedCertWithKey
genSignedCertWithKey
get
getHostByName
htpasswd
maxf
minf
mulf
mustAppend
mustChunk
mustCompact
mustDateModify
mustDeepCopy
mustFirst
mustFromJson
mustHas
mustInitial
mustLast
mustMerge
mustMergeOverwrite
mustPrepend
mustPush
mustRegexFind
mustRegexFindAll
mustRegexMatch
mustRegexReplaceAll
mustRegexReplaceAllLiteral
mustRegexSplit
mustRest
mustReverse
mustSlice
mustToDate
mustToJson
mustToPrettyJson
mustToRawJson
mustUniq
mustWithout
must_date_modify
osBase
osClean
osDir
osExt
osIsAbs
randBytes
randInt
regexQuoteMeta
seq
subf
toDecimal
toRawJson
urlJoin
urlParse
https://github.com/werf/werf/pull/3220
Installation
Fixes
Do not print warnings for MANIFEST_UNKNOWN and BLOB_UNKNOWN registry errors until werf-cleanup does not delete these images automatically.
Installation
Fixes
[BLOB_UNKNOWN] Completely ignore images with BLOB_UNKNOWN errors from the standard docker registry
- Consider image is not exists in that case.
- Log warnings on BLOB_UNKNOWN and MANIFEST_UNKNOWN for default docker registry impl.
- Removed experimental WERF_EXPERIMENTAL_IGNORE_IMAGE_METADATA_BY_COMMIT_BLOB_UNKNOWN env variable.
Installation
Experimental debug
Ignore image-metadata-by-commit BLOB_UNKNOWN errors when WERF_EXPERIMENTAL_IGNORE_IMAGE_METADATA_BY_COMMIT_BLOB_UNKNOWN=1 is set
Installation
Fixes
Do not parse values with strict yaml parser to allow usage of yaml-anchors
Installation
DEPRECATION WARNING
Add deprecation warning for the fromImageArtifact directive.
Fixes
Fix WERF_SET_* arguments are not active when –set has been specified issue. Add ability to specify WERF_SET_* order of processing by sorting environment variables alphanumerically by the name.
https://github.com/werf/werf/issues/3128
https://github.com/werf/werf/issues/3137
Complete list of refactored list arguments:
- –set + WERF_SET_*;
- –set-string + WERF_SET_STRING_*;
- –set-file + WERF_SET_FILE_*;
- –add-label + WERF_ADD_LABEL_*;
- –add-annotation + WERF_ADD_ANNOTATION_*;
- –ssh-key + WERF_SSH_KEY_*;
- –secondary-repo + WERF_SECONDARY_REPO_*;
Process params as follows:
- Sort WERF__* environment variables alphanumerically.
- Merge all params passed by environment variables with CLI-params (CLI-params have more priority).
Installation
Add deploy values debug info
Print “Helm values params” block when –debug has been specified.
Installation
Small features
Added .Values.global.werf.version service value with werf cli util version.
Installation
Fixes
Fix redundant stapel container initialization for dockerfile image
Installation
Fixes
Fix “Error: parsing file … failed: yaml: unmarshal errors … cannot unmarshal !!map into string”
Installation
Fixes
Update kubedog: fix Job hanging case (https://github.com/werf/kubedog/pull/184), fix color disabled in CI/CD systems.
Installation
Fix “docker: invalid reference format: repository name must be lowercase”
Fix –docker-options option cannot contain multiple spaces.
Installation
Fix ‘Error: UPGRADE FAILED: another operation (install/upgrade/rollback) is in progress’ error
https://github.com/werf/helm/pull/61
Installation
Fix incompatible change for Dockerfile-builder introduced in v1.1.22+fix34
- [dockerfile, digest] Fix ENV/ARG replacement for dockefile instructions, fix incompatible diges change.
- [dockerfile] Fix error handling typo.
Docs
- Fix cli documentation structure for ru-version.
Installation
Fixes
- Fix release status does not print last RESOURCES status line.
Installation
Fix incompatible change for Dockerfile-builder introduced in v1.1.22+fix34
- [dockerfile, digest] Fix ENV/ARG replacement for dockefile instructions, fix incompatible diges change.
- [dockerfile] Fix error handling typo.
Docs
- Fix cli documentation structure for ru-version.
Installation
Fixes
- [dockerfile] Fix FROM with composite ARG variable value is not resolved.
- [ci-env] Fix extra annotation with workflow run url for GitHub Actions.
Installation
Fixes
- [dockerfile] Fix unable to build from scratch.
- [dockerfile, digest] Fix ENV/ARG replacement for dockefile instructions.
Installation
Fixes
- [dockerfile] Fix FROM with composite ARG variable value is not resolved.
- [ci-env] Fix extra annotation with workflow run url for GitHub Actions.
Installation
Fix publish report related panic ‘fatal error: concurrent map writes’
Installation
Fix artifact import error ‘@ERROR: invalid uid root’
Use 0:0 instead of root:root symbolic names in rsyncd which runs inside artifact.
Issue reproduces when artifact image does not have root line in the /etc/passwd file.
Refs https://github.com/werf/werf/issues/2762
Installation
Fixes
- Fix system ssh agent forwarding on macOS.
- Fix GitLab Container Registry delete method selection.
Installation
Fixes
commit c4d8ccd9dcd8335493dbabd692d8d115841cab5e (HEAD -> master, origin/master, origin/HEAD)
Merge: d74ae250f a72139b32
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Wed Sep 23 14:37:54 2020 +0300
Merge pull request #2698 from werf/dismiss
[helm-v3] Helm-v3 support for werf-dismiss command
commit a72139b3231926b9bf0e21fec15b7b4745ddb533 (origin/dismiss, dismiss)
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Mon Sep 21 20:19:12 2020 +0300
[helm-v3] Helm-v3 support for werf-dismiss command
commit d74ae250f81f9687439c54301206cb9f47ab234c
Merge: 80ba58ce9 c8a62aef9
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:33 2020 +0300
Merge pull request #2697 from werf/cleanup_fix_runtime_error
[cleanup] Fix runtime error when MANIFEST_UNKNOWN occurs
commit 80ba58ce9a0223adec34aaf9bc964fb18a4fa717
Merge: eb77f0bff ec0bde19c
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:00 2020 +0300
Merge pull request #2696 from werf/cleanup_tests_disable_parallel_for_quay
[tests] Cleanup Suite: disable parallel for quay tests
commit eb77f0bffdb982a71ec7358a39ce77141eb95599
Merge: 5615d25b9 75bd2adf5
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:39:34 2020 +0300
Merge pull request #2695 from werf/cleanup_default_parallel_tasks_limit
[cleanup] Increase default parallel tasks limit to 10
commit ec0bde19cafb751b91b5b0bd02e7083c83f844b9 (origin/cleanup_tests_disable_parallel_for_quay)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:22:00 2020 +0100
[tests] Cleanup Suite: disable parallel for quay tests
commit c4d8ccd9dcd8335493dbabd692d8d115841cab5e (HEAD -> master, origin/master, origin/HEAD)
Merge: d74ae250f a72139b32
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Wed Sep 23 14:37:54 2020 +0300
Merge pull request #2698 from werf/dismiss
[helm-v3] Helm-v3 support for werf-dismiss command
commit a72139b3231926b9bf0e21fec15b7b4745ddb533 (origin/dismiss, dismiss)
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Mon Sep 21 20:19:12 2020 +0300
[helm-v3] Helm-v3 support for werf-dismiss command
commit d74ae250f81f9687439c54301206cb9f47ab234c
Merge: 80ba58ce9 c8a62aef9
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:33 2020 +0300
Merge pull request #2697 from werf/cleanup_fix_runtime_error
[cleanup] Fix runtime error when MANIFEST_UNKNOWN occurs
commit 80ba58ce9a0223adec34aaf9bc964fb18a4fa717
Merge: eb77f0bff ec0bde19c
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:40:00 2020 +0300
Merge pull request #2696 from werf/cleanup_tests_disable_parallel_for_quay
[tests] Cleanup Suite: disable parallel for quay tests
commit eb77f0bffdb982a71ec7358a39ce77141eb95599
Merge: 5615d25b9 75bd2adf5
Author: Timofey Kirillov timofey.kirillov@flant.com
Date: Tue Sep 22 19:39:34 2020 +0300
Merge pull request #2695 from werf/cleanup_default_parallel_tasks_limit
[cleanup] Increase default parallel tasks limit to 10
commit ec0bde19cafb751b91b5b0bd02e7083c83f844b9 (origin/cleanup_tests_disable_parallel_for_quay)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:22:00 2020 +0100
[tests] Cleanup Suite: disable parallel for quay tests
commit c8a62aef9e817b1b1904d0d22d917a4dd5c2c1ab (origin/cleanup_fix_runtime_error)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:30:34 2020 +0100
[cleanup] Fix runtime error when MANIFEST_UNKNOWN occurs
commit b4b4d3e2d01fffb0e89f81a2274800e5393d41d4
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:19:20 2020 +0100
[cleanup] Fix typo
commit 75bd2adf588ec972428d747f5a6ea6f395010a7a (origin/cleanup_default_parallel_tasks_limit)
Author: Alexey Igrychev alexey.igrychev@flant.com
Date: Mon Sep 21 11:15:46 2020 +0100
[cleanup] Increase default parallel tasks limit to 10
Installation
More helm-v3
- [helm-v3] Added auto-rollback and timeout params for werf-deploy command, fixed ‘has no deployed releases’ error when initial install has failed during waiting for resources readiness.
- [helm-v3] Added missed
--log-*and--set-filesflags.- Also print all persistent flags in help messages (fixed werf help template).
Other
- [logging] Print main time prefix with parallel task time prefix
Installation
Parallelize operations with docker registry
- parallel read/delete stages from stages-storage operations;
- parallel read/delete images info from images-repo operations;
- image metadata cache;
- add –parallel and –parallel-tasks-limit options for cleanup (by default, parallelization is enabled and tasks limit is 5).
Other fixes
- Do not lock cleanup process by images-repo.
- Push image with retries when specific errors occur.
- http2: server sent GOAWAY and closed the connection
- http2: Transport received Server’s graceful shutdown GOAWAY
- REDACTED: UNKNOWN
Helm-v3 support
Add support for werf helpers templates (werf_container_image, werf_container_env) and service values for werf-deploy command when WERF_DEPLOY_HELM_V3=1 is set.
Installation
Fixes and docs
- [dockerfile] Fix “panic: interface conversion” while discarding newly built image.
- [docs] a hyphen for low-level.
Git-history based cleanup
- [cleanup] Git history-based algorithm: fix keep policies with many references might work too slow.
- [logging] Stages cleanup: print skipped stages by ignore period policy.
- [cleanup] Git history-based algorithm: skip content signatures without related images.
Helm-v3 based deploy
- Use helm-v3 in
werf deploycommand whenWERF_DEPLOY_HELM_V3=1is set.- Supported features:
- extra labels/annotations;
- distributed release locking using kubernetes to prevent simultaneous deploy of the same release;
- kubedog-based deploy tracking;
- autogeneration of Chart.yaml based on the werf.yaml when Chart.yaml file is not present in
.helmdir; - secret values; werf does not write secret values into the release data, which will be accessible by
werf helm-v3 get all RELEASE.
- Supported features:
- Lock release by name in werf helm-v3 install/upgrade and werf deploy commands.
- Unsupported yet features:
- adoption of resources by werf.io annotation;
- auto migration of existing helm-v2 release to helm-v3;
werf dismisscommand should use helm-v3 and wait until resources deleted;- extra values and template-helpers to generate full werf-images names in the chart templates.
Installation
More helm-v3 support
- Secret values support for
werf helm-v3subcommands.
Cleanup
- [cleanup] Cleanup meta images
- Reject lazy scanning: check each policy references even it is not needed
- Keep a minimum number of commit hashes in stages storage preserving only the latest commit for each content signature in each policy references
- [tests] Cleanup Suite: init stages storage docker registry options.
- [tests] Cleanup Suite: meta images cleanup
Docs
- [docs] Add new publication: “Werf: Fully customizable GitOps”
Installation
Dockerfile improvements
- [dockerfile] Support for –network and –ssh options (with docs in Configuration/Dockerfile)
- [dockerfile] Fix git head reference failed error when git repository without any commit
Cleanup option to restrict namespaces scan for used images
--scan-context-namespace-only=true|false (false by default), refs https://github.com/werf/werf/pull/2660
[helm-v3] Extra annotations and labels support for helm v3
Set werf.io/version annotation by default in werf helm-v3 upgrade/install commands.
Docs, tests and werf CI fixes
- [tests] Cleanup Suite: fix random test fail
- [tests] Refactor: update RunCommandWithOptions interface
- [tests] Cleanup Suite: test remote stages storage
- [tests] Cleanup Suite: fix quay tests
- [tests] Guides Suite: sync with documentation
- [doc] Fix guide First step
- [ci] Daily Tests Workflow: disable werf debug mode
Installation
Fixes
- [logging] Ignore klog reflector ERROR messages.
- [logging] More details in concurrent builds plan.
- [harbor] Fix NOT_FOUND error during synchronization client id saving.
- [build] Fix getting inspect for built stage.
Installation
Fixes
- [dockerfile] Handle unsupported Media Type (especially v1 manifest version).
- Image metadata by commit: long image name support.
Installation
Logging fixes
- Fix klog warning messages: suppress klog info/warning log levels, print error level through logboek.
- Fix empty size info message for built stage.
- Cleanup logging: hide empty block.
- Fix collapsible GitLab log section:
- apply only for top-level processes;
- disable for blocks.
Installation
Fixes
- [dockerfile] Handle unsupported Media Type (especially v1 manifest version).
- Image metadata by commit: long image name support.
Installation
Helmfile support and fixes
Werf now can be used as helm-3 replacement with helmfile as follows:
-
Install latest helmfile.
-
Install helm-diff plugin (which is needed for the helmfile):
source $(multiwerf use 1.1 alpha --as-file)
werf helm-v3 plugin install https://github.com/databus23/helm-diff
- Prepare helmfile.yaml:
repositories:
- name: stable
url: https://kubernetes-charts.storage.googleapis.com
releases:
- name: mytraefik
namespace: aaa
createNamespace: true
wait: true
chart: stable/traefik
labels:
name: traefik-public
version: 1.86.2
- Set environment variable
WERF_HELM3_MODE=1— this will make werf binary CLI interface helm-compatible, then runhelmfilewith `–helm-binary=werf
.
source $(multiwerf use 1.1 alpha --as-file)
export WERF_HELM3_MODE=1
helmfile -b werf apply
- Check installed release with werf:
werf list --namespace aaa
NOTE Helmfile does not support live output from helm binary, so werf rich live output will be available only after werf command is done. However werf output with --wait flag is more useful than helm’s output even in that condition and werf will “fail fast” when something goes wrong during deploy process.
Refs https://github.com/werf/werf/issues/1606
Refs https://github.com/werf/helm/pull/50
Installation
Parallel builds fixes
- Maximum 5 parallel tasks by default.
- Fix index out of range [0] with length 0.
Installation
Documentation
- Reference/Deploy into Kubernetes: reviewed documentation, rephrased
- Reference/Working with docker registries: upd
Installation
[build] Fix makeslice len out of range (without images in werf.yaml)
Installation
GitHub Actions: fix case sensitive of repository name
[introspection] Fix the input device is not a TTY
Installation
[logging] Fix closing a collabsible GitLab section in parallel jobs
Installation
Support for Jfrog Artifactory implementation
- [docker registry] JFrog Artifactory: fix MANIFEST_INVALID: manifest invalid error
- [docs] Working with Docker registries: JFrog Artifactory
Installation
Concurrent builds of images defined in werf.yaml
-
werf divides all images into dependent sets and runs each set in parallel.
-
Option -p/–parallel to enable/disable concurrency builds (by default on)
-
Option –parallel-tasks-limit to regulate the maximum number of images
-
Introspection does not support parallel builds and disables it
Auto GC in top-level commands only
logboek@v0.4
- GitLab collapsible sections for top-level blocks and processes
Installation
Docs fixes
- [docs] Configuration/Introduction: fix adding file with shell example
- [docs] Configuration/Cleanup: add directives overview
- [readme] Hide code coverage badge
Fix managed images records with capital letters
Ignore image names with capital letters for managed-images records.
Installation
More helm 3 support
Support kubedog-based –wait for ‘werf helm-v3 *’ commands.
For example, run install/upgrade with the following command to get kubedog output:
werf helm-v3 upgrade --namespace myproj myproj . --install --create-namespace --wait
Installation
Fixes
- Fix deploy error: at <.Values.global.env>: nil pointer evaluating interface {}.env.
- Fix docs vulnerability warning: update ruby jekyll and kramdown.
Installation
Fix helm release name not slugged by default anymore
Fix deploy meta config templates defaults.
Installation
[helm-v3] Initial helm v3 support
- Added “werf helm-v3 *” cli commands to access helm-v3 directly.
- Added pkg/deploy/helm_v3 package with basic helm-v3 operations.
- werf helm deploy-chart will use v3 when WERF_HELM3=1 is set.
refs https://github.com/werf/helm/pull/47
refs https://github.com/werf/werf/issues/1606
NOTE: helm v2 and v3 are avaiable at the same time in v1.1.22.
Updated golang dependencies and golang
- K8s deps updated to v0.18.6.
- Other updates related to usage of helm v3.
- Use 1.14 golang.
Installation
Fix helm release name not slugged by default anymore
Fix deploy meta config templates defaults.
Installation
Fix ‘patch does not apply’ error in github actions
"virtual-merge-commit": "true" label in the docker image of the stage was not reset to false when building a new image for regular commit (not virtual merge commit).
Installation
Fix werf cleanup slow deletion of images
[cleanup] Gitlab registry implementation: use deleting a tag method instead of an image
More details in gitlab-org/container-registry documentation:
- https://gitlab.com/gitlab-org/container-registry/-/blob/release/2.9-gitlab/docs/spec/api.md#deleting-a-tag
- https://gitlab.com/gitlab-org/container-registry/-/blob/release/2.9-gitlab/docs/spec/api.md#deleting-an-image
Installation
Synchronization fixes
- Http server main page landing with some info.
- Added http synchronization server docs.
Installation
Public http synchronization server by default in distributed mode
Use https://synchronization.werf.io by default when stages-storage not :local.
Fixes
Fix manifest-cache not filled by manifests of newly built stages. Store manifest of newly built stage into manifest stage immediately after building, before storing this stage into stages storage.
Installation
Synchronization fixes
- Http server main page landing with some info.
- Added http synchronization server docs.
Installation
Synchronization server basic health check procedure
- GET /health gives
{"err":"","echo":"","status":"OK"}. - POST /heath + body
{"echo": "HEY!"}gives:{"err":"","echo":"HEY!","status":"OK"}.
Installation
Http synchronization server rework
Autogenerate persisntent clientID and store in the stages-storage for each project.
Docs translation
Git history based cleanup related doc articles has been translated.
Other fixes
- Use kubernetes locker in cm/werf-synchronization in the namespace of the release to lock releases by release-name. Note that cm/werf-synchronization will be automatically created in the namespace of the release.
- Fix werf converge command: do not require stages-storage and images-repo params when no images has been defined in the werf.yaml.
Installation
Doc CI/CD integration fixes
- Updating supported CI systems + a few corrections.
- Fixed intro for unsupported CI systems.
- Renaming generic CI/CD integration doc.
Installation
New implementation of distributed locking http server: use new version of lockgate
https://github.com/werf/lockgate/pull/26
Installation
Do origin fetch only when git shallow clone is detected
The complete commit history is enough for all commands except images cleanup
Installation
Optimistic git shallow clone handling
- Ignore git shallow clone when it is possible
- Force using git shallow clone despite restrictions with option –allow-git-shallow-clone
The new cleanup algorithm: config syntax updates and fixes
[cleanup] Change configuration syntax: replace createdIn/publishedIn with in
[docs] Configuration/Cleanup: replace createdIn/publishedIn with in
[tests] Cleanup Suite: update syntax
[tests] Cleanup Suite: fix checks are not compatible with monorepo implementations
[cleanup] UNAUTHORIZED error should not be ignored
Installation
Docs and small fixes
[docs] Reference/Cleanup: more details for git history based cleanup
[managed images] Ignore artifact names
[docs] Adding href rels to topnav.html
[docs] Adding rels to slack_popup.html links for security
Installation
Fix dockerfile ONBUILD
[dockerfile] Take into account ONBUILD instructions from base image
The new cleanup algorithm updates
[tests] Cleanup Suite: fix commit commiter when overriding
[cleanup] Improve logging: shortify commit and content signature
[cleanup] Disable meta images fetching by default
Installation
The new cleanup algorithm updates
[cleanup] Update default cleanup policies
[docs] Git history based cleanup
Fix AWS ECR build and publish
Fix AWS ECR build/publish problem ‘Image has to have at least 1 layer’
Create empty zero-bytes layer for manifest-only images: managed images records and image metadata by commit id records.
Other
[docs] Reference/Publish process: fix typo
Installation
Deploy updates and fixes
- Update Helm core: 2.16.1 => 2.16.9.
- Fix support –kube-config-base64 for helm subsystem (fix deploy command failure when –kube-config-base64 option used).
Installation
Rework syncrhonization kubernetes param
- Rework –syncrhonization param (WERF_SYNCHRONIZATION env var): specify all kube-related params in the URI-like string:
-
kubernetes://NAMESPACE[:CONTEXT][@(base64:BASE64_CONFIG_DATA) CONFIG_PATH]
-
- Added –kube-config-base64 param (or WERF_KUBECONFIG_BASE64, or WERF_KUBE_CONFIG_BASE64, or KUBECONFIG_BASE64 env var) support.
- Werf does not generate config file and unpacks specified base64 yaml config into memory.
- Improve validation of kube-connection params validation for synchronization components
- Check KUBECONFIG env var setting and print warnings.
The new cleanup algorithm improvements and fixes
[cleanup] Git history based cleanup: improve algorithm
- Add opportunity to keep images by publishing period.
- Update default policies
- Update cleanup section configuration:
cleanup:
keepPolicies:
- references:
branch: /.*/
tag: /.*/
limit:
last: 10
createdIn: 168h
operator: And
imagesPerReference:
last: 10
publishedIn: 168h
operator: And
[cleanup] Fail purge commands if deletion failed due to using by containers without the force option
[logging] Optimize verbose output
[tests] Cleanup Suite: git history based tests
Other: werf ci, tests, docs and small fixes
[ci] Enable GitHub Docker registry implementation tests
[ci] Fix shallow git clone makes several tests fail
[render] Fix stub env value ENV
Error: bad Helm release ‘project-ENV’ rendered by template ‘[[ project ]]-[[ env ]]’: helm release name should be a valid DNS-1123 subdomain and be maximum 53 chars: a DNS-1123 subdomain must consist of lower case alphanumeric characters, ‘-‘ or ‘.’, and must start and end with an alphanumeric character (e.g. ‘example.com’, regex used for validation is a-z0-9?(.a-z0-9?)*’)
Installation
Added separate kube-connection for synchronization components (part 2, fixes)
Fix –synchronization-kube-context param not working.
Installation
Added separate kube-connection for synchronization components (lock-manager and stages-storage-cache)
Also print warnings when user explicitly specified –kube-config or context, but not specified –synchronization-kube-config or context.
Fixes
Git history synchronization: sync tags only with git >=2.17.0.
[ci_env] GitHub: skip defaults for images repo and stages storage if werf.yaml is not found.
Installation
Cleanup fixes
[cleanup] Fix UNAUTHORIZED: authentication required error with gitlab.com
[cleanup] Policy priority
[cleanup] Fix default policy
[docker_registry] Autodetect gitlab.com implementation
Installation
New synchronization http server
- werf synchronization command
- Implement LockManagerHttpClient LockManager interface compatible.
- Implement LockManagerHttpHandler which respond to corresponding methods of LockManager interface.
- Implement StagesStorageCacheHttpClient StagesStorageCache interface compatible.
- Implement StagesStorageCacheHttpHandler which respond to corresponding methods of StagesStorageCache interface.
- Http handlers can be initialized with arbitrary lock-manager or stages-storage-cache implementation by interface param.
- werf synchronization –local –local-lock-manager-base-dir –local-stages-storage-cache-base-dir (store locks and cache in local filesystem, this server will not keep data on restarts).
- werf synchronization –kubernetes –kubernetes-namespace-prefix=”werf-synchronization-“ (store locks and cache in the kubernetes)
- Implemented syncrhonization server with clientID support.
Not implemented yet:
- –ttl param for werf synchronization command: should set ttl for locks and cache records;
- /new-client-id command for werf-synchronization server to generate new unique clientID.
The new cleanup algorithm
[docs] Reference/Git history based cleanup.
[cleanup] Use creation date for annotated tags instead of head commit date.
Other
[docs] Improve stapel git directive and stapel assembly instructions reference pages grammar and wording.
[docs] Fix broken links
[completion] Improve zsh completion
Installation
[dismiss] Env support for –with-namespace / –with-hooks options
Installation
The new cleanup algorithm improvements
[cleanup] Keep old repo images (without meta images)
[cleanup] Skip image with warning if deletion fails
[logging] More details, small fixes and improvements
Installation
Fix ‘Stage is no longer available in the repo stages storage’ on cleanup
- Only enable strict checking of stage manifest existance when getting stage from stages-storage-cache.
- Ignore bad stages without manifests otherwise
- Print warning for these stages
Installation
Git history synchronization option and shallow clone validation
Installation
The new cleanup
[cleanup] Fix deployed docker images are not logged by default
Fix distributed werf hangs sometimes
The hanging case was in the lockgate library, described in the https://github.com/werf/lockgate/pull/24.
Other
[tests] Cleanup Suite: update test
[ci] Tests Workflow: update go modules caching
[logging] Small improvements and fix
Cleanup legacy: dapp rolling-versions file :tada:
Installation
Fix ‘UNAUTHORIZED: authentication required;’ when saving managed-images and image-metadata-by-commit
Installation
[git] Do not use ‘git worktree add –no-checkout’ option
This option is not avaiable in the v2.17.1 version of git, which is the latest avaiable version for 18.04 lts ubuntu.
Specify target commit instead (which is by the way a faster and more precise approach).
Installation
New cleanup git-history based algorithm
- [cleanup] Implement git history based cleanup
- [cleanup] Git shallow clone validation
- [cleanup] Syncing git branches and tags
- Refine publish procedure: check already existing image by content signature
- Rework publish procedure for cleanup
- Store metadata for images in the stages storage: map stages-signature by commit-id and image-name.
- Store project repository commit in the separate dedicated label: werf-project-repo-commit.
- [publish-for-cleanup] Add image.WerfContentSignatureLabel=werf-content-signature label to all published images
- Set content-signature for all stages into label
Other
- [docs] Reference/Configuration: small style fixes
- [docs] CSS: fix extra padding in lists
- [docs] Guides/GitLab CI/CD integration: update multiwerf installation part
- Init local git repo independently of werf.yaml config git directive usage
Installation
[git] Do not use ‘git worktree add –no-checkout’ option
This option is not avaiable in the v2.17.1 version of git, which is the latest avaiable version for 18.04 lts ubuntu.
Specify target commit instead (which is by the way a faster and more precise approach).
Installation
Enable ‘Waiting for locked …’ log messages for kubernetes locker in distributed werf mode
Installation
Distributed mode fix
- Use locker-with-retry for all kubernetes locks in werf (10 attempts for acquire/release operations are configured).
- Log all failed attempts to the stderr.
More fixes for werf.io landing
- Fix grammar and rephrase sentences.
- Added extra step to introduction presentation.
Installation
Move to werf org
Rename flant to werf in docs and all references.
Other
[docs] Configuration/Running assembly instructions: actualize
Installation
Fix release
[git] Change git repo and worktree cache version to fix compatibility issues
Installation
[config] Add .Files.Glob Go template function
{{ range $path, $content := .Files.Glob ".werf/files/**" }}
{{ $path }}: |
{{ $content | indent 2 }}
{{ end }}
-
Process symlinks with .Files.Glob
.Files.Glob returns the hash of regular files and their contents for the paths that are matched pattern. This function follows only symlinks pointed to a regular file (not to a directory). -
.Files.Glob: has an ability to merge results
The function supports shell pattern matching +**. Results can be merged withmergesprig function (e.g{{ $filesDict := merge (.Files.Glob "*/*.txt") (.Files.Glob "app/ **/*.txt") }})
Installation
[config] Add tpl function to evaluate strings as templates inside a template
Project file wih template:
{{ tpl (.Files.Get "service/werf_partial.yaml") . }}
Environment variable with template:
{{ tpl (env "EXTERNAL_TEMPLATE") . }}
Installation
Fix publish performance for --tag-by-stages-signature
Do not check published docker-image-ID for images being published with content-based stages-signature.
Ability to write published images report json to the file
Write publish report when --publish-report-path PATH option has been given.
There is --publish-report-format options which accepts only json for now.
Installation
Fixes
[ci_env] GitHub: fix typo os.Getenv(“IMAGES REPO”)
[cli] Fix –release and –namespace params: add WERF_RELEASE and WERF_NAMESPACE options
Installation
Fix virtual merge commits creation
- Set user.email and user.name when creating a temporal detached merge commits.
- Fix detached merge commit creation condition.
- Do not create detached merge commit for the previous built image when previously built image contains latest commit.
Installation
[git] Hotfix remote git mapping patch creation procedure
Will fix errors like:
1.505684042s status 128
1.505687944s unrecognized output:
1.505692711s fatal: bad object 07bd04e9e2ad4fb03136943a2bbcb8aa50ff7471
Installation
Fix git worktree switch and optimize working with remote git-mappings
[git] Refine git worktree switch procedure, fix remote git-mapping clone on every build if token has been specified
- do not write excess files
werf_work_tree_cache_dirin the git-repo dir;- new work tree validation procedure: check work tree list of the repo;
- this procedure is needed to check if repo has been recreated to recreate a related work tree in this case;
- store cloned repos in the cache dirs like:
~/.werf/local_cache/git_repos/2/protocol-https/mydomain.mygitlab.com/grp/proj.git. - store worktrees in the dirs like:
~/.werf/local_cache/git_worktrees/5/protocol-git/mydomain.mygitlab.com:22/grp/proj.git.
Installation
Fixes
[docs] Guides/GitHub CI/CD Integration: open code snippets by default
[docs] Fix russian switch-to-distributed-mode guide: lost switch-from-local command example
[docs] Cli docs regen: added –virtual-merge-* options
[docs] Adding source code from git repositories sidebar: documentation: hide irrelevant documentation
Commit message with “[werf reset]” is no longer supported, but documentation section might be reused in the nearest future
[ci_env] GitHub: fix typo os.Getenv(“IMAGES REPO”)
[cli] Fix –release and –namespace params: add WERF_RELEASE and WERF_NAMESPACE options
Installation
Fix publish performance for --tag-by-stages-signature
Do not check published docker-image-ID for images being published with content-based stages-signature.
Installation
[config] .Files.Glob: add ability to merge results
The function supports shell pattern matching + **. Results can be merged with merge sprig function (e.g {{ $filesDict := merge (.Files.Glob "*/*.txt") (.Files.Glob "app/**/*.txt") }})
Installation
[config] Add tpl function to evaluate strings as templates inside a template
Project file wih template:
{{ tpl (.Files.Get "service/werf_partial.yaml") . }}
Environment variable with template:
{{ tpl (env "EXTERNAL_TEMPLATE") . }}
Experimental virtual/ephemeral merge commit support for GitHub Actions
Werf stages related commands accepts following args:
--virtual-merge;--virtual-merge-from-commit;--virtual-merge-into-commit.
In virtual merge mode werf will try to reuse stages built for virtual merge commits.
Cli fixes
Implement env support for some options:
- WERF_DRY_RUN: –dry-run (run / cleanup commands)
- WERF_DOCKER_OPTIONS: –docker-options (run)
- WERF_KUBE_CONFIG: –kube-config
- WERF_PUBLISH_REPORT_PATH: –publish-report-path
Docs
Translated Guides/GitHub CI/CD integration guide.
Installation
Docs updates for GitHub Actions
[docs] Guides/GitHub CI/CD Integration: Up configurations and add note for weak workflows
[ci_env] GitHub: Reorganize cleanup section in env-file
Installation
Fixes and small changes
[docs] Update activesupport in werf.io due to vunlerability alert
[docs] Guides/GitHub CI/CD integration: add werf-actions
[dismiss] Return exit code 0 when release is not found
[cli] Add envs support for string array flags
- WERF_SSH_KEY* for –ssh-key
- WERF_SET* for –set
- WERF_SET_STRING* for –set-string
- WERF_VALUES* for –values
- WERF_SECRET_VALUES* for –secret-values
Installation
Fixes
- [build] Fix remote git-mapping uses wrong commit-id for ls-tree when creating git-archive.
- [config] Process symlinks with .Files.Glob.
.Files.Globreturns the hash of regular files and their contents for the paths that are matched pattern.
This function follows only symlinks pointed to a regular file (not to a directory).
- [docs] Add converge command and actualize some information.
- [docs] Reference/CI/CD workflows overview: Align tables content
- [docs] Sidebar: Add Организация CI/CD с помощью werf
- [docs] Fix broken links
- [docs] Regen cli docs: added converge command cli docs
- [docs] Reference/Working with Docker registries: JFrog and Nexus note
- [docs] Stapel Image/Running assembly instructions: actualize
- [docs][readme_ru] Actualize
- [ci_env] Fix redundant empty lines when using ci-env-file with verbose
Installation
Ability to write published images report json to the file
Write publish report when --publish-report-path PATH option has been given.
There is --publish-report-format option which accepts only json for now.
Example of publish report:
{
"Images": {
"one": {
"WerfImageName": "one",
"DockerRepo": "localhost:5000/alo/one",
"DockerTag": "a64687f43f31179cf069574a455e53d75937da81f82042fbfc7ad096",
"DockerImageID": "sha256:0151dda79c0c076aa00b4c90367d9e067e854fdb10027fc06e30ed0f1498016f"
},
"three": {
"WerfImageName": "three",
"DockerRepo": "localhost:5000/alo/three",
"DockerTag": "baff2bc90cf6c17e47e37e7b14ce0296ade95cbf35f7d33d3e72d0bb",
"DockerImageID": "sha256:c71998ef1916a6fddee0b334a28d2efa1f9f541fa0094c3f5f7aad6d2d98b44f"
},
"two": {
"WerfImageName": "two",
"DockerRepo": "localhost:5000/alo/two",
"DockerTag": "f5a8ee43e363925176e78230a467fa7e9e58324916f4be9e0b7ad6d7",
"DockerImageID": "sha256:1edd54ae5a18734a268ddeb9ea8eeb83137ac71dd8c0cf52afdc5c5039c0360d"
}
}
}
Installation
Docs
- NEW! Translated to english Reference/CI/CD workflows overview.
- NEW! Translated to english Guides/GitLab CI/CD integration.
- Replaced external links to image with local links.
- Reference/Configuration: .Files.Glob Go template function.
- Guides/GitLab CI/CD integration: Update scheme image.
Add .Files.Glob Go template function
{{ range $path, $content := .Files.Glob ".werf/files/**" }}
{{ $path }}: |
{{ $content | indent 2 }}
{{ end }}
Allow storing of arbitrary docker images in the stages storage
Ignore stages storage tags not matching the expected format for the remote stages storage of the werf.
Virtual merge commit support for git_repo and true_git
- New way of creating worktree for git-worktree-cache: use “git worktree add” tool.
- This way of creating worktrees is native for git and prevent main worktree corruption.
- New true_git.CreateDetachedMergeCommit function which operates within the git-worktree-cache, creates and returns some commit id, whic
h can be used later until this commit is pruned (because it is detached). - Playground demo cli tool which creates diff between two virtual-merge-commits: specify 4 commits on input, get a patch between virtual
merge commits of 1 into 2 and 3 into 4.
Fix GetStageDescription when docker registry repo is not exist
Error: reading image “docker.pkg.github.com/user/repo/stages:de07eea0119baca706320aa6b7f32f887f5700b6fd82e1864867a1ee-1590064872402”: GET
https://docker.pkg.github.com/v2/user/repo/stages/manifests/de07eea0119baca706320aa6b7f32f887f5700b6fd82e1864867a1ee-1590064872402: NAME_U
NKNOWN: docker package “stages” does not exist under owner “user/repo”
Ci-env
- GitHub: default WERF_REPO_GITHUB_TOKEN and optional werf config.
- GitLab: improve stages storage/images repo auto-detection.
Installation
Fix hard-coded images repo/stages storage implementation for ci-env
Automatically set gitlab docker registry implementation for stages storage and images repo.
Installation
Ci-env mode option for github actions
Add –as-env-file / –output-file-path options.
Fixes
Add cli docs for ‘werf converge’ command.
Installation
Added ‘werf converge’ command
Shortcut for ‘werf stages build’, ‘werf images publish’ and ‘werf deploy’ commands.
Command always builds all images defined in the werf.yaml and uses content-based-tagging and three-way-merge mode.
Command is supposed to bring your application git state into some environment of Kubernetes cluster in one shot: build and publish images defined in the werf.yaml, then release application chart.
Installation
Added ability to use slash “/” and plus “+” signs in werf.yaml image names
Installation
Docs
- Guides/GitHub CI/CD integration.
- Add distributed-werf reference info and switch to distributed guide.
Installation
Fixes
[dockerfile] Fix COPY –from arbitrary image (not only dockerfile stage)
Docs
Guides/GitLab CI/CD integration: added overview
Installation
Fixes
- Fix kubernetes namespace slug: should not allow capital letters.
- Fix git-related stages selection when multiple git-mappings by the same url used.
https://github.com/flant/werf/issues/2407
Installation
Fix v1.1 stage selection algorithm
Fix for a case when two images of the same stage for two consecutive commits are building at the same time. This can lead to the bug, because werf uses oldest stage by creation timestamp and not oldest by save-into-stages-storage timestamp.
Use selection based on saving-into-stages-storage-timestamp instead of docker-image creation timestamp.
After this change stages-storage-cache will be invalidated automatically.
Installation
Docs
New CI/CD articles in russian:
- CI/CD workflows overview: https://ru.werf.io/v1.1-alpha/documentation/reference/ci_cd_workflows_overview.html
- GitLab CI/CD guide: https://ru.werf.io/v1.1-alpha/documentation/guides/gitlab_ci_cd_integration.html.
English versions are coming soon.
Installation
Fix “@ERROR: invalid gid nogroup” on imports stage
The error occurs when artifact image does not contain “nogroup” group, which was used by default when running rsync server. For example when using “maven:3.6.3-jdk-13” as artifact base image.
Fixed by running rsync server using root group.
Retry docker pull/push on internal server error
Werf automatically retries pull/push operations on internal server errors.
Installation
Docs
New CI/CD articles in russian:
- CI/CD workflows overview: https://ru.werf.io/v1.1-alpha/documentation/reference/ci_cd_workflows_overview.html
- GitLab CI/CD guide: https://ru.werf.io/v1.1-alpha/documentation/guides/gitlab_ci_cd_integration.html.
English versions are coming soon.
Installation
Fix “@ERROR: invalid gid nogroup” on imports stage
The error occurs when artifact image does not contain “nogroup” group, which was used by default when running rsync server. For example when using “maven:3.6.3-jdk-13” as artifact base image.
Fixed by running rsync server using root group.
Retry docker pull/push on internal server error
Werf automatically retries pull/push operations on internal server errors.
Installation
[completion] Support zsh
# Load zsh completion
source <(werf completion --shell=zsh)
Fixes
Fixed panic during sub chart validation
- https://github.com/flant/helm/pull/44
- refs https://github.com/flant/werf/issues/2388
Installation
Custom helm chart directory (.helm) and werf configuration file (werf.yaml) options
- Add –helm-chart-dir option.
- Add –config / –config-templates-dir options to override default paths.
Installation
Fix dockerInstructions stage not isolated when used after gitCache
After this release dockerInstructions signature will change if used after gitCache. Stage-not-exists in stages-storage errors may occur in this case.
Other
Detailed error messages for git submodules when submodule exists in the repo in the incorrect state.
[test] Test “Check broken links” hase been updated.
Installation
Fix cleanup deletes stages that are related to deployed images
Installation
Fix cleanup fails to delete stages from repo stages storage
Automatically set gitlab repo implementation option (--stages-storage-repo-implementation=gitlab) in werf ci-env gitlab.
Other
[docs] Added three new publications about werf.
[docs] Publications on the site updated. Added three new publications about werf.
[docs] Fix README coming soon and features list
Installation
Introducing distributed builds
Werf now can store stages in the docker registry by specifying --stages-storage REPO instead of --stages-storage :local.
There is no more restriction that a single host should be used to run all werf commands for a project:
- All werf commands that need an access to the stages should specify the same stages storage.
- It is irrelevant on which host werf command is running as long as the same stages storage is used for the commands like: build, publish, cleanup and deploy.
- Also all hosts which run werf should have a connection to the Kubernetes cluster, because werf uses Kubernetes to synchronize multiple build/publish/deploy processes running from different machines.
“werf ci-env gitlab” default mode now is to store stages by path CI_REGISTRY_IMAGE/stages. All existing werf installations will continue to use :local stages storage because --stages-storage :local param has explicitly been specified in all CI-configs. So to use distributed builds user needs to remove this param, see guided steps below about migrating to the distributed stages storage.
Synchronization, caches and cleanup
Introduced a new parameter named --synchronization, which controls how multiple werf processes are synchronized between each other. There is a stages-storage cache and locks components that werf needs to run. Stages storage cache is an internal component used to boost werf performance for idle builds when calculating stages. Locks are needed to organize correct publishing of new stages into stages-storage, publishing images into images-repo and for deploy process also.
When --synchronization :local used (which is factually by default in the previous versions < v1.1.10), then stages-storage cache reside in the local host file system (~/.werf/shared_context/storage/stages_storage_cache/1/PROJECT) and local file locks are used.
When --synchronization kubernetes://NAMESPACE is used, then werf creates a ConfigMap for project in this namespace, stores stages storage cache in this ConfigMap and uses distributed locking over this ConfigMap to synchronize multiple werf processes running from multiple hosts.
By default, when non local stages-storage is used werf automatically use kubernetes://werf-synchronization namespace and create a ConfigMap named werf-PROJECT in this namespace. User can specify arbitrary non-standard namespace on own needs.
What about local docker images cache on hosts where werf is running? — There is such a cache, which is cleared either by the werf itself or can freely be removed by other tools (docker rmi
.
New commands to sync and switch stages
werf stages sync --from=:local|REPO --to=:local|REPO.- Command will copy stages from one stages-storage to another.
- Command will copy only difference of stages from one stages-storage to another.
- Furthermore command will copy multiple stages in parallel.
- Command run result is idempotent: sync can be called multiple times, interrupted, then called again — the result will be the sam
. - There are delete options:
--remove-sourceand--cleanup-local-cache, which controls whether werf will delete synced stages from source stages-storage and wheter werf will cleanup localhost from temporary docker images created during sync process. - This command can be used to download project stages-storage to the localhost for development purpose as well as backuping and migrating purposes.
werf stages switch-from-local --to=REPO- Command will automatically sync existing stages from :local stages storage to the specified REPO.
- Command will block project from being used with :local stages-storage.
- This means after werf stages switch-from-local is done, any werf command that specifies :local stages-storage for the project will fail preventing storing and using build results from different stages-storages.
- Note that project is blocked after all existing stages has been synced.
How to migrate to distributed mode
Let’s say you have some project in Gitlab CI, that uses werf. Steps:
- Go to the directory where your project reside on the runner host.
- Create an MR in the project, that removes
--stages-storage :localparam from all werf invocations. There is no need to set this param explicitly anymore, becausewerf ci-envwill automatically set it toCI_REGISTRY_IMAGE/stages. - Run
werf stages switch-from-local --to=CI_REGISTRY_IMAGE/stageswhere CI_REGISTRY_IMAGE is the address of container registry for your gitlab project. After this step all werf invocations from different git branches of the project that uses--stages-storage :localwill refuse to work, because switch-from-local command sets a block to use :local. - Merge your MR with correct stages-storage param.
- All git branches that need to run werf should rebase then onto the new changes to use new stages-storage, otherwise werf will refuse to use
:localstages storage.
Other changes since last release v1.1.10-alpha.7
[docs] Remove –stages-storage :local from Gitlab CI docs
[ci] Cache GO modules
[tests] New tests
[tests] K8S/Cleanup Suite: test cleanup with deployed image
[tests] K8S/Guides Suite: actualize after each
Error: –stages-storage=ADDRESS param required
[tests] Helm/Render Suite: add extra annotations and labels check
[tests] Cleanup Suite: test stages storage repo
Installation
Github Actions support (alpha)
[ci-env] GitHub support.
[testing] CI-env Suite: add github.
werf ci-env github --as-file
Remote storages support: fix lockgate ConfigMap data key invalid
https://github.com/flant/lockgate/pull/12
Refactor and fix kubernetes cm key name
- Renamed LockHandle.ID to LockHandle.UUID to be more clear.
- Use “lockgate.io/SHA3_224(lock-name)” for ConfigMap key names due to ConfigMap names restrictions.
Other
Fix provided shell ‘WERF_SHELL’ not supported.
Retry docker pull/push when specific errors occur.
[logging] Turn off Docker Registry API debug messages by default.
[ci] Fix Cleanup Docker Hub test repositories.
[ci] Split test workflow into two independent workflows.
[ci] Disable GitHub Packages in cleanup test suite.
[docs] Regen commands cli documentation reference.
Installation
Remote storages support
- Refactor –synchronization param:
-Sshorthand;- synchronization=:local by default when –stages-storage=:local;
- synchronization=kubernetes://werf-synchronization by default when stages-storage != :local;
- ability to specify arbitrary namespace with
--synchronization kubernetes://mynamespace.
- Auto reset stages-storage-cache and retry action when docker-image has been deleted from stages-storage, but exists in the stages-storage-cache.
Fixes
Fix gitLatestPatch signature not stable to index line in patches.
Installation
Remote storages support: fix kubernetes locks
Distributed builds can be used now by specifying –stages-storage=REPO_ADDRESS.
- Fixed bugs in kubernetes locks subsystem.
- Fixed panic in ‘werf images purge’ due to missed synchronization param setup
Werf ci-env refactor
Added –as-file option, which is more portable way to use werf ci-env. Example:
. $(werf ci-env gitlab --as-file)
Changed all docs and guides to use --as-file.
Installation
Remote storages support
- Refactor –synchronization param:
-Sshorthand;- synchronization=:local by default when –stages-storage=:local;
- synchronization=kubernetes://werf-synchronization by default when stages-storage != :local;
- ability to specify arbitrary namespace with
--synchronization kubernetes://mynamespace.
- Auto reset stages-storage-cache and retry action when docker-image has been deleted from stages-storage, but exists in the stages-storage-cache.
Fixes
Fix gitLatestPatch signature not stable to index line in patches.
Installation
Fix release
Fix problems related to changes introduced in v1.1.10-alpha.2: signatures has been accidentally changed in v1.1.10-alpha.2.
Fixes that were made in v1.1.10-alpha.2 are also available in this v1.1.10-alpha.3 release:
[cleanup] Skip images with MANIFEST_UNKNOWN error
[build][git] Handle “commits not present” line in git diffs
[build][git] Fix gitLatestPatch not built when previous stage has been discarded during build
- remove git-building-stage map from Conveyor;
- remove AfterSignatureCalculated Stage hook.
[dockerfile] Fix ONBUILD instruction processing
- dockerfile stage signature should take into account ONBUILD value;
- dockerfile stage that uses another one with ONBUILD instructions should calculate files checksum for COPY/ADD trigger instruction in addition to ONBUILD value.
Installation
Fix dockerfile builder
[dockerfile] Fix checksum calculation algorithm does not count ignored files (.gitignore).
https://github.com/flant/werf/issues/2315
Remote storages support: distributed builds support
‘werf switch-from-local’ command implemented
werf switch-from-local –to registry.mycompany.com/mygroup/myproject
This command will:
- Sync existing local stages to the specified stages storage.
- Globally lock stages and images after sync.
- Set block to prevent any werf-command with –stages-storage=:local:
user should change this param to the new stages-storage repo. - Sync existing local stages to the specified stages storage second time.
Implemented stages-storage-cache in ConfigMap
- Store stages-storage-cache in ns/werf-synchronization cm/werf-PROJECT_NAME
when--synchronization=:kuberneteshas been specified.- The same ConfigMap also used for kubernetes-based locks in annotations
when--synchronization=:kuberneteshas been specified.
- The same ConfigMap also used for kubernetes-based locks in annotations
Kubernetes based locks support
When --synchronization=:kubernetes werf will use kubernetes-locks in
ConfigMap for project: cm/werf-PROJECT_NAME.
Installation
Fixes
[cleanup] Skip images with MANIFEST_UNKNOWN error
[build][git] Handle “commits not present” line in git diffs
[build][git] Fix gitLatestPatch not built when previous stage has been discarded during build
- remove git-building-stage map from Conveyor;
- remove AfterSignatureCalculated Stage hook.
[dockerfile] Fix ONBUILD instruction processing
- dockerfile stage signature should take into account ONBUILD value;
- dockerfile stage that uses another one with ONBUILD instructions should calculate files checksum for COPY/ADD trigger instruction in addition to ONBUILD value.
Installation
Fix dockerfile builder
[dockerfile] Fix git status checksum algorithm does not count symlinks properly
[dockerfile] Skip outside the build context wildcard till fail on docker build
[dockerfile] Take into account ignored files by .gitignore files
[dockerfile] Change checksum calculation without git logic
- Add relative file path and mode to checksum
- Add symlink link to checksum instead of linked data
[dockerfile] Fix git status with not initialized submodules
Installation
Remote storages support: safe switch between alpha and beta werf versions
Invalidate stages-storage-cache directory when old cache directory exists. Remove old and current stages-storage-cache directories in this case.
Installation
Fixes and docs
- Use 1.1 stable in README installation and docs.
- Fix: change stages-storage-cache dir to
~/.werf/shared_context/storage/stages_storage_cache/1/.
Installation
Support non-default docker registries implementations and dockerhub
- AWS ECR, Azure CR, GCR, GitHub Packages, GitLab Registry, Harbor, Quay.
- Documentation: https://werf.io/v1.1-beta/documentation/reference/working_with_docker_registries.html
Support stages-storage in docker registry
For now only when building from single host. Building from multiple hosts will be available soon in v1.1.
Installation
New builder features ported from v1.1 beta to v1.1 alpha
- build artifacts only by
werf build -s :local ARTIFACT ARTIFACT; - import from certain artifact/image stage by specifying
stagedirective in imports.
Fixes
Fix gitLatestPatch signature differs in different worktrees: handle git-diff patch index line in v1.1 compatible manner.
Werf stages-storage commands and changes
- improved ‘werf stages sync’ command: implemented –remove-source and –cleanup-local-cache options;
- implemented restriction for project to use only single stages-storage at a time,
use ‘werf stages switch’ command to switch stages-storage for the project. - removed ‘werf stages mv’ command, use ‘werf stages sync –remove-source’ + ‘werf stages switch’ instead.
Installation
INCOMPATIBLE CHANGE Fix stages and images isolation problem for git-latest-patch and stages-signature
-
gitLatestPatch stage was not isolated between different branches when the same changes
has been made in two different commits: added commits ancestry check when selecting
suitable stage from cache for gitLatestPatch. The signature of gitLatestPatch stage
has not been changed. -
Stages-signature was not isolated between different branches when the same
changes has been maed in two different commits. IMPORTANT stages-signature
calculation algorithm has been changed, stages-signature will change for all projects
that use stages-signature tagging strategy (content-based).
Remote storages support: more registries support, docs and fixes
- Full support for Azure CR implementation.
- Documented docker registry implementations.
- Added separate stages docker images manifests cache in
~/.werf/local_cache/manifests/1/DOCKER_IMAGE_NAME_SHA256. - Added global locks for build, publish and images cleaning commands, to prevent collissions: only a single cleanup command
can be active at one time, no build, publish or deploy command is allowed while cleanup command is wokring;
but build, publish and deploy commands can work simultaneously.
Installation
Fix cleanup-procedure that breaks builds and descriptive error in that case
- Cleanup procedure in the previous version was not cleaning stages-storage-cache
~/.werf/local_cache/stages_storage_v3, while deleting images from stages-storage. This behaviour lead to unexpected error during build-procedure, like: “Error response from daemon: No such image: sha256:69dcb14a44993b7…”. - When image has been found in the stages-storage-cache and does not exists in the stages-storage werf will print descriptive error prompting to delete stages-storage-cache. In the future versions werf will reset build automatically in this case.
Installation
Remote storages support: fix ‘error creating docker registry accessor for repo … registries must be valid RFC 3986 URI’
Support registry or Docker Hub account as –images-repo value.
Installation
Fix extra annotations/labels are not added to k8s resources
Installation
Remote storages support: fix ‘error creating docker registry accessor for repo … registries must be valid RFC 3986 URI’
Support registry or Docker Hub account as –images-repo value.
Installation
Remote storages support: distributed builds
- optimized distributed builds by not cleaning up built and fetched stages from localhost — these will be reused as a cache on retrie
; - added ‘werf stages mv’ command prototype to move built stages from one stages storage to another.
- fix panic ‘github.com/flant/werf/pkg/build/stage.(*BaseStage).getServiceMountsFromLabels’, which occured when base image has been pulled from the registry.
Remote storages support: more registries available and fixes
- full support for AWS ECR implementation;
- enabled cleanup for stages storage when docker registry is used as a stages storage;
- fix “UNSUPPORTED: Invalid parameter at ‘maxResults’” when using ECR, by updating github.com/google/go-containerregistry, the issue: https://github.com/google/go-containerregistry/issues/681.
- support registry/account as –images-repo value;
- detailed error when user specifies incorrect credentials;
- add options to specify Docker Hub token;
- auto images repo mode: default repo mode for images repo implementation.
Other
- fix lint and render panic: –insecure-registry, –skip-tls-verify-registry for images repo and stages repo by default, change stub “IMAGES_REPO” to “stub/repository”.
Installation
More docker registries implementations support
- harbor;
- GitHub packages registry.
Installation
Remote storages support: first working build with –stages-storage=REPO + fixes
-
fixed panic ‘should_be_built_phase.go:47 +0x45’;
-
working
werf build --stages-storage=REPOcommand on the single host for now.
Installation
Remote storages support (1): full dockerhub support
Added ability to cleanup images published into dockerhub.
Specify following params for werf build-and-publish or werf publish commands:
--images-repo-docker-hub-password='':
Docker Hub password for images repo implementation (default $WERF_IMAGES_REPO_DOCKER_HUB_PASSWORD or
$WERF_REPO_DOCKER_HUB_PASSWORD).
--images-repo-docker-hub-username='':
Docker Hub username for images repo implementation (default $WERF_IMAGES_REPO_DOCKER_HUB_USERNAME or
$WERF_REPO_DOCKER_HUB_USERNAME).
--images-repo-implementation='':
Choose images repo implementation.
The following docker registry implementations are supported: ecr, default, dockerhub, gcr, github, gitlab, quay.
Default $WERF_IMAGES_REPO_IMPLEMENTATION, $WERF_REPO_IMPLEMENTATION or auto mode (detect implementation by a registry).
For example:
source <(multiwerf use 1.1 alpha)
...
export WERF_IMAGES_REPO_DOCKER_HUB_USERNAME=myuser
export WERF_IMAGES_REPO_DOCKER_HUB_PASSWORD=password
export WERF_IMAGES_REPO_IMPLEMENTATION=dockerhub
source <(werf ci-env gitlab)
werf build-and-publish -s :local
It is better to define WERF_IMAGES_REPO_DOCKER_HUB_USERNAME, WERF_IMAGES_REPO_DOCKER_HUB_PASSWORD and WERF_IMAGES_REPO_IMPLEMENTATION variables as gitlab secret environment variables.
Installation
An option to enable cleanup for stages-signature tagging strategy
When running werf in ci with werf ci-env command, then either:
-
create cleanup config for the host:
mkdir ~/.werf/config echo "stagesSignatureStrategyExpiryDays: 30" >> ~/.werf/config/cleanup.yaml echo "stagesSignatureStrategyLimit: 50" >> ~/.werf/config/cleanup.yaml -
export environment variables in your CI/CD system:
WERF_STAGES_SIGNATURE_STRATEGY_LIMIT=50andWERF_STAGES_SIGNATURE_STRATEGY_EXPIRY_DAYS=30.
By default werf will not cleanup images published with stages-signature tagging strategy and related stages.
Installation
[deploy] Hotfix werf helm render: fix problem with ‘|’ rendered as ‘|-‘
Change gopkg.in/yaml.v2 to github.com/ghodss/yaml in custom werf templates engine,
as github.com/ghodss/yaml is used everywhere in the helm 2.
In the helm 3 sigs.k8s.io/yaml is used, but it is not possible for now to switch to sigs.k8s.io/yaml before switching to helm 3 codebase.
https://github.com/helm/helm/blob/master/pkg/engine/funcs.go#L27
Installation
Secret unsafe option to disable git minimum version constraint
Set WERF_DISABLE_GIT_MIN_VERSION_CONSTRAINT=1 to disable min git version constraint.
Installation
Docs
Added rss feeds for werf releases.
Fixes
[dockerfile] Fail build if dockerfile COPY instruction refers to nonexistent stage.
Installation
[dockerfile] Fix dockerfile stage name from “COPY –from” is not detected
Docker parser coverts dockerfile stage name (AS) to lowercase but in places of the name usage (COPY –from) does not.
This affected dockerfile stage signature calculation and occurred false warning messages.
Installation
Fix release
[dockerfile] Fix calculating context files checksum when submodule is used
Installation
Fix git problems release
Fix creating an archive fails when nested submodules is used
Installation
Fix git problem release
Fix creating an archive fails when submodule dir/file is specified in git.add
Installation
Official v1.1 beta
Implemented content based tagging and new stages storage architecture as a big step toward distributed and concurrent builds.
New stages format and cache selection algorithm
New stage name generation rule. Every build of the stage generates uniq stage-cache image name, which consists of 2 parts: signature (as in v1.0) plus unique identifier.
For example, full stage image name will look like:
werf-stages-storage/myproject:d2c5ad3d2c9fcd9e57b50edd9cb26c32d156165eb355318cebc3412b-1582656767835
werf-stages-storage/PROJECT:SIGNATURE-TIMESTAMP_MILLISEC
Signature identifier of the stage represents content of the stage and depends on git history which lead to this content.
Werf stage selection algorithm is based on the git commits ancestry detection:
-
Werf calculates a stage signature for some stage.
-
There may be multiple stages in the stages storage by this signature, werf selects all suitable stages by the signature.
-
If current stage is related to git (git-archive, user stage with git patch or git latest patch), then werf selects only
those stages which are relaed to the commit that is ancestor of current git commit.
-
Select from the remaining stages the oldest by the creation timestamp.
There may be multiple built images for a single signature.
Stage for different git branches can have the same signature, but werf will prevent cache of different git branches from
being reused for totally different branch.
For more info see documentation: https://werf.io/v1.1-beta/documentation/reference/stages_and_images.html#stage-naming.
New stage building and saving algorithm
If suitable stage has not been found during stage selection, werf starts building a new image for stage.
Note that multiple processes (on a single or multiple hosts) may start building the same stage at the same time. Werf uses optimistic locking when saving newly built image into the stages storage: when a new stage has been built werf locks stages storage and saves newly built stage image into storage stages cache only if there are no suitable already existing stages exists. Newly saved image will have a guaranteed unique identifier
by TIMESTAMP_MILLISEC. In the case when already existing stage has been found in the stages storage werf will discard newly built image and
use already existing one as a cache.
In other words: the first process which finishes the build (the fastest one) will have a chance to save newly built stage into the stages storage. The slow build process will not block faster processes from saving build results and building next stages.
For more info see documentation: https://werf.io/v1.1-beta/documentation/reference/stages_and_images.html#stage-building-and-saving.
Benefits of new stages storage architecture
This new stages storage architecture opens up an opportunities to build stages concurrently and even distributely
from multiple build machines. For now there are no theoretical limitations of current werf architecture to implement
distributed or concurrent builds. So we consider this release as a big step towards distributed and concurrent builds.
Werf have an --stages-storage and --synchronization parameters which specify addresses of the stages storage and
synchronization lock manager. For now there is only :local implementations of both primitives. By changing these implementations to docker-registry for the stages storage and Redis or Kubernetes server based synchronization lock manager, werf will implement distributed builds out of the box.
Content based tagging
Werf v1.1 supports so called content based tagging. Tags of resulting docker images depend on the content of these images.
When using werf publish --tags-by-stages-signature or werf ci-env --tagging-strategy=stages-signature werf will tag result images by so called image stages signature. Each image tagged by own stages signature which calculated by the same rules as regular signature of image stage.
Image stages signature depends on content of the image and depends on the git history which lead to this content.
There may be dummy commits into the git repo that do not change resulting images. For example empty commits, merge commits or commits which change files that are not imported into the resulting image.
When using tagging by git-commits these dummy commits will cause werf to create new images names even if content of these images is the same. New images names in turn will cause restarts of application Pods in Kubernetes which is totally not a desired behaviour. All in all this is the reason preventing storing multiple application services in the single git repo.
Stages signature on the countrary will not change on dummy commits, so these commits will not cause restarts of application Pods in kubernetes, yet it similarly to commit-id relates to the git history of edits and depends on the content of the files.
Also tagging by stages signatures is more realiable tagging method than tagging by a git-branch for example, because resulting images content does not depend on order of pipelines execution. Stages signature leads to stable immutable images names which represent the address of the certain image content.
NOTE From now and on stages-signature is the default tagging strategy and the only recommended one for usage.
For more info see documentation: https://werf.io/v1.1-beta/documentation/reference/publish_process.html#content-based-tagging.
Improved performance of stages selection and verification
Idle builds when all stages are exist in the stages storage are really fast now (in the most cases build retry will run under 1sec).
Stages verification performance during werf deploy and werf run commands also improved a lot.
Improved performance of Dockerfile builder
Werf Dockerfile builder signature calculation performance is improved due to using git ls-tree checksums
for docker context files. Signature calculation does not depend on the docker context size.
Improved performance of artifact/image imports
Werf uses rsync server to import files from artifacts and images.
MacOS imports performance does not depend on docker implementation of volumes now. MacOS imports performance is the same as in Linux and Windows.
Other changes
-
Added
--log-debug,--log-verboseand--log-quietmodes. -
Removed legacy params and code (dapp secret key legacy support removed;
--insecure-repolegacy option removed, only –skip-tls-verify-regis