Guides on

Guide: docker compose

Sun, 01 Aug 2021 22:10:01 +0200

Docker Compose

In this scenario, we have to maintain a docker-compose file and we want to be sure that we always use the latest available docker.

Requirement

Updatecli
IDE
A GitHub Personal Access Token

Updatecli Binary

Updatecli is a declarative dependency management tool. The application is available as a command line compatible with Linux/macOS/Windows. Updatecli is designed to work the same way, both from a local machine and a CI environment.

It can easily be installed using Homebrew by running the two following commands:

-> brew tap updatecli/updatecli
-> brew install updatecli

Additional installation instructions are available at www.updatecli.io/docs/prologue/installation.

IDE

The best way to write Updatecli manifest(s) is by using an IDE compatible with jsonschema hosted on schemastore.org, such as Vscode, Intellij, or Neovim. The full list of compatible IDE is available on www.schemastore.org.

For the IDE to load the correct Updatecli jsonschema, Updatecli manifest must have both a parent directory named "updatecli.d" and one of the file extension ".yaml", or ".yml". This provides auto-completion and validation out of the box.

For example, from VScode, typing [ctrl][space] should display a box with various suggestions

GitHub Personal Access Token

As we are going to interact with resources on Github such as github repository and GitHub packages, we need a personal access token. More information on the GitHub documentation creating-a-personal-access-token

Description

Docker image versions are identified using two methods, either the docker image tag or the docker image digest.

Tag

A docker image tag is useful when we want to have an easy to identify human-readable version like in nginx:1.17 where 1.17 is the tag. Unfortunately, a docker image tag can be easily overridden, whether on purpose or not. As when using the tag latest. This means that a docker image tag can’t be considered as a source of truth.

Digest

The second method to identify a docker image version is by using its digest like in nginx@sha256:8269a7352a7dad1f8b3dc83284f195bac72027dd50279422d363d49311ab7d9b where the string following nginx@sha256: is the digest. A digest is a hash referencing a specific version of a docker image tag, in this case, nginx:1.17. A digest is immutable, we can always rollback to a specific digest if needed. However, it’s not convenient to identify which version we’re referencing without a little further investigation.

The problem could be summarized as, "do we want to only use a well-defined docker image by using digests but with the cost of losing readability?" or, "are we accepting the risk of running an unwanted version by using image tags?"

It depends on how important we value readability versus uniqueness, but a useful rule of thumb is to consider the docker image build strategy. What’s the risk of running an unwanted version and how big is the chance of having to rollback. If we only build one image by version and that version is regularly updated then preferring readability by using docker image tag seems reasonable otherwise in any other scenarios it’s usually safer to use docker image digest.

Now that we know what to update and why, let see how to do it using updatecli.

Docker Image Tag

When we want to automate image tag updates, we have to identify the process used to define those tags, and if that process is automated. Then you can more than likely also automate the version update and this is where updatecli can play a role.

Using updatecli, you specify this process in the source stage, then if needed you check a condition and if the condition is satisfied then you update the version in your target file. Considering again, the example of regularly updating the Jenkins version. We know that a release is published every week on a maven repository, then a docker image build is published on DockerHub. Considering the build strategy, it would seem to be fine to use a docker image tag.

Let’s consider this docker compose file, which the three examples below all share:

docker-compose.yaml

---
version: '3'
services:
 jenkins:
 image: jenkins/jenkins:2.245
 ports:
 - 8080:8080

Now we write an updatecli configuration with a source, conditions, and one target

updatecli.1.yaml

name: Example 1

sources:
 jenkins:
 name: "Get latest Jenkins version from maven repo"
 kind: maven
 transformers:
 # - addprefix: "v"
 - addsuffix: "-jdk11"
 spec:
 url: "repo.jenkins-ci.org"
 repository: "releases"
 groupid: "org.jenkins-ci.main"
 artifactid: "jenkins-war"

conditions:
 docker:
 name: "Docker Image Published on Registry"
 kind: dockerimage
 spec:
 image: "jenkins/jenkins"
 architecture: "amd64"

targets:
 imageTag:
 name: "Update docker-compose jenkins service"
 kind: yaml
 transformers:
 - addprefix: "jenkins/jenkins:"
 spec:
 file: "./docker-compose.yaml"
 key: "services.jenkins.image"

updatecli pipeline diff --config updatecli.1.yaml

Updatecli queries the maven repository 'releases' located on repo.jenkins-ci.org, to determine the latest version. If it finds one, a source transformer appends the suffix -jdk11, as we are looking specifically for the java 11 version, and a target transformer prepends the prefix jenkins/jenkins: to build the full image reference. We validate that the docker image exists on DockerHub and then we test that the value in the file 'docker-compose.yaml' for the key services.jenkins.image is correct. If not then we bump it.

If happy with the changes then we apply them by using apply instead of diff.

updatecli pipeline apply --config ./updatecli.1.yaml

Docker Image Digest

As explained earlier, sometimes it’s preferable to use the docker image digest, if we need to rely on a specific version, or if we want to easily rollback. Most of the configuration remains the same as the previous example. This time the source is of kind dockerdigest and we don’t test if an image exists on DockerHub as we’re already asking DockerHub for the latest image.

We start from the same docker-compose.yaml shown above.

updatecli.2.yaml

name: Example 2

sources:
 jenkins:
 kind: dockerdigest
 spec:
 image: "jenkins/jenkins"
 tag: "lts-jdk11"
 architecture: "amd64"

targets:
 imageTag:
 name: "jenkins/jenkins:lts-jdk11 docker digest"
 kind: yaml
 transformers:
 - addprefix: "jenkins/jenkins@sha256:"
 spec:
 file: "./docker-compose.yaml"
 key: "services.jenkins.image"

This time updatecli queries DockerHub to retrieve the digest for the docker image jenkins/jenkins:lts-jdk11. If it finds one, then we test that the value in the file 'docker-compose.yaml' for the key services.jenkins.image is correct, otherwise we change it.

Again, if the changes are acceptable then we apply them by using apply instead of diff.

updatecli pipeline diff --config updatecli.2.yaml
updatecli pipeline apply --config updatecli.2.yaml

Git/GitHub

Now that we have an easy way to update docker image versions, we are missing a way to save, review, and rollback those changes. For this, git is an excellent tool. Either we directly commit and push to a git repository, or we use the GitHub workflow by pushing to a temporary branch. If using GitHub we can also submit our changes via a pull request.

We keep the same docker-compose.yaml as before, and extend the manifest.

While the configuration remains similar to the earlier example, this time two new elements are introduced. We can define generic values in the file or read values from an environment variable like {{ requiredEnv "UPDATECLI_GITHUB_TOKEN" }}. The scms block defines the github repository configuration, and the actions block turns the pushed branch into a pull request.

updatecli.3.yaml

name: Example 3

sources:
 jenkins:
 name: "Get latest Jenkins version from maven repo"
 kind: "maven"
 transformers:
 # - addprefix: "v"
 - addsuffix: "-jdk11"
 spec:
 url: "repo.jenkins-ci.org"
 repository: "releases"
 groupid: "org.jenkins-ci.main"
 artifactid: "jenkins-war"

conditions:
 docker:
 name: "Docker Image Published on Registry"
 kind: "dockerimage"
 spec:
 image: "jenkins/jenkins"
 architecture: "amd64"

targets:
 imageTag:
 name: "Update docker-compose jenkins service"
 kind: "yaml"
 scmid: "updatecli/website"
 transformers:
 - addprefix: "jenkins/jenkins:"
 spec:
 file: "assets/code_example/docs/guides/docker-compose/docker-compose.yaml"
 key: "services.jenkins.image"

scms:
 updatecli/website:
 kind: "github"
 spec:
 user: "John"
 email: "info@updatecli.io"
 owner: "updatecli"
 repository: "website"
 token: '{{ requiredEnv "GITHUB_TOKEN" }}'
 username: '{{ requiredEnv "GITHUB_ACTOR" }}'
 branch: "master"

actions:
 default:
 kind: "github/pullrequest"
 scmid: "updatecli/website"
 spec:
 labels:
 - "dependencies"

And now you can use the same commands as before, with the credentials the manifest expects:

export UPDATECLI_GITHUB_TOKEN=<your_PAT>
export UPDATECLI_GITHUB_ACTOR=<your GH username>

updatecli pipeline diff --config updatecli.3.yaml
updatecli pipeline apply --config updatecli.3.yaml

Conclusion

With this scenario, we saw how to automatically update a docker-compose file using custom strategies with updatecli. Updatecli is a small single binary that is suitable for use in your favorite CI environment.

Guide: Npm

Tue, 07 Feb 2023 22:10:01 +0200

npm

Goal

The purpose of this pipeline is to track one npm dependency update, and then update both the package.json and package-lock.json in case of a version change.

Requirement

This guide requires:

updatecli
npm
IDE
GitHub Personal Access Token

updatecli

It can easily be installed using Homebrew by running the two following commands:

-> brew tap updatecli/updatecli
-> brew install updatecli

Additional installation instructions are available at www.updatecli.io/docs/prologue/installation.

IDE

For example, from VScode, typing [ctrl][space] should display a box with various suggestions

npm

npm is a package manager for the Javascript programming language.

GitHub PAT

The GitHub Personal Access Token is the token used to interact with the GitHub API Documentation

Pipeline

Description

Updatecli relies on one or more pipeline manifests to describe the expected update pipeline. In this guide, we use the four following components

A source of kind npm to monitor the npm dependency
A target of kind shell to run a command for updating both the 'package-lock.json' and 'package.json'
An scm of kind github, referenced by the target through scmid: default, so the update happens on a clone of the repository
An action of kind github/pullrequest, bound to the same scm, to submit the change

The project we update holds the two files npm keeps in sync:

package.json

{
 "name": "guide-npm",
 "version": "1.0.0",
 "description": "An Updatecli guide for npm project",
 "author": "Olivier Vernin",
 "dependencies": {
 "axios": "^1.20.0"
 }
}

package-lock.json

{
 "name": "guide-npm",
 "version": "1.0.0",
 "lockfileVersion": 3,
 "requires": true,
 "packages": {
 "": {
 "name": "guide-npm",
 "version": "1.0.0",
 "dependencies": {
 "axios": "^1.20.0"
 }
 },
 "node_modules/agent-base": {
 "version": "6.0.2",
 "resolved": "https://registry.npmjs.org/agent-base/-/agent-base-6.0.2.tgz",
 "integrity": "sha512-RZNwNclF7+MS/8bDg70amg32dyeZGZxiDuQmZxKLAlQjr3jGyLx+4Kkk58UO7D2QdgFIQCovuSuZESne6RG6XQ==",
 "license": "MIT",
 "dependencies": {
 "debug": "4"
 },
 "engines": {
 "node": ">= 6.0.0"
 }
 },
 "node_modules/asynckit": {
 "version": "0.4.0",
 "resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
 "integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==",
 "license": "MIT"
 },
 "node_modules/axios": {
 "version": "1.20.0",
 "resolved": "https://registry.npmjs.org/axios/-/axios-1.20.0.tgz",
 "integrity": "sha512-r8aOh8j9cGKpgQAqpzrUHnSIc6a59Y3Xf/cv8sy1DrHCkZHzQGEuoq1tARk6qSyDdtQGSDgpb9kFlruzPvrgwg==",
 "license": "MIT",
 "dependencies": {
 "follow-redirects": "^1.16.0",
 "form-data": "^4.0.6",
 "https-proxy-agent": "^5.0.1",
 "proxy-from-env": "^2.1.0"
 }
 },
 "node_modules/call-bind-apply-helpers": {
 "version": "1.0.2",
 "resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz",
 "integrity": "sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==",
 "license": "MIT",
 "dependencies": {
 "es-errors": "^1.3.0",
 "function-bind": "^1.1.2"
 },
 "engines": {
 "node": ">= 0.4"
 }
 },
 "node_modules/combined-stream": {
 "version": "1.0.8",
 "resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
 "integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==",
 "license": "MIT",
 "dependencies": {
 "delayed-stream": "~1.0.0"
 },
 "engines": {
 "node": ">= 0.8"
 }
 },
 "node_modules/debug": {
 "version": "4.4.3",
 "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
 "integrity": "sha512-RGwwWnwQvkVfavKVt22FGLw+xYSdzARwm0ru6DhTVA3umU5hZc28V3kO4stgYryrTlLpuvgI9GiijltAjNbcqA==",
 "license": "MIT",
 "dependencies": {
 "ms": "^2.1.3"
 },
 "engines": {
 "node": ">=6.0"
 },
 "peerDependenciesMeta": {
 "supports-color": {
 "optional": true
 }
 }
 },
 "node_modules/delayed-stream": {
 "version": "1.0.0",
 "resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
 "integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==",
 "license": "MIT",
 "engines": {
 "node": ">=0.4.0"
 }
 },
 "node_modules/dunder-proto": {
 "version": "1.0.1",
 "resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz",
 "integrity": "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==",
 "license": "MIT",
 "dependencies": {
 "call-bind-apply-helpers": "^1.0.1",
 "es-errors": "^1.3.0",
 "gopd": "^1.2.0"
 },
 "engines": {
 "node": ">= 0.4"
 }
 },
 "node_modules/es-define-property": {
 "version": "1.0.1",
 "resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",
 "integrity": "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==",
 "license": "MIT",
 "engines": {
 "node": ">= 0.4"
 }
 },
 "node_modules/es-errors": {
 "version": "1.3.0",
 "resolved": "https://registry.npmjs.org/es-errors/-/es-errors-1.3.0.tgz",
 "integrity": "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw==",
 "license": "MIT",
 "engines": {
 "node": ">= 0.4"
 }
 },
 "node_modules/es-object-atoms": {
 "version": "1.1.2",
 "resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.2.tgz",
 "integrity": "sha512-HWcBoN6NileqtSydK2FqHbS/LoDd2pqrnQHLyJzBj4kOp/ky2MWMN694xOfkK8/SnUsW2DH7EfyVlydKCsm1Zw==",
 "license": "MIT",
 "dependencies": {
 "es-errors": "^1.3.0"
 },
 "engines": {
 "node": ">= 0.4"
 }
 },
 "node_modules/es-set-tostringtag": {
 "version": "2.1.0",
 "resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz",
 "integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==",
 "license": "MIT",
 "dependencies": {
 "es-errors": "^1.3.0",
 "get-intrinsic": "^1.2.6",
 "has-tostringtag": "^1.0.2",
 "hasown": "^2.0.2"
 },
 "engines": {
 "node": ">= 0.4"
 }
 },
 "node_modules/follow-redirects": {
 "version": "1.16.0",
 "resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.16.0.tgz",
 "integrity": "sha512-y5rN/uOsadFT/JfYwhxRS5R7Qce+g3zG97+JrtFZlC9klX/W5hD7iiLzScI4nZqUS7DNUdhPgw4xI8W2LuXlUw==",
 "funding": [
 {
 "type": "individual",
 "url": "https://github.com/sponsors/RubenVerborgh"
 }
 ],
 "license": "MIT",
 "engines": {
 "node": ">=4.0"
 },
 "peerDependenciesMeta": {
 "debug": {
 "optional": true
 }
 }
 },
 "node_modules/form-data": {
 "version": "4.0.6",
 "resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.6.tgz",
 "integrity": "sha512-vKatAh4SlVfgbv+YtmhiRjhEMJsYpsG1Y2rMQtR+SVSbytsSD1YGzDIcrAJmdFec88u/+VoGmxnl+80gL1tRCQ==",
 "license": "MIT",
 "dependencies": {
 "asynckit": "^0.4.0",
 "combined-stream": "^1.0.8",
 "es-set-tostringtag": "^2.1.0",
 "hasown": "^2.0.4",
 "mime-types": "^2.1.35"
 },
 "engines": {
 "node": ">= 6"
 }
 },
 "node_modules/function-bind": {
 "version": "1.1.2",
 "resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
 "integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==",
 "license": "MIT",
 "funding": {
 "url": "https://github.com/sponsors/ljharb"
 }
 },
 "node_modules/get-intrinsic": {
 "version": "1.3.0",
 "resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.3.0.tgz",
 "integrity": "sha512-9fSjSaos/fRIVIp+xSJlE6lfwhES7LNtKaCBIamHsjr2na1BiABJPo0mOjjz8GJDURarmCPGqaiVg5mfjb98CQ==",
 "license": "MIT",
 "dependencies": {
 "call-bind-apply-helpers": "^1.0.2",
 "es-define-property": "^1.0.1",
 "es-errors": "^1.3.0",
 "es-object-atoms": "^1.1.1",
 "function-bind": "^1.1.2",
 "get-proto": "^1.0.1",
 "gopd": "^1.2.0",
 "has-symbols": "^1.1.0",
 "hasown": "^2.0.2",
 "math-intrinsics": "^1.1.0"
 },
 "engines": {
 "node": ">= 0.4"
 },
 "funding": {
 "url": "https://github.com/sponsors/ljharb"
 }
 },
 "node_modules/get-proto": {
 "version": "1.0.1",
 "resolved": "https://registry.npmjs.org/get-proto/-/get-proto-1.0.1.tgz",
 "integrity": "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==",
 "license": "MIT",
 "dependencies": {
 "dunder-proto": "^1.0.1",
 "es-object-atoms": "^1.0.0"
 },
 "engines": {
 "node": ">= 0.4"
 }
 },
 "node_modules/gopd": {
 "version": "1.2.0",
 "resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
 "integrity": "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==",
 "license": "MIT",
 "engines": {
 "node": ">= 0.4"
 },
 "funding": {
 "url": "https://github.com/sponsors/ljharb"
 }
 },
 "node_modules/has-symbols": {
 "version": "1.1.0",
 "resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz",
 "integrity": "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==",
 "license": "MIT",
 "engines": {
 "node": ">= 0.4"
 },
 "funding": {
 "url": "https://github.com/sponsors/ljharb"
 }
 },
 "node_modules/has-tostringtag": {
 "version": "1.0.2",
 "resolved": "https://registry.npmjs.org/has-tostringtag/-/has-tostringtag-1.0.2.tgz",
 "integrity": "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==",
 "license": "MIT",
 "dependencies": {
 "has-symbols": "^1.0.3"
 },
 "engines": {
 "node": ">= 0.4"
 },
 "funding": {
 "url": "https://github.com/sponsors/ljharb"
 }
 },
 "node_modules/hasown": {
 "version": "2.0.4",
 "resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.4.tgz",
 "integrity": "sha512-T2UbfbBEF32wiepXIsMlTW9+dDYC6wMh/t/vYA4tuOMKqWz/n3vr1NFSxQiyP+zk2mXsoMA/i/7qV6LKut1t1A==",
 "license": "MIT",
 "dependencies": {
 "function-bind": "^1.1.2"
 },
 "engines": {
 "node": ">= 0.4"
 }
 },
 "node_modules/https-proxy-agent": {
 "version": "5.0.1",
 "resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-5.0.1.tgz",
 "integrity": "sha512-dFcAjpTQFgoLMzC2VwU+C/CbS7uRL0lWmxDITmqm7C+7F0Odmj6s9l6alZc6AELXhrnggM2CeWSXHGOdX2YtwA==",
 "license": "MIT",
 "dependencies": {
 "agent-base": "6",
 "debug": "4"
 },
 "engines": {
 "node": ">= 6"
 }
 },
 "node_modules/math-intrinsics": {
 "version": "1.1.0",
 "resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz",
 "integrity": "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==",
 "license": "MIT",
 "engines": {
 "node": ">= 0.4"
 }
 },
 "node_modules/mime-db": {
 "version": "1.52.0",
 "resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
 "integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
 "license": "MIT",
 "engines": {
 "node": ">= 0.6"
 }
 },
 "node_modules/mime-types": {
 "version": "2.1.35",
 "resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
 "integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
 "license": "MIT",
 "dependencies": {
 "mime-db": "1.52.0"
 },
 "engines": {
 "node": ">= 0.6"
 }
 },
 "node_modules/ms": {
 "version": "2.1.3",
 "resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
 "integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
 "license": "MIT"
 },
 "node_modules/proxy-from-env": {
 "version": "2.1.0",
 "resolved": "https://registry.npmjs.org/proxy-from-env/-/proxy-from-env-2.1.0.tgz",
 "integrity": "sha512-cJ+oHTW1VAEa8cJslgmUZrc+sjRKgAKl3Zyse6+PV38hZe/V6Z14TbCuXcan9F9ghlz4QrFr2c92TNF82UkYHA==",
 "license": "MIT",
 "engines": {
 "node": ">=10"
 }
 }
 }
}

Manifest

updatecli.d/manifest.yaml

name: npm manifest guide

sources:
 npm:
 name: Get latest axios version
 kind: npm
 spec:
 name: axios
 versionfilter:
 kind: semver
 pattern: ^1.0.0

targets:
 npm:
 name: Ensure package.json and package-lock.json are up to date
 kind: shell

 # The default behavior for a target is to have one of its spec fields defaulting to the source resource output.
 #
 # More specifically for a resource of kind "shell", updatecli would append the value after the command such as
 # `npm install --package-lock-only --dry-run=$DRY_RUN axios@ 1.0.0`
 # Please note the space between axios@ and 1.0.0
 #
 # In this context, it doesn't work as the npm command requires axios@1.0.0 to be attached without a space
 #
 # Therefore we disable the default source output by using the parameter "disablesourceinput: true"
 # And then we use the Updatecli specific templating function {{ source "<sourceID>" }}
 disablesourceinput: true

 # scmid binds a target with a scm configuration.
 # Updatecli clones the git repository in a temporary location, then updates it
 # according to the current declarative pipeline.
 scmid: default

 # Spec contains the specific parameter for a specific kind of resource which in this case is the "shell" resource
 spec:

 # The DRY_RUN environment variable is set and used by Updatecli to identify:
 # 1. If we are in "dry-run" mode by running `updatecli pipeline diff --config manifest.yaml`
 # 2. If we are in "apply" mode by running `updatecli pipeline apply --config manifest.yaml`
 command: npm install --package-lock-only --dry-run=$DRY_RUN axios@{{ source "npm" }}

 # By default, shell command only have access to the DRY_RUN environment variable.
 # But in this situation, we would also need the PATH value to identify where the npm command is located.
 #
 # As the PATH environment value is not defined it inherits its value from the Updatecli process.
 environments:
 - name: PATH

 # changedif defines how Updatecli should interpret a failure or a change.
 # In this case we use the "file/checksum" changedif kind.
 # A filechecksum change would report a change.
 changedif:
 kind: file/checksum
 spec:
 files:
 - package-lock.json
 - package.json

 # npm expects to run the command from the directory containing the package.json
 # workdir allows to specify the path where to run the command.
 # If the resource has a scmid, then the workdir relative path is from the cloned scm directory
 # in the temporary directory such as /tmp/updatecli on POSIX system.
 workdir: "assets/code_example/docs/guides/npm/"

# scms defines the git repository configuration used by the target and the action.
# Updatecli will clone that repository in the OS temporary location such as /tmp on Darwin or Linux system.
scms:
 default:
 kind: github
 spec:
 owner: updatecli
 repository: website
 # To handle secrets, Updatecli can either use environment variable or sops secrets.
 # When executed from GitHub Action, using GITHUB_TOKEN and GITHUB_ACTOR is quite convenient.
 token: '{{ requiredEnv "GITHUB_TOKEN" }}'
 username: '{{ requiredEnv "GITHUB_ACTOR" }}'
 branch: master

# actions are actions to execute if the state of a binded target is changed.
# Ex: opening a pullrequest.
actions:
 default:
 # The github/pullrequest means Updatecli will create a temporary branch containing all the changes
 # generated by the different targets using the same scmid.
 #
 # Then if there is a change, it will opens a pull request targeting the scm branch referenced by scmid.
 kind: github/pullrequest
 scmid: default
 spec:
 labels:
 - dependencies
 merge:
 strategy: auto
 usetitleforautomerge: true
 mergemethod: squash

Then you can run:

export UPDATECLI_GITHUB_TOKEN=<your_PAT>
export UPDATECLI_GITHUB_ACTOR=<your GH username>
updatecli pipeline diff --config updatecli.d/manifest.yaml

Going further

Once you’re happy with the Update pipeline, you can use your CI environment to run it regularly. You’ll need updatecli installed in your CI environment, and the right credentials. More information in the corresponding GitHub Action documentation

This guide demonstrated a simple update pipeline, but real-world situations usually require either updating all dependencies specified in a package.json, or updating additional resources such as documentation, CI environment, etc.

For updating all dependencies from a package.json at once, you do not need to write a manifest at all. The npm crawler is enabled by default, so running updatecli pipeline diff from a repository containing a package.json discovers the dependencies and generates the pipeline for you. It also refreshes package-lock.json or yarn.lock when the matching package manager is available. The "npm autodiscovery" page covers what it detects and how to tune it.

In the situation of a more tailored update scenario, you may need to extend the current pipeline using additional resources such as:

Dockerfile Adding a target of kind dockerfile, updatecli could update the npm dependency used within a Dockerfile as part of the same operation.

Yaml Adding a target of kind yaml, updatecli could also update the npm dependency used in any yaml as part of the same operation.

Feel free to chat on the Updatecli Matrix channel.

Guide: helm chart

Sun, 27 Feb 2022 22:10:01 +0200

Helm Chart

Helm is a popular package manager for deploying applications on Kubernetes. But, those packages can be outdated quickly. In this guide we explore how we can automatically update Helm charts based on third-party dependencies.

Requirement

To use this guide, we need the following elements:

Updatecli
A Helm chart to update
A GitHub Personal Access Token

Updatecli

It can easily be installed using Homebrew by running the two following commands:

-> brew tap updatecli/updatecli
-> brew install updatecli

Additional installation instructions are available at www.updatecli.io/docs/prologue/installation.

IDE

For example, from VScode, typing [ctrl][space] should display a box with various suggestions

Helm chart to update

Within this example, we’ll use a Helm chart from the Jenkins infrastructure project located on github.com/jenkins-infra/helm-charts. We want to check that the Helm chart is always using the latest chart application which, in this case, is the jenkins-wiki-exporter.

GitHub Personal Access Token

As we’re going to interact with GitHub repository and GitHub packages, we need a personal access token. More information on the GitHub documentation creating-a-personal-access-token

Pipeline

Manifest

Updatecli expects one or more pipeline manifests to know what to do. We use the command updatecli pipeline diff --config updatecli.yaml to load the manifest. The diff allows us to run updatecli in dry-mode, identifying what updatecli would change. Then we execute updatecli pipeline apply --config updatecli.yaml to apply our updatecli pipeline.

As you’ll see in our updatecli manifest, we define:

One "source" of kind "githubrelease" which describes where to retrieve the latest version of our application, jenkins-wiki-exporter, published on GitHub Release.
Two "conditions" which need to be satisfied before updating our Helm chart. The first one, "DockerImagePublished" of kind "dockerimage", ensures that a Docker image "jenkinsciinfra/jenkins-wiki-exporter" exists on DockerHub where the tag is the same as the source output, the latest jenkins-wiki-exporter version. The second one, "DockerContainerUsed" of kind "yaml", ensures that our Helm chart uses the right container name, "jenkinsciinfra/jenkins-wiki-exporter".
A "target" of kind "helmchart", with a chart name "charts/jenkins-wiki-exporter" and a file "values.yaml" where the key "image.tag" is set to the latest jenkins-wiki-exporter version. If it’s not the case then bump the patch version of the chart, as requested by versionincrement: "patch".
An "scms" entry of kind "github", that describes which GitHub repository stores the Helm chart "code".
An "actions" entry of kind "github/pullrequest", that describes pull request parameters such as the labels assigned to the pull request created by updatecli, and the auto-merge strategy.

updatecli.yaml

name: "Bump `jenkins-wiki-exporter` docker image and helm chart versions"

sources:
 jenkins-wiki-exporter:
 kind: "githubrelease"
 name: "Get jenkins-infra/jenkins-wiki-exporter latest version"
 spec:
 owner: "jenkins-infra"
 repository: "jenkins-wiki-exporter"
 token: '{{ requiredEnv "GITHUB_TOKEN" }}'
 username: '{{ requiredEnv "GITHUB_ACTOR" }}'
 # Added to take care of "v" prefix in version until the chart is updated to use the semver versioning (which doesn't have the "v" prefix)
 versionfilter:
 kind: "latest"

conditions:
 DockerImagePublished:
 name: "Test jenkinsciinfra/jenkins-wiki-exporter:<latest_version> docker image tag"
 kind: "dockerimage"
 sourceid: "jenkins-wiki-exporter"
 spec:
 image: "jenkinsciinfra/jenkins-wiki-exporter"
 architecture: "amd64"
 # tag retrieved from source output "jenkinswikiexporter"
 DockerContainerUsed:
 name: "Test jenkinsciinfra/jenkins-wiki-exporter:<latest_version> docker image tag"
 kind: "yaml"
 disablesourceinput: true
 scmid: "helm-charts"
 spec:
 file: "charts/jenkins-wiki-exporter/values.yaml"
 value: "jenkinsciinfra/jenkins-wiki-exporter"
 key: "image.repository"
targets:
 jenkins-wiki-exporter:
 name: "Update jenkins-wiki-exporter helm chart"
 kind: "helmchart"
 spec:
 name: "charts/jenkins-wiki-exporter"
 file: "values.yaml"
 key: "image.tag"
 versionincrement: "patch"
 scmid: "helm-charts"
 sourceid: "jenkins-wiki-exporter"

scms:
 helm-charts:
 kind: "github"
 spec:
 branch: "main"
 user: "updatecli-robot"
 email: "bot@updatecli.io"
 owner: "jenkins-infra"
 repository: "helm-charts"
 token: '{{ requiredEnv "GITHUB_TOKEN" }}'
 username: '{{ requiredEnv "GITHUB_ACTOR" }}'

actions:
 default:
 kind: "github/pullrequest"
 scmid: "helm-charts"
 spec:
 merge:
 strategy: auto
 labels:
 - "dependencies"
 - "jenkins-wiki-exporter"

Now we run our updatecli pipeline using the following commands:

export UPDATECLI_GITHUB_TOKEN=xxx
export UPDATECLI_GITHUB_ACTOR=yyy
updatecli pipeline diff --config updatecli.yaml

Warning

Using environment variables to store credentials is convenient for running updatecli from a CI environment. Another option is to use sops but in either case, you should not leave your credentials unprotected.

Spoiler Alert: Command output

+++++++++++
+ PREPARE +
+++++++++++

Loading Pipeline "assets/code_example/doc/guides/helm-chart/updatecli.yaml"
Repository retrieved: 1


++++++++++++
+ PIPELINE +
++++++++++++



#####################################################################
# BUMP `JENKINS-WIKI-EXPORTER` DOCKER IMAGE AND HELM CHART VERSIONS #
#####################################################################


SOURCES
=======

jenkins-wiki-exporter
---------------------
Searching for version matching pattern "latest"
✔ Github Release version "v1.12.1" found matching pattern "latest"


CHANGELOG:
----------

Release published on the 2021-10-02 18:36:27 +0000 UTC at the url https://github.com/jenkins-infra/jenkins-wiki-exporter/releases/tag/v1.12.1

<!-- Optional: add a release summary here -->
* Dont allow non pull requests in the project from breaking the report (#303) @halkeye

## 📦 Dependency updates

* Bump node from 16.7.0 to 16.8.0 (#288) @dependabot
* Bump jest from 27.0.6 to 27.1.0 (#287) @dependabot



CONDITIONS:
===========

DockerContainerUsed
-------------------
✔ Key "image.repository", in YAML file "/tmp/updatecli/jenkins-infra/helm-charts/charts/jenkins-wiki-exporter/values.yaml", is correctly set to "jenkinsciinfra/jenkins-wiki-exporter"

DockerImagePublished
--------------------
^[[B✔ The Docker image registry-1.docker.io/jenkinsciinfra/jenkins-wiki-exporter:v1.12.1 exists and is available.


TARGETS
========

jenkins-wiki-exporter
---------------------

**Dry Run enabled**

✔ Key 'image.tag', from file '/tmp/updatecli/jenkins-infra/helm-charts/charts/jenkins-wiki-exporter/values.yaml', already set to v1.12.1, nothing else need to be done


PULL REQUESTS
=============


=============================

REPORTS:


✔ UPDATECLI.YAML:
	Sources:
		✔ [jenkins-wiki-exporter] Get jenkins-infra/jenkins-wiki-exporter latest version (kind: githubRelease)
	Condition:
		✔ [DockerContainerUsed] Test jenkinsciinfra/jenkins-wiki-exporter:<latest_version> docker image tag (kind: yaml)
		✔ [DockerImagePublished] Test jenkinsciinfra/jenkins-wiki-exporter:<latest_version> docker image tag (kind: dockerimage)
	Target:
		✔ [jenkins-wiki-exporter] Update jenkins-wiki-exporter helm chart (kind: helmchart)



Run Summary
===========
Pipeline(s) run:
 * Changed:	0
 * Failed:	0
 * Skipped:	0
 * Succeeded:	1
 * Total:	1

Go Further

Updatecli is used in more scenarios and with many more to come. Please reach out if you have questions. Meanwhile, here are additional resources that you can use to go further.

Resource

To extend this pipeline, you can find more customization on the updatecli documentation website.

In this pipeline we used the following resources:

YAML - the yaml resource used by the second condition.
GitHub Release - the githubrelease source.
Helm Chart - the helmchart target.
GitHub - the github scm.
GitHub Pull Request - the github/pullrequest action.
Docker Image - the dockerimage resource used by the first condition.

Additional Workflows

Updatecli shines and saves us time in many other situations such as:

Docker Compose - keeping a docker-compose.yaml on the latest image tag or digest.
Dockerfile - how the Jenkins project keeps its Dockerfiles on the latest JDK.
npm - tracking an npm dependency and refreshing the lock file.

Contributing

As a community-oriented project, all contributions are greatly appreciated!

Here is a non-exhaustive list of possible contributions:

⭐️ this updatecli/updatecli repository.
Propose a new feature request.
Highlight an existing feature request with :thumbsup:.
Contribute to any repository in the updatecli organization
Share the love

More information is available at CONTRIBUTING

Guide: dockerfile

Fri, 20 Sep 2024 22:10:01 +0200

Dockerfile

Goal

This guide analyzes how the Jenkins project automates its Dockerfile file updates.

It’s one thing to publish a Docker image, but it’s another one to maintain it over time.