Self-Serve Release Jobs
=======================
-Self-serve release jobs allow project committers to direct Jenkins to
-promote a jar file, container image or Python package from a staging
-area to a release area.
-
-To use the self-release process, create a releases/ or .releases/
-directory at the root of the project repository, add one release yaml
-file to it, and submit a change set with that release yaml file. The
-required contents of the release yaml file are different for each type
-of release, see the schemas and examples shown below. The version in
-the release yaml file must be a valid Semantic Versioning (SemVer)
-string, matching either the pattern "v#.#.#" or "#.#.#" where "#" is
-one or more digits. Upon merge of the change, Jenkins will sign the
-reference extrapolated by log_dir and promote the artifact.
+Self-serve release jobs allow project committers to promote a jar
+file, container image, Python package or PackageCloud artifact from a
+staging area to a release area. A release yaml file controls the
+process, and Jenkins promotes the artifact when a project committer
+merges the release yaml file in Gerrit.
+
+To use the self-release process, create a releases/ or .releases/ directory at
+the root of the project repository, add one release yaml file to it, and submit
+a change set with that release yaml file. The required contents of the release
+yaml file are different for each release, see the schemas and examples shown
+below. The version string in the release yaml file should be a valid Semantic
+Versioning (SemVer) string, matching the pattern "#.#.#" where "#" is one or
+more digits. A version string matching the pattern "v#.#.#" is also accepted.
+Upon merge of the change, a Jenkins job promotes the artifact and pushes a
+gpg-signed tag to the repository.
.. note::
The release file regex is: (releases\/.*\.yaml|\.releases\/.*\.yaml).
In words, the directory name can be ".releases" or "releases"; the file
- name can be anything with suffix ".yaml".
+ name can be anything with suffix ".yaml". Some release jobs require
+ a specific prefix on the file, as described below.
The build node for all release jobs must be CentOS, which supports the
-sigul client for accessing a signing server. The build node for
-container release jobs must have Docker installed.
+sigul client for accessing a signing server to sign a tag. The build
+node for container release jobs must have Docker installed.
A Jenkins admin user can also trigger a release job via the "Build
with parameters" action, removing the need to create and merge a
USE_RELEASE_FILE = false
DRY_RUN = false
+It's recommended to use Semantic Versions (SemVer) for releases. Refer to
+https://semver.org for more details on SemVer. For projects that do not
+follow SemVer can use a build parameter (OVERRIDE_SEMVER_REGEX) with the
+release job. This build param overrides the default SemVer regex.
+
+
Maven Release Files
-------------------
:Required Parameters:
:distribution_type: Must be "maven".
- :log_dir: The suffix of the logs URL reported on completion by the
- Jenkins stage job that created and pushed the artifact
+ :log_dir: The suffix of the logs URL reported on successful completion
+ by the Jenkins stage job that created and pushed the artifact
to the staging repository. For example, use value
"example-project-maven-stage-master/17" for the logs URL
https://logs.lf-project.org/production/vex-sjc-lfp-jenkins-prod-1/example-project-maven-stage-master/17
:project: The name of the project.
:version: The semantic version string used for the artifact.
-The JSON schema for a maven release file appears below.
+:Optional Parameters:
-.. code-block:: none
+ :git_tag: The tag string to sign and push to the Git repository.
+ (default: the semantic version string)
+ :tag_release: Tag Gerrit Repo during the release process.
+ (default: true)
- ---
- $schema: "http://json-schema.org/schema#"
- $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-schema.yaml"
-
- required:
- - "distribution_type"
- - "log_dir"
- - "project"
- - "version"
-
- properties:
- distribution_type:
- type: "string"
- log_dir:
- type: "string"
- project:
- type: "string"
- version:
- type: "string"
+The JSON schema for a maven release file appears below.
+
+.. literalinclude:: ../../schema/release-schema.yaml
+ :language: yaml
Container Release Files
Docker images in the container-pull registry to promote to the
container-push registry.
-The JSON schema for a container release file appears below.
+:Optional Parameters:
-.. code-block:: none
+ :git_tag: The tag string to sign and push to the Git repository.
+ (default: the semantic version string)
+ :tag_release: Tag Gerrit Repo during the release process.
+ (default: true)
- ---
- $schema: "http://json-schema.org/schema#"
- $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-container-schema.yaml"
-
- required:
- - "containers"
- - "distribution_type"
- - "project"
- - "container_release_tag"
- - "ref"
-
- properties:
- containers:
- type: "array"
- properties:
- name:
- type: "string"
- version:
- type: "string"
- additionalProperties: false
- distribution_type:
- type: "string"
- project:
- type: "string"
- container_release_tag:
- type: "string"
- container_pull_registry"
- type: "string"
- container_push_registry"
- type: "string"
- ref:
- type: "string"
+The JSON schema for a container release file appears below.
+
+.. literalinclude:: ../../schema/release-container-schema.yaml
+ :language: yaml
PyPI Release Files
:Required Parameters:
- :log_dir: The suffix of the logs URL reported on completion by the
- Jenkins merge job that created and pushed the distribution files
- to the staging repository. For example, use value
+ :log_dir: The suffix of the logs URL reported on successful completion
+ by the Jenkins merge job that created and pushed the distribution
+ files to the staging repository. For example, use value
"example-project-pypi-merge-master/17" for the logs URL
https://logs.lf-project.org/production/vex-sjc-lfp-jenkins-prod-1/example-project-pypi-merge-master/17
:pypi_project: The PyPI project name at the staging and
:version: The semantic version string used for the package in the
setup.py file.
-The JSON schema for a PyPI release file appears below.
+:Optional Parameters:
-.. code-block:: none
+ :git_tag: The tag string to sign and push to the Git repository.
+ (default: the semantic version string)
+ :tag_release: Tag Gerrit Repo during the release process.
+ (default: true)
- ---
- $schema: "http://json-schema.org/schema#"
- $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-pypi-schema.yaml"
-
- required:
- - "log_dir"
- - "pypi_project"
- - "python_version"
- - "version"
-
- properties:
- log_dir:
- type: "string"
- pypi_project:
- type: "string"
- python_version:
- type: "string"
- version:
- type: "string"
+The JSON schema for a PyPI release file appears below.
+
+.. literalinclude:: ../../schema/release-pypi-schema.yaml
+ :language: yaml
PackageCloud Release Files
$ cat releases/packagecloud-1.6-tree.yaml
---
- package_name:
- - name: tree-1.6.0-10.el7.x86_64.rpm
- - name: test.rpm
+ package_name: tree
+ packages:
+ - name: tree_1.6.0_amd64.deb
+ - name: tree-dev_1.6.0_amd64.deb
+ - name: tree-devel-1.6.0-1.x86_64.rpm
+ - name: tree-1.6.0-1.x86_64.rpm
+ ref: 5555cd2dd345fbeec0d3e2162e00835852342cda
+ log_dir: example-project-packagecloud-merge/21
+ version: 1.6.0
The following parameters must appear in the PackageCloud release yaml file.
These are not part of the Jenkins job definition to allow independent
:Required Parameters:
- :package_name: A list of names that specify the packages to promote.
- (Found in jenkins console log when using gem to push package eg.
+ :package_name: Name of the release package.
+ :packages: A list of names that specify the packages to promote.
+ Found in jenkins console log when using gem to push package eg.
"Pushing /path/of/package/name-of-package.rpm... success!"
OR using rest api call to query packagecloud.io repo
"curl https://packagecloud.io/api/v1/repos/test_user/test_repo/search?q=
| yq -r .[].filename"
+ :ref: The git commit reference (SHA-1 code) to tag with the version string.
+ :log_dir: The suffix of the logs URL reported on successful completion
+ by the Jenkins merge job that created and pushed the distribution
+ files to the staging repository. For example, use value
+ "example-project-packagecloud-merge-/21" for the logs URL
+ https://logs.lf-project.org/production/vex-sjc-lfp-jenkins-prod-1/example-project-packagecloud-merge/21
+ :version: The semantic version string used for the package.
+
+:Optional Parameters:
+
+ :git_tag: The tag string to sign and push to the Git repository.
+ (default: the semantic version string)
+ :tag_release: Tag Gerrit Repo during the release process.
+ (default: true)
The JSON schema for a PackageCloud release file appears below.
-.. code-block:: none
+.. literalinclude:: ../../schema/release-packagecloud-schema.yaml
+ :language: yaml
- ---
- $schema: "http://json-schema.org/schema#"
- $id: "https://github.com/lfit/releng-global-jjb/blob/master/packagecloud-release-schema"
- required:
- - "package_name"
+Job Groups
+==========
+
+Below is a list of Release job groups:
+
+.. literalinclude:: ../../jjb/lf-release-job-groups.yaml
+ :language: yaml
- properties:
- package_name:
- type: "array"
- properties:
- name:
- type: "string"
Jenkins Jobs
------------
This template supports Maven and Container release jobs.
-:Template Name: {project-name}-release-merge
+This template uses a git commit choosing strategy that builds the merged
+commit with the release yaml file, not the tip of the target branch, so
+projects can repeat the release action in case of merge job failure.
+
+:Template Name: {project-name}-release-merge-{stream}
:Comment Trigger: remerge
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
:build-timeout: Timeout in minutes before aborting build. (default: 15)
+ :stream: Keyword that represents a release code-name.
+ Often the same as the branch. (default: master)
:gerrit_merge_triggers: Override Gerrit Triggers.
:gerrit_trigger_file_paths: Override file paths filter which checks which
- file modifications will trigger a build.
- **default**::
-
- - compare-type: REG_EXP
- pattern: '(releases\/.*\.yaml|\.releases\/.*\.yaml)'
+ file modifications will trigger a build. The default pattern is the
+ regular expression ``(releases\/.*\.yaml|\.releases\/.*\.yaml)``
Release Verify
This template supports Maven and Container release jobs.
-:Template Name: {project-name}-release-verify
+:Template Name: {project-name}-release-verify-{stream}
:Comment Trigger: recheck|reverify
:build-timeout: Timeout in minutes before aborting build. (default: 15)
:gerrit-skip-vote: Skip voting for this job. (default: false)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+ :stream: Keyword that represents a release code-name.
+ Often the same as the branch. (default: master)
:gerrit_verify_triggers: Override Gerrit Triggers.
:gerrit_trigger_file_paths: Override file paths filter which checks which
- file modifications will trigger a build.
- **default**::
-
- - compare-type: REG_EXP
- pattern: '(releases\/.*\.yaml|\.releases\/.*\.yaml)'
+ file modifications will trigger a build. The default pattern is the
+ regular expression ``(releases\/.*\.yaml|\.releases\/.*\.yaml)``
PyPI Release Merge
signs the tag and pushes the tag to the git server. The release merge
template accepts neither a branch nor a stream parameter.
+These templates use a git commit choosing strategy that builds the merged
+commit with the release yaml file, not the tip of the target branch, so
+projects can repeat the release action in case of merge job failure.
+
:Template Names:
- {project-name}-pypi-release-merge
should be the repository pypy.org. (default: pypi)
:use-release-file: Whether to use the release file. (default: true)
- :gerrit_trigger_file_paths: Override file paths filter which checks which
- file modifications will trigger a build.
- **default**::
-
- - compare-type: REG_EXP
- pattern: '(releases\/pypi.*\.yaml|\.releases\/pypi.*\.yaml)'
+ :gerrit_release_trigger_file_paths: Override file paths filter which checks
+ which file modifications will trigger a build. The default pattern is the
+ regular expression ``(releases\/pypi.*\.yaml|\.releases\/pypi.*\.yaml)``
PyPI Release Verify
~~~~~~~~~~~~~~~~~~~
should be the repository pypy.org (default: pypi)
:use-release-file: Whether to use the release file. (default: true)
- :gerrit_trigger_file_paths: Override file paths filter which checks which
- file modifications will trigger a build.
- **default**::
-
- - compare-type: REG_EXP
- pattern: '(releases\/pypi.*\.yaml|\.releases\/pypi.*\.yaml)'
+ :gerrit_release_trigger_file_paths: Override file paths filter which checks
+ which file modifications will trigger a build. The default pattern is the
+ regular expression ``(releases\/pypi.*\.yaml|\.releases\/pypi.*\.yaml)``
PackageCloud Release Verify
~~~~~~~~~~~~~~~~~~~~~~~~~~~
-This template supports PackageCloud release jobs.
+This template supports PackageCloud release jobs. Checks that the specified
+packages are present in the staging repository and absent from the release
+repository. The file path trigger uses the regular expression
+``(releases\/packagecloud.*\.yaml|\.releases\/packagecloud.*\.yaml)``
:Template Name: {project-name}-packagecloud-release-verify
:gerrit-skip-vote: Skip voting for this job. (default: false)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
- :gerrit_verify_triggers: Override Gerrit Triggers.
- :gerrit_trigger_file_paths: Override file paths filter which checks which
- file modifications will trigger a build.
- **default**::
-
- - compare-type: REG_EXP
- pattern: '(releases\/packagecloud.*\.yaml|\.releases\/packagecloud.*\.yaml)'
-
-
PackageCloud Release Merge
~~~~~~~~~~~~~~~~~~~~~~~~~~
-This template supports PackageCloud release jobs.
+This template supports PackageCloud release jobs. Promotes the specified
+packages from the staging repository to the release repository.
+The file path trigger uses the regular expression
+``(releases\/packagecloud.*\.yaml|\.releases\/packagecloud.*\.yaml)``
+
+This template uses a git commit choosing strategy that builds the merged
+commit with the release yaml file, not the tip of the target branch, so
+projects can repeat the release action in case of merge job failure.
-:template name: {project-name}-packagecloud-release-merge
+:Template Name: {project-name}-packagecloud-release-merge
-:comment trigger: remerge
+:Comment Trigger: remerge
-:required parameters:
+:Required Parameters:
:build-node: the node to run build on.
:jenkins-ssh-release-credential: credential to use for ssh. (generally set
:project: git repository name
:project-name: jenkins job name prefix
-:optional parameters:
+:Optional Parameters:
:build-days-to-keep: days to keep build logs in jenkins. (default: 7)
:build-timeout: timeout in minutes before aborting build. (default: 15)
- :gerrit_merge_triggers: override gerrit triggers.
- :gerrit_trigger_file_paths: override file paths filter which checks which
- file modifications will trigger a build.
- **default**::
-
- - compare-type: reg_exp
- pattern: '(releases\/packagecloud.*\.yaml|\.releases\/packagecloud.*\.yaml)'
-
-
Setup for LFID, Nexus, Jenkins and Gerrit
-----------------------------------------