Self-Serve Release Jobs
=======================
-Self-serve release jobs allow project committers to direct Jenkins to
-promote a jar file, container image or Python package from a staging
-area to a release area.
-
-To use the self-release process, create a releases/ or .releases/
-directory at the root of the project repository, add one release yaml
-file to it, and submit a change set with that release yaml file. The
-required contents of the release yaml file are different for each type
-of release, see the schemas and examples shown below. The version in
-the release yaml file must be a valid Semantic Versioning (SemVer)
-string, matching either the pattern "v#.#.#" or "#.#.#" where "#" is
-one or more digits. Upon merge of the change, Jenkins will sign the
-reference extrapolated by log_dir and promote the artifact.
+Self-serve release jobs allow project committers to promote a jar
+file, container image, Python package or PackageCloud artifact from a
+staging area to a release area. A release yaml file controls the
+process, and Jenkins promotes the artifact when a project committer
+merges the release yaml file in Gerrit.
+
+To use the self-release process, create a releases/ or .releases/ directory at
+the root of the project repository, add one release yaml file to it, and submit
+a change set with that release yaml file. The required contents of the release
+yaml file are different for each release, see the schemas and examples shown
+below. The version string in the release yaml file should be a valid Semantic
+Versioning (SemVer) string, matching the pattern "#.#.#" where "#" is one or
+more digits. A version string matching the pattern "v#.#.#" is also accepted.
+Upon merge of the change, a Jenkins job promotes the artifact and pushes a
+gpg-signed tag to the repository.
.. note::
The release file regex is: (releases\/.*\.yaml|\.releases\/.*\.yaml).
In words, the directory name can be ".releases" or "releases"; the file
- name can be anything with suffix ".yaml".
+ name can be anything with suffix ".yaml". Some release jobs require
+ a specific prefix on the file, as described below.
The build node for all release jobs must be CentOS, which supports the
-sigul client for accessing a signing server. The build node for
-container release jobs must have Docker installed.
+sigul client for accessing a signing server to sign a tag. The build
+node for container release jobs must have Docker installed.
A Jenkins admin user can also trigger a release job via the "Build
with parameters" action, removing the need to create and merge a
USE_RELEASE_FILE = false
DRY_RUN = false
+It's recommended to use Semantic Versions (SemVer) for releases. Refer to
+https://semver.org for more details on SemVer. For projects that do not
+follow SemVer can use a build parameter (OVERRIDE_SEMVER_REGEX) with the
+release job. This build param overrides the default SemVer regex.
+
+
Maven Release Files
-------------------
:Required Parameters:
:distribution_type: Must be "maven".
- :log_dir: The suffix of the logs URL reported on completion by the
- Jenkins stage job that created and pushed the artifact
+ :log_dir: The suffix of the logs URL reported on successful completion
+ by the Jenkins stage job that created and pushed the artifact
to the staging repository. For example, use value
"example-project-maven-stage-master/17" for the logs URL
https://logs.lf-project.org/production/vex-sjc-lfp-jenkins-prod-1/example-project-maven-stage-master/17
:project: The name of the project.
:version: The semantic version string used for the artifact.
-The JSON schema for a maven release file appears below.
+:Optional Parameters:
-.. code-block:: none
+ :git_tag: The tag string to sign and push to the Git repository.
+ (default: the semantic version string)
+ :tag_release: Tag Gerrit Repo during the release process.
+ (default: true)
- ---
- $schema: "http://json-schema.org/schema#"
- $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-schema.yaml"
-
- required:
- - "distribution_type"
- - "log_dir"
- - "project"
- - "version"
-
- properties:
- distribution_type:
- type: "string"
- log_dir:
- type: "string"
- project:
- type: "string"
- version:
- type: "string"
+The JSON schema for a maven release file appears below.
+
+.. literalinclude:: ../../schema/release-schema.yaml
+ :language: yaml
Container Release Files
Docker images in the container-pull registry to promote to the
container-push registry.
-The JSON schema for a container release file appears below.
+:Optional Parameters:
-.. code-block:: none
+ :git_tag: The tag string to sign and push to the Git repository.
+ (default: the semantic version string)
+ :tag_release: Tag Gerrit Repo during the release process.
+ (default: true)
- ---
- $schema: "http://json-schema.org/schema#"
- $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-container-schema.yaml"
-
- required:
- - "containers"
- - "distribution_type"
- - "project"
- - "container_release_tag"
- - "ref"
-
- properties:
- containers:
- type: "array"
- properties:
- name:
- type: "string"
- version:
- type: "string"
- additionalProperties: false
- distribution_type:
- type: "string"
- project:
- type: "string"
- container_release_tag:
- type: "string"
- container_pull_registry"
- type: "string"
- container_push_registry"
- type: "string"
- ref:
- type: "string"
+The JSON schema for a container release file appears below.
+
+.. literalinclude:: ../../schema/release-container-schema.yaml
+ :language: yaml
PyPI Release Files
------------------
-An example of a PyPI release file appears below.
+An example of a PyPI release file appears below. Name of the release file must
+start with "pypi". For example releases/pypi-1.0.0-mypackage.yaml
.. code-block:: none
- $ cat releases/pypi-release.yaml
+ $ cat releases/pypi-1.0.0-mypackage.yaml
---
- distribution_type: pypi
pypi_project: mypackage
- python_version: 3.4
+ python_version: '3.4'
version: 1.0.0
+ log_dir: example-project-pypi-merge-master/17
The following parameters must appear in the PyPI release yaml file.
:Required Parameters:
- :distribution_type: Must be "pypi".
- :log_dir: The suffix of the logs URL reported on completion by the
- Jenkins merge job that created and pushed the distribution files
- to the staging repository. For example, use value
+ :log_dir: The suffix of the logs URL reported on successful completion
+ by the Jenkins merge job that created and pushed the distribution
+ files to the staging repository. For example, use value
"example-project-pypi-merge-master/17" for the logs URL
https://logs.lf-project.org/production/vex-sjc-lfp-jenkins-prod-1/example-project-pypi-merge-master/17
:pypi_project: The PyPI project name at the staging and
release repositories, for example "mypackage".
:python_version: The Python interpreter version to use for pip
- "Requires-Python" compatibility checks, for example "3" or "3.7.0".
+ "Requires-Python" compatibility checks, for example '3', '3.7' or 3.7.4.
+ Put valid decimal values such as 3 or 3.7 in quotes to pass schema validation.
:version: The semantic version string used for the package in the
setup.py file.
+:Optional Parameters:
+
+ :git_tag: The tag string to sign and push to the Git repository.
+ (default: the semantic version string)
+ :tag_release: Tag Gerrit Repo during the release process.
+ (default: true)
+
The JSON schema for a PyPI release file appears below.
+.. literalinclude:: ../../schema/release-pypi-schema.yaml
+ :language: yaml
+
+
+PackageCloud Release Files
+--------------------------
+
+An example of a PackageCloud release file appears below. Name of release file
+must start with "packagecloud". For example releases/packagecloud-1.6-tree.yaml
+
.. code-block:: none
+ $ cat releases/packagecloud-1.6-tree.yaml
---
- $schema: "http://json-schema.org/schema#"
- $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-pypi-schema.yaml"
-
- required:
- - "distribution_type"
- - "log_dir"
- - "pypi_project"
- - "python_version"
- - "version"
-
- properties:
- distribution_type:
- type: "string"
- log_dir:
- type: "string"
- pypi_project:
- type: "string"
- python_version:
- type: "string"
- version:
- type: "string"
+ package_name: tree
+ packages:
+ - name: tree_1.6.0_amd64.deb
+ - name: tree-dev_1.6.0_amd64.deb
+ - name: tree-devel-1.6.0-1.x86_64.rpm
+ - name: tree-1.6.0-1.x86_64.rpm
+ ref: 5555cd2dd345fbeec0d3e2162e00835852342cda
+ log_dir: example-project-packagecloud-merge/21
+ version: 1.6.0
+
+The following parameters must appear in the PackageCloud release yaml file.
+These are not part of the Jenkins job definition to allow independent
+self-release of a package maintained in a git repository with other
+packages.
+
+:Required Parameters:
+
+ :package_name: Name of the release package.
+ :packages: A list of names that specify the packages to promote.
+ Found in jenkins console log when using gem to push package eg.
+ "Pushing /path/of/package/name-of-package.rpm... success!"
+ OR using rest api call to query packagecloud.io repo
+ "curl https://packagecloud.io/api/v1/repos/test_user/test_repo/search?q=
+ | yq -r .[].filename"
+ :ref: The git commit reference (SHA-1 code) to tag with the version string.
+ :log_dir: The suffix of the logs URL reported on successful completion
+ by the Jenkins merge job that created and pushed the distribution
+ files to the staging repository. For example, use value
+ "example-project-packagecloud-merge-/21" for the logs URL
+ https://logs.lf-project.org/production/vex-sjc-lfp-jenkins-prod-1/example-project-packagecloud-merge/21
+ :version: The semantic version string used for the package.
+
+:Optional Parameters:
+
+ :git_tag: The tag string to sign and push to the Git repository.
+ (default: the semantic version string)
+ :tag_release: Tag Gerrit Repo during the release process.
+ (default: true)
+
+The JSON schema for a PackageCloud release file appears below.
+
+.. literalinclude:: ../../schema/release-packagecloud-schema.yaml
+ :language: yaml
+
+
+Job Groups
+==========
+
+Below is a list of Release job groups:
+
+.. literalinclude:: ../../jjb/lf-release-job-groups.yaml
+ :language: yaml
Jenkins Jobs
This template supports Maven and Container release jobs.
-:Template Name: {project-name}-release-merge
+This template uses a git commit choosing strategy that builds the merged
+commit with the release yaml file, not the tip of the target branch, so
+projects can repeat the release action in case of merge job failure.
+
+:Template Name: {project-name}-release-merge-{stream}
:Comment Trigger: remerge
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
:build-timeout: Timeout in minutes before aborting build. (default: 15)
+ :stream: Keyword that represents a release code-name.
+ Often the same as the branch. (default: master)
:gerrit_merge_triggers: Override Gerrit Triggers.
:gerrit_trigger_file_paths: Override file paths filter which checks which
- file modifications will trigger a build.
- **default**::
-
- - compare-type: REG_EXP
- pattern: '(releases\/.*\.yaml|\.releases\/.*\.yaml)'
+ file modifications will trigger a build. The default pattern is the
+ regular expression ``(releases\/.*\.yaml|\.releases\/.*\.yaml)``
Release Verify
This template supports Maven and Container release jobs.
-:Template Name: {project-name}-release-verify
+:Template Name: {project-name}-release-verify-{stream}
:Comment Trigger: recheck|reverify
:build-timeout: Timeout in minutes before aborting build. (default: 15)
:gerrit-skip-vote: Skip voting for this job. (default: false)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+ :stream: Keyword that represents a release code-name.
+ Often the same as the branch. (default: master)
:gerrit_verify_triggers: Override Gerrit Triggers.
:gerrit_trigger_file_paths: Override file paths filter which checks which
- file modifications will trigger a build.
- **default**::
-
- - compare-type: REG_EXP
- pattern: '(releases\/.*\.yaml|\.releases\/.*\.yaml)'
+ file modifications will trigger a build. The default pattern is the
+ regular expression ``(releases\/.*\.yaml|\.releases\/.*\.yaml)``
PyPI Release Merge
file. Checks the format of the version string, downloads the package
artifacts from the PyPI staging repository, uploads the package
artifacts to the PyPI release repository, tags the git repository,
-signs the tag and pushes the tag to the git server. The release verify
+signs the tag and pushes the tag to the git server. The release merge
template accepts neither a branch nor a stream parameter.
+These templates use a git commit choosing strategy that builds the merged
+commit with the release yaml file, not the tip of the target branch, so
+projects can repeat the release action in case of merge job failure.
+
:Template Names:
- {project-name}-pypi-release-merge
should be the repository pypy.org. (default: pypi)
:use-release-file: Whether to use the release file. (default: true)
+ :gerrit_release_trigger_file_paths: Override file paths filter which checks
+ which file modifications will trigger a build. The default pattern is the
+ regular expression ``(releases\/pypi.*\.yaml|\.releases\/pypi.*\.yaml)``
PyPI Release Verify
~~~~~~~~~~~~~~~~~~~
should be the repository pypy.org (default: pypi)
:use-release-file: Whether to use the release file. (default: true)
+ :gerrit_release_trigger_file_paths: Override file paths filter which checks
+ which file modifications will trigger a build. The default pattern is the
+ regular expression ``(releases\/pypi.*\.yaml|\.releases\/pypi.*\.yaml)``
+
+PackageCloud Release Verify
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This template supports PackageCloud release jobs. Checks that the specified
+packages are present in the staging repository and absent from the release
+repository. The file path trigger uses the regular expression
+``(releases\/packagecloud.*\.yaml|\.releases\/packagecloud.*\.yaml)``
+
+:Template Name: {project-name}-packagecloud-release-verify
+
+:Comment Trigger: recheck|reverify
+
+:Required Parameters:
+
+ :build-node: The node to run build on.
+ :jenkins-ssh-credential: Credential to use for SSH. (Generally set
+ in defaults.yaml)
+ :project: Git repository name
+ :project-name: Jenkins job name prefix
+
+:Optional Parameters:
+
+ :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
+ :build-node: The node to run build on.
+ :build-timeout: Timeout in minutes before aborting build. (default: 15)
+ :gerrit-skip-vote: Skip voting for this job. (default: false)
+ :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+
+PackageCloud Release Merge
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This template supports PackageCloud release jobs. Promotes the specified
+packages from the staging repository to the release repository.
+The file path trigger uses the regular expression
+``(releases\/packagecloud.*\.yaml|\.releases\/packagecloud.*\.yaml)``
+
+This template uses a git commit choosing strategy that builds the merged
+commit with the release yaml file, not the tip of the target branch, so
+projects can repeat the release action in case of merge job failure.
+
+:Template Name: {project-name}-packagecloud-release-merge
+
+:Comment Trigger: remerge
+
+:Required Parameters:
+
+ :build-node: the node to run build on.
+ :jenkins-ssh-release-credential: credential to use for ssh. (generally set
+ in defaults.yaml)
+ :project: git repository name
+ :project-name: jenkins job name prefix
+
+:Optional Parameters:
+
+ :build-days-to-keep: days to keep build logs in jenkins. (default: 7)
+ :build-timeout: timeout in minutes before aborting build. (default: 15)
Setup for LFID, Nexus, Jenkins and Gerrit
-----------------------------------------
Code Review / releng / global-jjb.git / blobdiff