1 .. _lf-global-jjb-release:
3 Self-Serve Release Jobs
4 =======================
6 Self-serve release jobs allow project committers to direct Jenkins to
7 promote a jar file, container image or Python package from a staging
8 area to a release area.
10 To use the self-release process, create a releases/ or .releases/
11 directory at the root of the project repository, add one release yaml
12 file to it, and submit a change set with that release yaml file. The
13 required contents of the release yaml file are different for each type
14 of release, see the schemas and examples shown below. The version in
15 the release yaml file must be a valid Semantic Versioning (SemVer)
16 string, matching either the pattern "v#.#.#" or "#.#.#" where "#" is
17 one or more digits. Upon merge of the change, Jenkins will sign the
18 reference extrapolated by log_dir and promote the artifact.
22 The release file regex is: (releases\/.*\.yaml|\.releases\/.*\.yaml).
23 In words, the directory name can be ".releases" or "releases"; the file
24 name can be anything with suffix ".yaml".
26 The build node for all release jobs must be CentOS, which supports the
27 sigul client for accessing a signing server. The build node for
28 container release jobs must have Docker installed.
30 A Jenkins admin user can also trigger a release job via the "Build
31 with parameters" action, removing the need to create and merge a
32 release yaml file. The user must enter parameters in the same way as
33 a release yaml file, except for the special USE_RELEASE_FILE and
34 DRY_RUN check boxes. The user must uncheck the USE_RELEASE_FILE check
35 box if the job should run without a release file, instead passing the
36 required information as build parameters. The user can check the
37 DRY_RUN check box to test the job while skipping upload of files to
38 the release repository.
40 For example, the parameters for a Maven release are as follows::
42 GERRIT_BRANCH = master
44 LOG_DIR = example-project-maven-stage-master/17/
45 DISTRIBUTION_TYPE = maven
46 USE_RELEASE_FILE = false
52 An example of a maven release file appears below. Name of release file
53 must start with "maven". eg. releases/maven-1.0.0-test.yaml
57 $ cat releases/maven-release.yaml
59 distribution_type: maven
60 log_dir: example-project-maven-stage-master/17/
61 project: example-project
65 The following parameters must appear in a maven release yaml file.
69 :distribution_type: Must be "maven".
70 :log_dir: The suffix of the logs URL reported on completion by the
71 Jenkins stage job that created and pushed the artifact
72 to the staging repository. For example, use value
73 "example-project-maven-stage-master/17" for the logs URL
74 https://logs.lf-project.org/production/vex-sjc-lfp-jenkins-prod-1/example-project-maven-stage-master/17
75 :project: The name of the project.
76 :version: The semantic version string used for the artifact.
78 The JSON schema for a maven release file appears below.
83 $schema: "http://json-schema.org/schema#"
84 $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-schema.yaml"
103 Container Release Files
104 -----------------------
106 An example of a container release file appears below.
110 $ cat releases/container-release.yaml
112 distribution_type: container
113 container_release_tag: 1.0.0
114 container_pull_registry: nexus.onap.org:10003
115 container_push_registry: nexus.onap.org:10002
117 ref: d1b9cd2dd345fbeec0d3e2162e008358b8b663b2
120 version: 1.0.0-20190806T184921Z
121 - name: test-frontend
122 version: 1.0.0-20190806T184921Z
125 The following parameters must appear in a container release yaml file.
127 :Required Parameters:
129 :distribution_type: Must be "container".
130 :container_release_tag: The string to use as a Docker tag on all
132 :container_pull_registry: The Nexus registry that supplies the staged
134 :container_push_registry: The Nexus registry that receives the released
136 :project: The name of the project.
137 :ref: The git commit reference (SHA-1 code) to tag with the version string.
138 :containers: A list of name and version (tag) pairs that specify the
139 Docker images in the container-pull registry to promote to the
140 container-push registry.
142 The JSON schema for a container release file appears below.
147 $schema: "http://json-schema.org/schema#"
148 $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-container-schema.yaml"
152 - "distribution_type"
154 - "container_release_tag"
165 additionalProperties: false
170 container_release_tag:
172 container_pull_registry"
174 container_push_registry"
183 An example of a PyPI release file appears below. Name of the release file must
184 start with "pypi". eg. releases/pypi-1.0.0-mypackage.yaml
188 $ cat releases/pypi-1.0.0-mypackage.yaml
190 distribution_type: pypi
191 pypi_project: mypackage
192 python_version: '3.4'
196 The following parameters must appear in the PyPI release yaml file.
197 These are not part of the Jenkins job definition to allow independent
198 self-release of a package maintained in a git repository with other
201 :Required Parameters:
203 :distribution_type: Must be "pypi".
204 :log_dir: The suffix of the logs URL reported on completion by the
205 Jenkins merge job that created and pushed the distribution files
206 to the staging repository. For example, use value
207 "example-project-pypi-merge-master/17" for the logs URL
208 https://logs.lf-project.org/production/vex-sjc-lfp-jenkins-prod-1/example-project-pypi-merge-master/17
209 :pypi_project: The PyPI project name at the staging and
210 release repositories, for example "mypackage".
211 :python_version: The Python interpreter version to use for pip
212 "Requires-Python" compatibility checks, for example '3', '3.7' or 3.7.4.
213 Put valid decimal values such as 3 or 3.7 in quotes to pass schema validation.
214 :version: The semantic version string used for the package in the
217 The JSON schema for a PyPI release file appears below.
222 $schema: "http://json-schema.org/schema#"
223 $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-pypi-schema.yaml"
226 - "distribution_type"
245 PackageCloud Release Files
246 --------------------------
248 An example of a PackageCloud release file appears below. Name of release file
249 must start with "packagecloud". eg. releases/packagecloud-1.6.0-tree.yaml
253 $ cat releases/packagecloud-1.6.0-tree.yaml
255 distribution_type: packagecloud
257 - name: 'tree-1.6.0-10.el7.x86_64.rpm'
260 The following parameters must appear in the PackageCloud release yaml file.
261 These are not part of the Jenkins job definition to allow independent
262 self-release of a package maintained in a git repository with other
265 :Required Parameters:
267 :distribution_type: Must be "packagecloud".
268 :package_name: A list of names that specify the packages to promote.
269 (Found via jenkins log when using gem to initally push package up eg.
270 "Pushing /path/of/package/name-of-package.rpm... success!"
271 OR using rest api call with generated token from packagecloud.io
272 "curl https://packagecloud.io/api/v1/repos/test_user/test_repo/search?q=
273 | yq -r .[].filename"
275 The JSON schema for a PackageCloud release file appears below.
280 $schema: "http://json-schema.org/schema#"
281 $id: "https://github.com/lfit/releng-global-jjb/blob/master/packagecloud-release-schema"
285 - "distribution_type"
299 An example of a Jenkins job configuration that uses the global-jjb
300 templates for maven and container release jobs appears next.
305 name: my-project-release
307 project-name: my-project
308 build-node: centos7-docker-4c-4g
309 mvn-settings: my-project-settings
311 - '{project-name}-gerrit-release-jobs'
316 Release Engineers: please follow the setup guide below before adding the job definition.
325 Release verify and merge jobs are the same except for their scm,
326 trigger, and builders definition. This anchor is the common template.
334 This template supports Maven and Container release jobs.
336 :Template Name: {project-name}-release-merge
338 :Comment Trigger: remerge
340 :Required parameters:
342 :build-node: The node to run build on.
343 :jenkins-ssh-release-credential: Credential to use for SSH. (Generally set
345 :project: Git repository name
346 :project-name: Jenkins job name prefix
348 :Optional parameters:
350 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
351 :build-timeout: Timeout in minutes before aborting build. (default: 15)
353 :gerrit_merge_triggers: Override Gerrit Triggers.
354 :gerrit_trigger_file_paths: Override file paths filter which checks which
355 file modifications will trigger a build.
358 - compare-type: REG_EXP
359 pattern: '(releases\/.*\.yaml|\.releases\/.*\.yaml)'
365 This template supports Maven and Container release jobs.
367 :Template Name: {project-name}-release-verify
369 :Comment Trigger: recheck|reverify
371 :Required Parameters:
373 :build-node: The node to run build on.
374 :jenkins-ssh-credential: Credential to use for SSH. (Generally set
376 :project: Git repository name
377 :project-name: Jenkins job name prefix
379 :Optional Parameters:
381 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
382 :build-node: The node to run build on.
383 :build-timeout: Timeout in minutes before aborting build. (default: 15)
384 :gerrit-skip-vote: Skip voting for this job. (default: false)
385 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
387 :gerrit_verify_triggers: Override Gerrit Triggers.
388 :gerrit_trigger_file_paths: Override file paths filter which checks which
389 file modifications will trigger a build.
392 - compare-type: REG_EXP
393 pattern: '(releases\/maven.*\.yaml|\.releases\/maven.*\.yaml)'
399 Publishes a Python package on merge of a patch set with a release yaml
400 file. Checks the format of the version string, downloads the package
401 artifacts from the PyPI staging repository, uploads the package
402 artifacts to the PyPI release repository, tags the git repository,
403 signs the tag and pushes the tag to the git server. The release verify
404 template accepts neither a branch nor a stream parameter.
408 - {project-name}-pypi-release-merge
409 - gerrit-pypi-release-merge
410 - github-pypi-release-merge
412 :Comment Trigger: remerge
414 :Required Parameters:
416 :build-node: The node to run build on, which must be Centos.
417 :jenkins-ssh-release-credential: Credential to use for SSH. (Generally set
419 :project: Git repository name
420 :project-name: Jenkins job name prefix
422 :Optional Parameters:
424 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
425 :build-timeout: Timeout in minutes before aborting build. (default: 15)
426 :disable-job: Whether to disable the job (default: false)
427 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
428 :pypi-stage-index: Base URL of the PyPI staging repository.
429 (default https://test.pypi.org/simple)
430 :pypi-repo: Key for the PyPI release repository in the .pypirc file,
431 should be the repository pypy.org. (default: pypi)
432 :use-release-file: Whether to use the release file. (default: true)
434 :gerrit_trigger_file_paths: Override file paths filter which checks which
435 file modifications will trigger a build.
438 - compare-type: REG_EXP
439 pattern: '(releases\/pypi.*\.yaml|\.releases\/pypi.*\.yaml)'
444 Verifies a Python package project on creation of a patch set with a
445 release yaml file. Checks the contents of the release yaml file,
446 checks the format of the version string, and downloads the release
447 artifacts from the specified PyPI staging repository. The release
448 verify template accepts neither a branch nor a stream parameter.
452 - {project-name}-pypi-release-verify
453 - gerrit-pypi-release-verify
454 - github-pypi-release-verify
456 :Comment Trigger: recheck
458 :Required Parameters:
460 :build-node: The node to run build on, which must be Centos.
461 :jenkins-ssh-credential: Credential to use for SSH. (Generally set
463 :project: Git repository name
464 :project-name: Jenkins job name prefix
466 :Optional Parameters:
468 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
469 :build-timeout: Timeout in minutes before aborting build. (default: 15)
470 :disable-job: Whether to disable the job (default: false)
471 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
472 :pypi-stage-index: Base URL of the PyPI staging repository.
473 (default https://test.pypi.org/simple)
474 :pypi-repo: Key for the PyPI release repository in the .pypirc file,
475 should be the repository pypy.org (default: pypi)
476 :use-release-file: Whether to use the release file. (default: true)
478 :gerrit_trigger_file_paths: Override file paths filter which checks which
479 file modifications will trigger a build.
482 - compare-type: REG_EXP
483 pattern: '(releases\/pypi.*\.yaml|\.releases\/pypi.*\.yaml)'
485 PackageCloud Release Verify
486 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
488 This template supports PackageCloud release jobs.
490 :Template Name: {project-name}-packagecloud-release-verify
492 :Comment Trigger: recheck|reverify
494 :Required Parameters:
496 :build-node: The node to run build on.
497 :jenkins-ssh-credential: Credential to use for SSH. (Generally set
499 :project: Git repository name
500 :project-name: Jenkins job name prefix
502 :Optional Parameters:
504 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
505 :build-node: The node to run build on.
506 :build-timeout: Timeout in minutes before aborting build. (default: 15)
507 :gerrit-skip-vote: Skip voting for this job. (default: false)
508 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
510 :gerrit_verify_triggers: Override Gerrit Triggers.
511 :gerrit_trigger_file_paths: Override file paths filter which checks which
512 file modifications will trigger a build.
515 - compare-type: REG_EXP
516 pattern: '(releases\/packagecloud.*\.yaml|\.releases\/packagecloud.*\.yaml)'
519 PackageCloud Release Merge
520 ~~~~~~~~~~~~~~~~~~~~~~~~~~
522 This template supports PackageCloud release jobs.
524 :template name: {project-name}-packagecloud-release-merge
526 :comment trigger: remerge
528 :required parameters:
530 :build-node: the node to run build on.
531 :jenkins-ssh-release-credential: credential to use for ssh. (generally set
533 :project: git repository name
534 :project-name: jenkins job name prefix
536 :optional parameters:
538 :build-days-to-keep: days to keep build logs in jenkins. (default: 7)
539 :build-timeout: timeout in minutes before aborting build. (default: 15)
541 :gerrit_merge_triggers: override gerrit triggers.
542 :gerrit_trigger_file_paths: override file paths filter which checks which
543 file modifications will trigger a build.
546 - compare-type: reg_exp
547 pattern: '(releases\/packagecloud.*\.yaml|\.releases\/packagecloud.*\.yaml)'
550 Setup for LFID, Nexus, Jenkins and Gerrit
551 -----------------------------------------
553 This section is for the Linux Foundation release engineering team.
558 Create an ``lfid`` and an ``ssh-key``
560 ``YOUR_RELEASE_USERNAME`` for example: onap-release
562 ``YOUR_RELEASE_EMAIL`` for example: collab-it+onap-release@linuxfoundation.org
568 ssh-keygen -t rsa -C "collab-it+odl-release@linuxfoundation.org" -f /tmp/odl-release
571 `Create an LFID with the above values <https://identity.linuxfoundation.org>`_
577 Create a Nexus account called ``'jenkins-release'`` with promote privileges.
579 .. image:: ../_static/nexus-promote-privs.png
584 Log into your Gerrit with ``YOUR_RELEASE_USERNAME``, upload the public
585 part of the ``ssh-key`` you created earlier. Log out of Gerrit and log
586 in again with your normal account for the next steps.
589 In Gerrit create a new group called ``self-serve-release`` and give it
590 direct push rights via ``All-Projects`` Add ``YOUR_RELEASE_USERNAME``
591 to group ``self-serve-release`` and group ``Non-Interactive Users``
594 In All project, grant group self-serve-release the following:
598 [access "refs/heads/*"]
599 push = group self-serve-release
600 [access "refs/tags/*"]
601 createTag = group self-serve-release
602 createSignedTag = group self-serve-release
603 forgeCommitter = group self-serve-release
604 push = group self-serve-release
610 Add a global credential to Jenkins called ``jenkins-release`` and set
611 the ID: ``'jenkins-release'`` as its value insert the private half of
612 the ``ssh-key`` that you created for your Gerrit user.
614 Add Global variables in Jenkins:
615 Jenkins configure -> Global properties -> Environment variables::
617 RELEASE_USERNAME = YOUR_RELEASE_USERNAME
618 RELEASE_EMAIL = YOUR_RELEASE_EMAIL
623 Add these variables to your global-vars-$SILO.sh file or they will
626 Jenkins configure -> Managed Files -> Add a New Config -> Custom File
631 Name: SIGNING_PUBKEY (optional)
632 Comment: SIGNING_PUBKEY (optional)
634 Content: (Ask Andy for the public signing key)
635 -----BEGIN PGP PUBLIC KEY BLOCK-----
638 Add or edit the managed file in Jenkins called ``lftoolsini``,
639 appending a nexus section: Jenkins Settings -> Managed files -> Add
640 (or edit) -> Custom file
645 username=jenkins-release
646 password=<plaintext password>
651 Upgrade your project's global-jjb if needed, then add the following to
652 your global defaults file (e.g., jjb/defaults.yaml).
656 jenkins-ssh-release-credential: jenkins-release