1 .. _lf-global-jjb-release:
3 Self-Serve Release Jobs
4 =======================
6 Self-serve release jobs allow project committers to direct Jenkins to
7 promote a jar file, container image or Python package from a staging
8 area to a release area.
10 To use the self-release process, create a releases/ or .releases/
11 directory at the root of the project repository, add one release yaml
12 file to it, and submit a change set with that release yaml file. The
13 required contents of the release yaml file are different for each type
14 of release, see the schemas and examples shown below. The version in
15 the release yaml file must be a valid Semantic Versioning (SemVer)
16 string, matching either the pattern "v#.#.#" or "#.#.#" where "#" is
17 one or more digits. Upon merge of the change, Jenkins will sign the
18 reference extrapolated by log_dir and promote the artifact.
22 The release file regex is: (releases\/.*\.yaml|\.releases\/.*\.yaml).
23 In words, the directory name can be ".releases" or "releases"; the file
24 name can be anything with suffix ".yaml".
26 The build node for all release jobs must be CentOS, which supports the
27 sigul client for accessing a signing server. The build node for
28 container release jobs must have Docker installed.
30 A Jenkins admin user can also trigger a release job via the "Build
31 with parameters" action, removing the need to create and merge a
32 release yaml file. The user must enter parameters in the same way as
33 a release yaml file, except for the special USE_RELEASE_FILE and
34 DRY_RUN check boxes. The user must uncheck the USE_RELEASE_FILE check
35 box if the job should run without a release file, instead passing the
36 required information as build parameters. The user can check the
37 DRY_RUN check box to test the job while skipping upload of files to
38 the release repository.
40 For example, the parameters for a Maven release are as follows::
42 GERRIT_BRANCH = master
44 LOG_DIR = example-project-maven-stage-master/17/
45 DISTRIBUTION_TYPE = maven
46 USE_RELEASE_FILE = false
52 An example of a maven release file appears below.
56 $ cat releases/maven-release.yaml
58 distribution_type: maven
59 log_dir: example-project-maven-stage-master/17/
60 project: example-project
64 The following parameters must appear in a maven release yaml file.
68 :distribution_type: Must be "maven".
69 :log_dir: The suffix of the logs URL reported on completion by the
70 Jenkins stage job that created and pushed the artifact
71 to the staging repository. For example, use value
72 "example-project-maven-stage-master/17" for the logs URL
73 https://logs.lf-project.org/production/vex-sjc-lfp-jenkins-prod-1/example-project-maven-stage-master/17
74 :project: The name of the project.
75 :version: The semantic version string used for the artifact.
77 The JSON schema for a maven release file appears below.
82 $schema: "http://json-schema.org/schema#"
83 $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-schema.yaml"
102 Container Release Files
103 -----------------------
105 An example of a container release file appears below.
109 $ cat releases/container-release.yaml
111 distribution_type: container
112 container_release_tag: 1.0.0
113 container_pull_registry: nexus.onap.org:10003
114 container_push_registry: nexus.onap.org:10002
116 ref: d1b9cd2dd345fbeec0d3e2162e008358b8b663b2
119 version: 1.0.0-20190806T184921Z
120 - name: test-frontend
121 version: 1.0.0-20190806T184921Z
124 The following parameters must appear in a container release yaml file.
126 :Required Parameters:
128 :distribution_type: Must be "container".
129 :container_release_tag: The string to use as a Docker tag on all
131 :container_pull_registry: The Nexus registry that supplies the staged
133 :container_push_registry: The Nexus registry that receives the released
135 :project: The name of the project.
136 :ref: The git commit reference (SHA-1 code) to tag with the version string.
137 :containers: A list of name and version (tag) pairs that specify the
138 Docker images in the container-pull registry to promote to the
139 container-push registry.
141 The JSON schema for a container release file appears below.
146 $schema: "http://json-schema.org/schema#"
147 $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-container-schema.yaml"
151 - "distribution_type"
153 - "container_release_tag"
164 additionalProperties: false
169 container_release_tag:
171 container_pull_registry"
173 container_push_registry"
182 An example of a PyPI release file appears below. Name of the release file must
183 start with "pypi". For example releases/pypi-1.0.0-mypackage.yaml
187 $ cat releases/pypi-1.0.0-mypackage.yaml
189 pypi_project: mypackage
190 python_version: '3.4'
192 log_dir: example-project-pypi-merge-master/17
195 The following parameters must appear in the PyPI release yaml file.
196 These are not part of the Jenkins job definition to allow independent
197 self-release of a package maintained in a git repository with other
200 :Required Parameters:
202 :log_dir: The suffix of the logs URL reported on completion by the
203 Jenkins merge job that created and pushed the distribution files
204 to the staging repository. For example, use value
205 "example-project-pypi-merge-master/17" for the logs URL
206 https://logs.lf-project.org/production/vex-sjc-lfp-jenkins-prod-1/example-project-pypi-merge-master/17
207 :pypi_project: The PyPI project name at the staging and
208 release repositories, for example "mypackage".
209 :python_version: The Python interpreter version to use for pip
210 "Requires-Python" compatibility checks, for example '3', '3.7' or 3.7.4.
211 Put valid decimal values such as 3 or 3.7 in quotes to pass schema validation.
212 :version: The semantic version string used for the package in the
215 The JSON schema for a PyPI release file appears below.
220 $schema: "http://json-schema.org/schema#"
221 $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-pypi-schema.yaml"
240 PackageCloud Release Files
241 --------------------------
243 An example of a PackageCloud release file appears below. Name of release file
244 must start with "packagecloud". For example releases/packagecloud-1.6-tree.yaml
248 $ cat releases/packagecloud-1.6-tree.yaml
251 - name: tree-1.6.0-10.el7.x86_64.rpm
254 The following parameters must appear in the PackageCloud release yaml file.
255 These are not part of the Jenkins job definition to allow independent
256 self-release of a package maintained in a git repository with other
259 :Required Parameters:
261 :package_name: A list of names that specify the packages to promote.
262 (Found in jenkins console log when using gem to push package eg.
263 "Pushing /path/of/package/name-of-package.rpm... success!"
264 OR using rest api call to query packagecloud.io repo
265 "curl https://packagecloud.io/api/v1/repos/test_user/test_repo/search?q=
266 | yq -r .[].filename"
268 The JSON schema for a PackageCloud release file appears below.
273 $schema: "http://json-schema.org/schema#"
274 $id: "https://github.com/lfit/releng-global-jjb/blob/master/packagecloud-release-schema"
289 An example of a Jenkins job configuration that uses the global-jjb
290 templates for maven and container release jobs appears next.
295 name: my-project-release
297 project-name: my-project
298 build-node: centos7-docker-4c-4g
299 mvn-settings: my-project-settings
301 - '{project-name}-gerrit-release-jobs'
306 Release Engineers: please follow the setup guide below before adding the job definition.
315 Release verify and merge jobs are the same except for their scm,
316 trigger, and builders definition. This anchor is the common template.
324 This template supports Maven and Container release jobs.
326 :Template Name: {project-name}-release-merge
328 :Comment Trigger: remerge
330 :Required parameters:
332 :build-node: The node to run build on.
333 :jenkins-ssh-release-credential: Credential to use for SSH. (Generally set
335 :project: Git repository name
336 :project-name: Jenkins job name prefix
338 :Optional parameters:
340 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
341 :build-timeout: Timeout in minutes before aborting build. (default: 15)
343 :gerrit_merge_triggers: Override Gerrit Triggers.
344 :gerrit_trigger_file_paths: Override file paths filter which checks which
345 file modifications will trigger a build.
348 - compare-type: REG_EXP
349 pattern: '(releases\/.*\.yaml|\.releases\/.*\.yaml)'
355 This template supports Maven and Container release jobs.
357 :Template Name: {project-name}-release-verify
359 :Comment Trigger: recheck|reverify
361 :Required Parameters:
363 :build-node: The node to run build on.
364 :jenkins-ssh-credential: Credential to use for SSH. (Generally set
366 :project: Git repository name
367 :project-name: Jenkins job name prefix
369 :Optional Parameters:
371 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
372 :build-node: The node to run build on.
373 :build-timeout: Timeout in minutes before aborting build. (default: 15)
374 :gerrit-skip-vote: Skip voting for this job. (default: false)
375 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
377 :gerrit_verify_triggers: Override Gerrit Triggers.
378 :gerrit_trigger_file_paths: Override file paths filter which checks which
379 file modifications will trigger a build.
382 - compare-type: REG_EXP
383 pattern: '(releases\/.*\.yaml|\.releases\/.*\.yaml)'
389 Publishes a Python package on merge of a patch set with a release yaml
390 file. Checks the format of the version string, downloads the package
391 artifacts from the PyPI staging repository, uploads the package
392 artifacts to the PyPI release repository, tags the git repository,
393 signs the tag and pushes the tag to the git server. The release merge
394 template accepts neither a branch nor a stream parameter.
398 - {project-name}-pypi-release-merge
399 - gerrit-pypi-release-merge
400 - github-pypi-release-merge
402 :Comment Trigger: remerge
404 :Required Parameters:
406 :build-node: The node to run build on, which must be Centos.
407 :jenkins-ssh-release-credential: Credential to use for SSH. (Generally set
409 :project: Git repository name
410 :project-name: Jenkins job name prefix
412 :Optional Parameters:
414 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
415 :build-timeout: Timeout in minutes before aborting build. (default: 15)
416 :disable-job: Whether to disable the job (default: false)
417 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
418 :pypi-stage-index: Base URL of the PyPI staging repository.
419 (default https://test.pypi.org/simple)
420 :pypi-repo: Key for the PyPI release repository in the .pypirc file,
421 should be the repository pypy.org. (default: pypi)
422 :use-release-file: Whether to use the release file. (default: true)
424 :gerrit_trigger_file_paths: Override file paths filter which checks which
425 file modifications will trigger a build.
428 - compare-type: REG_EXP
429 pattern: '(releases\/pypi.*\.yaml|\.releases\/pypi.*\.yaml)'
434 Verifies a Python package project on creation of a patch set with a
435 release yaml file. Checks the contents of the release yaml file,
436 checks the format of the version string, and downloads the release
437 artifacts from the specified PyPI staging repository. The release
438 verify template accepts neither a branch nor a stream parameter.
442 - {project-name}-pypi-release-verify
443 - gerrit-pypi-release-verify
444 - github-pypi-release-verify
446 :Comment Trigger: recheck
448 :Required Parameters:
450 :build-node: The node to run build on, which must be Centos.
451 :jenkins-ssh-credential: Credential to use for SSH. (Generally set
453 :project: Git repository name
454 :project-name: Jenkins job name prefix
456 :Optional Parameters:
458 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
459 :build-timeout: Timeout in minutes before aborting build. (default: 15)
460 :disable-job: Whether to disable the job (default: false)
461 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
462 :pypi-stage-index: Base URL of the PyPI staging repository.
463 (default https://test.pypi.org/simple)
464 :pypi-repo: Key for the PyPI release repository in the .pypirc file,
465 should be the repository pypy.org (default: pypi)
466 :use-release-file: Whether to use the release file. (default: true)
468 :gerrit_trigger_file_paths: Override file paths filter which checks which
469 file modifications will trigger a build.
472 - compare-type: REG_EXP
473 pattern: '(releases\/pypi.*\.yaml|\.releases\/pypi.*\.yaml)'
475 PackageCloud Release Verify
476 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
478 This template supports PackageCloud release jobs.
480 :Template Name: {project-name}-packagecloud-release-verify
482 :Comment Trigger: recheck|reverify
484 :Required Parameters:
486 :build-node: The node to run build on.
487 :jenkins-ssh-credential: Credential to use for SSH. (Generally set
489 :project: Git repository name
490 :project-name: Jenkins job name prefix
492 :Optional Parameters:
494 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
495 :build-node: The node to run build on.
496 :build-timeout: Timeout in minutes before aborting build. (default: 15)
497 :gerrit-skip-vote: Skip voting for this job. (default: false)
498 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
500 :gerrit_verify_triggers: Override Gerrit Triggers.
501 :gerrit_trigger_file_paths: Override file paths filter which checks which
502 file modifications will trigger a build.
505 - compare-type: REG_EXP
506 pattern: '(releases\/packagecloud.*\.yaml|\.releases\/packagecloud.*\.yaml)'
509 PackageCloud Release Merge
510 ~~~~~~~~~~~~~~~~~~~~~~~~~~
512 This template supports PackageCloud release jobs.
514 :template name: {project-name}-packagecloud-release-merge
516 :comment trigger: remerge
518 :required parameters:
520 :build-node: the node to run build on.
521 :jenkins-ssh-release-credential: credential to use for ssh. (generally set
523 :project: git repository name
524 :project-name: jenkins job name prefix
526 :optional parameters:
528 :build-days-to-keep: days to keep build logs in jenkins. (default: 7)
529 :build-timeout: timeout in minutes before aborting build. (default: 15)
531 :gerrit_merge_triggers: override gerrit triggers.
532 :gerrit_trigger_file_paths: override file paths filter which checks which
533 file modifications will trigger a build.
536 - compare-type: reg_exp
537 pattern: '(releases\/packagecloud.*\.yaml|\.releases\/packagecloud.*\.yaml)'
540 Setup for LFID, Nexus, Jenkins and Gerrit
541 -----------------------------------------
543 This section is for the Linux Foundation release engineering team.
548 Create an ``lfid`` and an ``ssh-key``
550 ``YOUR_RELEASE_USERNAME`` for example: onap-release
552 ``YOUR_RELEASE_EMAIL`` for example: collab-it+onap-release@linuxfoundation.org
558 ssh-keygen -t rsa -C "collab-it+odl-release@linuxfoundation.org" -f /tmp/odl-release
561 `Create an LFID with the above values <https://identity.linuxfoundation.org>`_
567 Create a Nexus account called ``'jenkins-release'`` with promote privileges.
569 .. image:: ../_static/nexus-promote-privs.png
574 Log into your Gerrit with ``YOUR_RELEASE_USERNAME``, upload the public
575 part of the ``ssh-key`` you created earlier. Log out of Gerrit and log
576 in again with your normal account for the next steps.
579 In Gerrit create a new group called ``self-serve-release`` and give it
580 direct push rights via ``All-Projects`` Add ``YOUR_RELEASE_USERNAME``
581 to group ``self-serve-release`` and group ``Non-Interactive Users``
584 In All project, grant group self-serve-release the following:
588 [access "refs/heads/*"]
589 push = group self-serve-release
590 [access "refs/tags/*"]
591 createTag = group self-serve-release
592 createSignedTag = group self-serve-release
593 forgeCommitter = group self-serve-release
594 push = group self-serve-release
600 Add a global credential to Jenkins called ``jenkins-release`` and set
601 the ID: ``'jenkins-release'`` as its value insert the private half of
602 the ``ssh-key`` that you created for your Gerrit user.
604 Add Global variables in Jenkins:
605 Jenkins configure -> Global properties -> Environment variables::
607 RELEASE_USERNAME = YOUR_RELEASE_USERNAME
608 RELEASE_EMAIL = YOUR_RELEASE_EMAIL
613 Add these variables to your global-vars-$SILO.sh file or they will
616 Jenkins configure -> Managed Files -> Add a New Config -> Custom File
621 Name: SIGNING_PUBKEY (optional)
622 Comment: SIGNING_PUBKEY (optional)
624 Content: (Ask Andy for the public signing key)
625 -----BEGIN PGP PUBLIC KEY BLOCK-----
628 Add or edit the managed file in Jenkins called ``lftoolsini``,
629 appending a nexus section: Jenkins Settings -> Managed files -> Add
630 (or edit) -> Custom file
635 username=jenkins-release
636 password=<plaintext password>
641 Upgrade your project's global-jjb if needed, then add the following to
642 your global defaults file (e.g., jjb/defaults.yaml).
646 jenkins-ssh-release-credential: jenkins-release