X-Git-Url: https://gerrit.linuxfoundation.org/infra/gitweb?a=blobdiff_plain;f=docs%2Fjjb%2Flf-release-jobs.rst;h=9d5b1886067393372e95fb0c81b6c8246e377199;hb=refs%2Fchanges%2F50%2F61650%2F2;hp=00c43cfba4d0f47ae4d498f09a58900672a3598b;hpb=1396009703a2baf17938da83891714ea3db52d83;p=releng%2Fglobal-jjb.git diff --git a/docs/jjb/lf-release-jobs.rst b/docs/jjb/lf-release-jobs.rst index 00c43cfb..9d5b1886 100644 --- a/docs/jjb/lf-release-jobs.rst +++ b/docs/jjb/lf-release-jobs.rst @@ -1,79 +1,166 @@ .. _lf-global-jjb-release: ####################### -Self Serve Release Jobs +Self-Serve Release Jobs ####################### -Self serve release jobs allow a project to create a releases directory and then place a release file in it. -Jenkins will pick this up and then promote the artifact from the staging log directory (log_dir) and tag the release -with the defined version. maven_central_url is optional +Self-serve release jobs allow a project team to direct Jenkins to +promote a jar file or container image from a staging area to a release +area. To trigger the action, create a releases/ or .releases/ +directory, add a release yaml file to it, and submit a change set with +one release yaml file to Gerrit. Upon merge of the change, Jenkins will +sign the reference extrapolated by log_dir and promote the artifact. The +expected format of the release yaml file appears in schemas and examples +below. + +The build node for maven and container release jobs must be CentOS, +which supports the sigul client for accessing a signing server. The +build node for container release jobs must have Docker installed. + +A Jenkins user can also trigger a release job via the "Build with +parameters" action, removing the need for a release yaml file. The +user must enter parameters in the same way as a release yaml file, +except for the special USE_RELEASE_FILE and DRY_RUN check boxes. The +user must uncheck the USE_RELEASE_FILE check box if the job should +run with a release file, while passing the required information as +build parameters. Similarly, the user must uncheck the DRY_RUN check +box to test the job while skipping repository promotion to Nexus. + +The special parameters are as follows:: + + GERRIT_BRANCH = master + VERSION = 1.0.0 + LOG_DIR = example-project-maven-stage-master/17/ + DISTRIBUTION_TYPE = maven + USE_RELEASE_FILE = false + DRY_RUN = false .. note:: - Example of a project's release file: + The release file regex is: (releases\/.*\.yaml|\.releases\/.*\.yaml). + In words, the directory name can be ".releases" or "releases"; the file + name can be anything with suffix ".yaml". + +The JSON schema for a maven release job appears below. + +.. code-block:: none + + --- + $schema: "http://json-schema.org/schema#" + $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-schema.yaml" + + required: + - "distribution_type" + - "log_dir" + - "project" + - "version" + + properties: + distribution_type: + type: "string" + log_dir: + type: "string" + project: + type: "string" + version: + type: "string" + + +Example of a maven release file: .. code-block:: bash - $ cat releases/1.0.0.yaml - --- - distribution_type: 'maven' - version: '1.0.0' - project: 'example-test-release' - log_dir: 'example-test-release-maven-stage-master/17/' - maven_central_url: 'oss.sonatype.org' + $ cat releases/1.0.0-maven.yaml + --- + distribution_type: 'maven' + version: '1.0.0' + project: 'example-project' + log_dir: 'example-project-maven-stage-master/17/' -.. note:: - Example of a terse Jenkins job to call global-jjb macro: +The JSON schema for a container release job appears below. .. code-block:: none - - project: - name: '{project-name}-gerrit-release-jobs' - project: 'example-test-release' - build-node: centos7-builder-2c-1g - project-name: example-test-release - jobs: - - '{project-name}-gerrit-release-jobs' + --- + $schema: "http://json-schema.org/schema#" + $id: "https://github.com/lfit/releng-global-jjb/blob/master/release-container-schema.yaml" + + required: + - "containers" + - "distribution_type" + - "project" + - "container_release_tag" + - "ref" + + properties: + containers: + type: "array" + properties: + name: + type: "string" + version: + type: "string" + additionalProperties: false + distribution_type: + type: "string" + project: + type: "string" + container_release_tag: + type: "string" + ref: + type: "string" + + +An example of a container release file appears below. The job applies the +container_release_tag string to all released containers. The job uses the +per-container version strings to pull images from the container registry. -.. note:: +.. code-block:: bash - Example of a verbose Jenkins job to call global-jjb macro: + $ cat releases/1.0.0-container.yaml + --- + distribution_type: 'container' + container_release_tag: '1.0.0' + project: 'test' + containers: + - name: test-backend + version: 1.0.0-20190806T184921Z + - name: test-frontend + version: 1.0.0-20190806T184921Z -.. code-block:: none - - project: - name: '{project-name}-releases-verify' - project: 'example-test-release' - build-node: centos7-builder-2c-1g - project-name: example-test-release - jobs: - - 'gerrit-releases-verify' +.. note:: + + Job should appear under gerrit-maven-stage + +Example of a terse Jenkins job to call the global-jjb macro: .. code-block:: none - - project: - name: '{project-name}-releases-merge' - project: 'example-test-release' - build-node: centos7-builder-2c-1g - project-name: example-test-release - jobs: - - 'gerrit-releases-merge' + - gerrit-maven-stage: + sign-artifacts: true + build-node: centos7-docker-8c-8g + maven-versions-plugin: true + - '{project-name}-gerrit-release-jobs': + build-node: centos7-docker-8c-8g .. note:: - Release Engineers Please follow the setup guide before adding the job definition: + Release Engineers: please follow the setup guide below before adding the job definition. -Setup for LFID Nexus Jenkins and Gerrit: -======================================== + +Setup for LFID, Nexus, Jenkins and Gerrit +========================================= LFID ==== Create an ``lfid`` and an ``ssh-key`` -``RELEASE_USERNAME`` -``RELEASE_EMAIL`` +``YOUR_RELEASE_USERNAME`` for example: onap-release + +``YOUR_RELEASE_EMAIL`` for example: collab-it+onap-release@linuxfoundation.org ssh-key example: @@ -82,7 +169,8 @@ ssh-key example: ssh-keygen -t rsa -C "collab-it+odl-release@linuxfoundation.org" -f /tmp/odl-release -`Create an LFID `_ +`Create an LFID with the above values `_ + Nexus ===== @@ -94,60 +182,69 @@ Create a Nexus account called ``'jenkins-release'`` with promote privileges. Gerrit ====== -Log into your Gerrit with ``RELEASE_USERNAME``, upload the ``ssh-key`` you created earlier. -Log out of Gerrit and log in again with your normal account for the next steps. +Log into your Gerrit with ``YOUR_RELEASE_USERNAME``, upload the public +part of the ``ssh-key`` you created earlier. Log out of Gerrit and log +in again with your normal account for the next steps. + + +In Gerrit create a new group called ``self-serve-release`` and give it +direct push rights via ``All-Projects`` Add ``YOUR_RELEASE_USERNAME`` +to group ``self-serve-release`` and group ``Non-Interactive Users`` -In Gerrit create a new group called ``self-serve-release`` and give it direct push rights via ``All-Projects`` -``push - refs/heads/*`` -1. Add a push reference -2. Set the ref as refs/heads/* -3. Make sure "force push" is not checked +In All project, grant group self-serve-release the following: + +.. code-block:: none -Add ``RELEASE_USERNAME`` to group ``self-serve-release`` and group ``Non-Interactive Users`` + [access "refs/heads/*"] + push = group self-serve-release + [access "refs/tags/*"] + createTag = group self-serve-release + createSignedTag = group self-serve-release + forgeCommitter = group self-serve-release + push = group self-serve-release -Give group ``self-serve-release`` Forge Committer rights on ``refs/tags/*`` -Give group ``self-serve-release`` Allow on ``Create Signed Tag`` -Give group ``self-serve-release`` Allow on ``Create Annotated Tag`` Jenkins ======= -Add a global credential to Jenkins called ``jenkins-release`` and set the ID: ``'jenkins-release'`` -as its value insert the ``ssh-key`` that you uploaded to Gerrit. +Add a global credential to Jenkins called ``jenkins-release`` and set +the ID: ``'jenkins-release'`` as its value insert the private half of +the ``ssh-key`` that you created for your Gerrit user. Add Global vars in Jenkins: -Jenkins configre -> Global properties -> Environment variables +Jenkins configure -> Global properties -> Environment variables -``RELEASE_USERNAME = $RELEASE_USERNAME`` -``RELEASE_EMAIL = $RELEASE_EMAIL`` +``RELEASE_USERNAME = YOUR_RELEASE_USERNAME`` +``RELEASE_EMAIL = YOUR_RELEASE_EMAIL`` -Jenkins configre -> Managed Files -> Custom File +Jenkins configure -> Managed Files -> Add a New Config -> Custom File id: signing-pubkey Name: SIGNING_PUBKEY (optional) Comment: SIGNING_PUBKEY (optional) -Content: (ask andy) +Content: (Ask Andy for the public signing key) -----BEGIN PGP PUBLIC KEY BLOCK----- -Add or edit the managed file in Jenkins called ``lftoolsini``, appending a nexus section: -Jenkins Settings -> Managed files -> Add (or edit) -> Custom file +Add or edit the managed file in Jenkins called ``lftoolsini``, +appending a nexus section: Jenkins Settings -> Managed files -> Add +(or edit) -> Custom file .. code-block:: none - [nexus] + [nexus.example.com] username=jenkins-release - password=redacted + password= Ci-management ============= -upgrade you projects global-jjb if needed -add this to your global defaults file (eg: jjb/defaults.yaml) +Upgrade your project's global-jjb if needed, then add the following to +your global defaults file (e.g., jjb/defaults.yaml). -.. code-block:: bash +.. code-block:: none jenkins-ssh-release-credential: 'jenkins-release' @@ -157,8 +254,8 @@ Macros lf-release ---------- -Release verify and merge jobs are the same except for their scm, trigger, and -builders definition. This anchor is the common template. +Release verify and merge jobs are the same except for their scm, +trigger, and builders definition. This anchor is the common template. Job Templates ============= @@ -166,21 +263,7 @@ Job Templates Release Merge ------------- -Runs: - -- sigul-install -- sigul-configuration -- checkout ref from taglist.log -- applies the $PROJECT.bundle -- signs, tags and pushes - -.. code-block:: bash - - lftools nexus release --server $NEXUS_URL $STAGING_REPO - - -:Template Name: - - {project-name}-release-merge-{stream} +:Template Name: {project-name}-release-merge :Comment Trigger: remerge @@ -189,11 +272,11 @@ Runs: :build-node: The node to run build on. :jenkins-ssh-release-credential: Credential to use for SSH. (Generally set in defaults.yaml) - :stream: run this job against: master + :stream: run this job against: ** :Optional parameters: - :branch: Git branch to fetch for the build. (default: master) + :branch: Git branch to fetch for the build. (default: all) :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7) :build-timeout: Timeout in minutes before aborting build. (default: 15) :project-pattern: Project to trigger build against. (default: \*\*) @@ -203,38 +286,27 @@ Runs: file modifications will trigger a build. **default**:: - - compare-type: ANT - pattern: 'releases/*.yaml' + - compare-type: REG_EXP + pattern: '(releases\/.*\.yaml|\.releases\/.*\.yaml)' Release Verify ------------------ -Release verify job checks the schema and ensures that the staging-repo.txt.gz -is available on the job. - -- sigul-install -- sigul-configuration -- checkout ref from taglist.log -- applies the $PROJECT.bundle -- signs and shows signature - - -:Template Names: - - {project-name}-release-verify-{stream} +:Template Name: {project-name}-release-verify :Comment Trigger: recheck|reverify :Required Parameters: :build-node: The node to run build on. - :jenkins-ssh-release-credential: Credential to use for SSH. (Generally set + :jenkins-ssh-credential: Credential to use for SSH. (Generally set in defaults.yaml) - :stream: run this job against: master + :stream: run this job against: ** :Optional Parameters: - :branch: Git branch to fetch for the build. (default: master) + :branch: Git branch to fetch for the build. (default: all) :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7) :build-node: The node to run build on. :build-timeout: Timeout in minutes before aborting build. (default: 15) @@ -249,5 +321,5 @@ is available on the job. file modifications will trigger a build. **default**:: - - compare-type: ANT - pattern: 'releases/*.yaml' + - compare-type: REG_EXP + pattern: '(releases\/.*\.yaml|\.releases\/.*\.yaml)'