X-Git-Url: https://gerrit.linuxfoundation.org/infra/gitweb?a=blobdiff_plain;f=docs%2Fjjb%2Flf-ci-jobs.rst;h=6a8caff691c27528b09a56ab0fced55b3eb728fb;hb=refs%2Fchanges%2F50%2F12550%2F4;hp=2cb767a0ac49ed469f37b203548c3913ed916ff2;hpb=c26afec952891def4ae050c4a9d9d3a13dedd149;p=releng%2Fglobal-jjb.git diff --git a/docs/jjb/lf-ci-jobs.rst b/docs/jjb/lf-ci-jobs.rst index 2cb767a0..6a8caff6 100644 --- a/docs/jjb/lf-ci-jobs.rst +++ b/docs/jjb/lf-ci-jobs.rst @@ -5,52 +5,40 @@ CI Jobs Job Groups ========== -{project-name}-ci-jobs ----------------------- +.. include:: ../job-groups.rst -Recommended jobs that should be deployed for CI using Gerrit. +Below is a list of CI job groups: -:Includes: - - - gerrit-jenkins-cfg-merge - - gerrit-jjb-deploy-job - - gerrit-jjb-merge - - gerrit-jjb-verify - -{project-name}-github-ci-jobs ------------------------------ - -Recommended jobs that should be deployed CI using GitHub. +.. literalinclude:: ../../jjb/lf-ci-job-groups.yaml + :language: yaml -:Includes: - - github-jenkins-cfg-merge - - github-jjb-deploy-job - - github-jjb-merge - - github-jjb-verify +Macros +====== -{project-name}-packer-jobs --------------------------- +lf-infra-jjb-parameters +----------------------- -Jobs related to Packer builds for CI using Gerrit. +:Required Parameters: -:Includes: + :jjb-version: Version of Jenkins Job Builder (JJB) to install and use in + the jjb jobs. - - gerrit-packer-merge - - gerrit-packer-verify +lf-jenkins-cfg-clouds +--------------------- -{project-name}-github-packer-jobs ---------------------------------- +Deploys Jenkins Cloud configuration read from the ``jenkins-clouds`` directory +in ci-management repositories. -Jobs related to Packer builds for CI using GitHub. +.. note:: -:Includes: + Requires the jjbini file in Jenkins CFP to contain JJB 2.0 style + config definitions for "production" and "sandbox" systems. - - github-packer-merge - - github-packer-verify +:Required Parameters: -Macros -====== + :jenkins-silos: Space-separated list of Jenkins silos to update + configuration for as defined in ~/.config/jenkins_jobs/jenkins_jobs.ini lf-jenkins-cfg-global-vars -------------------------- @@ -66,7 +54,7 @@ ci-management/jenkins-config/global-vars-SILO.sh script. :Required parameters: - :jenkins-silos: Space separated list of Jenkins silos to update + :jenkins-silos: Space-separated list of Jenkins silos to update configuration for as defined in ~/.config/jenkins_jobs/jenkins_jobs.ini lf-infra-jjbini @@ -81,20 +69,30 @@ Provides jenkins_jobs.ini configuration for Jenkins sandbox. .. todo:: This needs to be consolidated into lf-infra-jjbini when JJB 2.0 is available -lf-packer-verify-file-paths ---------------------------- +lf-packer-common +---------------- -Gerrit file-paths for packer verify jobs. +Common packer configuration. lf-packer-file-paths -------------------- Gerrit file-paths for packer jobs. -lf-packer-common ----------------- +lf-packer-parameters +-------------------- -Common packer configuration. +Parameters useful for packer related tasks. + +:Parameters: + + :packer-version: Version of packer to install / use. + (shell: PACKER_VERSION) + +lf-packer-verify-file-paths +--------------------------- + +Gerrit file-paths for packer verify jobs. Job Templates ============= @@ -150,6 +148,8 @@ Full Example: .. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-merge-full.yaml :language: yaml +.. _jenkins-cfg-envvar: + Global Environment Variables ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -177,8 +177,8 @@ currently the only cloud plugin supported. OpenStack Cloud plugin version supported: -* 2.30 -* 2.31 +* 2.30 - 2.34 +* 2.35 - 2.37 Cloud configuration are managed via a directory structure in ci-management as follows: @@ -217,9 +217,10 @@ configuration in the format ``KEY=value``. the one defined in default clouds will be inherited. :IMAGE_NAME: The image name to use for this template. - (default: "") + (required) + :HARDWARE_ID: OpenStack flavor to use. (required) + :LABELS: Labels to assign to the vm. (default: FILE_NAME) - :HARDWARE_ID: OpenStack flavor to use. (default: "") :NETWORK_ID: OpenStack network to use. (default: "") :USER_DATA_ID: User Data to pass into the instance. (default: jenkins-init-script) @@ -244,6 +245,8 @@ configuration in the format ``KEY=value``. :RETENTION_TIME: Number of minutes to wait for an idle slave to be used again before it's removed. If set to -1, the slave will be kept forever. (default: 0) + :CONNECTION_TYPE: The connection type for Jenkins to connect to the build + minion. Valid options: JNLP, SSH. (default: "SSH") For a live example see the OpenDaylight project jenkins-config directory. https://github.com/opendaylight/releng-builder/tree/master/jenkins-config @@ -258,6 +261,30 @@ Troubleshooting inspected. +Jenkins Sandbox Cleanup +----------------------- + +Cleanup Jenkins Sandbox of jobs and views periodically. + +:Template names: + + - {project-name}-jenkins-sandbox-cleanup + - gerrit-jenkins-sandbox-cleanup + - github-jenkins-sandbox-cleanup + +:Comment Trigger: NONE + +:Required parameters: + + :build-node: The node to run build on. + :jenkins-ssh-credential: Credential to use for SSH. (Generally + should be configured in defaults.yaml) + +:Optional parameters: + + :cron: Schedule to run job. (default: '0 8 * * 6') + + JJB Deploy Job -------------- @@ -302,20 +329,21 @@ Runs `jenkins-jobs update` to update production job configuration - gerrit-jjb-merge - github-jjb-merge +:Comment Trigger: remerge + :Required parameters: :build-node: The node to run build on. :jenkins-ssh-credential: Credential to use for SSH. (Generally should be configured in defaults.yaml) - :mvn-settings: The name of settings file containing credentials for - the project. :Optional parameters: :branch: Git branch to fetch for the build. (default: master) :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7) - :build-timeout: Timeout in seconds before aborting build. (default: 10) + :build-timeout: Timeout in minutes before aborting build. (default: 10) :git-url: URL clone project from. (default: $GIT_URL/$PROJECT) + :jjb-version: JJB version to install. (default: see job-template) :stream: Keyword that can be used to represent a release code-name. Often the same as the branch. (default: master) :submodule-recursive: Whether to checkout submodules recursively. @@ -337,20 +365,21 @@ Runs `jenkins-jobs test` to validate JJB syntax - gerrit-jjb-verify - github-jjb-verify +:Comment Trigger: recheck|reverify + :Required parameters: :build-node: The node to run build on. :jenkins-ssh-credential: Credential to use for SSH. (Generally should be configured in defaults.yaml) - :mvn-settings: The name of settings file containing credentials for - the project. :Optional parameters: :branch: Git branch to fetch for the build. (default: master) :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7) - :build-timeout: Timeout in seconds before aborting build. (default: 10) + :build-timeout: Timeout in minutes before aborting build. (default: 10) :git-url: URL clone project from. (default: $GIT_URL/$PROJECT) + :jjb-version: JJB version to install. (default: see job-template) :stream: Keyword that can be used to represent a release code-name. Often the same as the branch. (default: master) :submodule-recursive: Whether to checkout submodules recursively. @@ -361,6 +390,136 @@ Runs `jenkins-jobs test` to validate JJB syntax filter which file modifications will trigger a build. (default defined by lf_jjb_common) +.. _jjb-verify-upstream-gjjb: + +JJB Verify Upstream Global JJB +------------------------------ + +Runs ``jenkins-jobs test`` to validate JJB syntax for upstream global-jjb +patches. This job is useful to notify upstream that they may be breaking +project level jobs. + +:Template Names: + - {project-name}-jjb-verify-upstream-gjjb + - gerrit-jjb-verify-upstream-gjjb + +:Comment Trigger: recheck|reverify + +:Required parameters: + + :build-node: The node to run build on. + :jenkins-ssh-credential: Credential to use for SSH. (Generally should + be configured in defaults.yaml) + +:Optional parameters: + + :branch: Git branch to fetch for the build. (default: master) + :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7) + :build-timeout: Timeout in minutes before aborting build. (default: 10) + :git-url: URL clone project from. (default: $GIT_URL/$PROJECT) + :jjb-version: JJB version to install. (default: see job-template) + :stream: Keyword that can be used to represent a release code-name. + Often the same as the branch. (default: master) + +.. _info-yaml-verify: + +Info YAML Verify +---------------- + +Info YAML Verify job validates that INFO.yaml file changes are kept isolated from +other file changes. Verifies INFO.yaml files follow the schema defined in +`global-jjb/info-schema`. + +:Template Names: + - {project-name}-info-yaml-verify + - gerrit-info-yaml-verify + - github-info-yaml-verify + +:Required parameters: + + :build-node: The node to run build on. + :jenkins-ssh-credential: Credential to use for SSH. (Generally should + be configured in defaults.yaml) + +:Optional parameters: + + :branch: Git branch to fetch for the build. (default: master) + :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7) + :build-timeout: Timeout in minutes before aborting build. (default: 10) + :git-url: URL clone project from. (default: $GIT_URL/$PROJECT) + :stream: Keyword that can be used to represent a release code-name. + Often the same as the branch. (default: master) + :submodule-recursive: Whether to checkout submodules recursively. + (default: true) + :gerrit_verify_triggers: Override Gerrit Triggers. + +.. _license-checker: + +License Checker +--------------- + +Job to scan projects for files missing license headers. + +:Template Names: + - {project-name}-license-check + - gerrit-license-check + - github-license-check + +:Optional parameters: + + :build-timeout: Timeout in minutes before aborting build. (default: 15) + :file-patterns: Space-separated list of file patterns to scan. + (default: \*.go \*.groovy \*.java \*.py \*.sh) + :spdx-disable: Disable the SPDX-Identifier checker. (default: false) + :lhc-version: Version of LHC to use. (default: 0.2.0) + :license-exclude-paths: Comma-separated list of paths to exclude from the + license checker. The paths used here will be matched using a contains + rule so it is best to be as precise with the path as possible. + For example a path of '/src/generated/' will be searched as + '**/src/generated/**'. + Example: org/opendaylight/yang/gen,protobuff/messages + (default: '') + :licenses-allowed: Comma-separated list of allowed licenses. + (default: Apache-2.0,EPL-1.0,MIT) + :project-pattern: The ANT based pattern for Gerrit Trigger to choose which + projects to trigger job against. (default: '**') + +.. _gjjb-openstack-cron: + +OpenStack Cron +-------------- + +Cron job that runs regularly to perform periodic tasks against OpenStack. + +This job requires a Config File Provider file named ``clouds-yaml`` available +containing the credentials for the cloud. + +:Template Names: + - {project-name}-openstack-cron + - gerrit-openstack-cron + - github-openstack-cron + +:Required parameters: + + :build-node: The node to run build on. + :jenkins-ssh-credential: Credential to use for SSH. (Generally should + be configured in defaults.yaml) + +:Optional parameters: + + :branch: Git branch to fetch for the build. (default: master) + :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7) + :build-timeout: Timeout in minutes before aborting build. (default: 90) + :cron: Time when the packer image should be rebuilt (default: @daily) + :git-url: URL clone project from. (default: $GIT_URL/$PROJECT) + :openstack-cloud: OS_CLOUD setting to pass to openstack client. + (default: vex) + :stream: Keyword that can be used to represent a release code-name. + Often the same as the branch. (default: master) + :submodule-recursive: Whether to checkout submodules recursively. + (default: true) + +.. _gjjb-packer-merge: Packer Merge ------------ @@ -372,6 +531,8 @@ Packer Merge job runs `packer build` to build system images in the cloud. - gerrit-packer-merge - github-packer-merge +:Comment Trigger: remerge + :Required parameters: :build-node: The node to run build on. @@ -386,10 +547,14 @@ Packer Merge job runs `packer build` to build system images in the cloud. :Optional parameters: + :cron: Time when the packer image should be rebuilt (default: @monthly) :branch: Git branch to fetch for the build. (default: master) :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7) - :build-timeout: Timeout in seconds before aborting build. (default: 10) + :build-timeout: Timeout in minutes before aborting build. (default: 90) :git-url: URL clone project from. (default: $GIT_URL/$PROJECT) + :openstack: Packer template uses an OpenStack builder (default: true). + :openstack-cloud: Sets OS_CLOUD variable to the value of this parameter. + (default: vex). :packer-cloud-settings: Name of settings file containing credentials for the cloud that packer will build on. (default: packer-cloud-env) :packer-version: Version of packer to install / use in build. (default: 1.0.2) @@ -401,6 +566,8 @@ Packer Merge job runs `packer build` to build system images in the cloud. :gerrit_verify_triggers: Override Gerrit Triggers. +.. _gjjb-packer-verify: + Packer Verify ------------- @@ -411,6 +578,8 @@ Packer Verify job runs `packer validate` to verify packer configuration. - gerrit-packer-verify - github-packer-verify +:Comment Trigger: recheck|reverify + :Required parameters: :build-node: The node to run build on. @@ -423,7 +592,7 @@ Packer Verify job runs `packer validate` to verify packer configuration. :branch: Git branch to fetch for the build. (default: master) :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7) - :build-timeout: Timeout in seconds before aborting build. (default: 10) + :build-timeout: Timeout in minutes before aborting build. (default: 10) :git-url: URL clone project from. (default: $GIT_URL/$PROJECT) :packer-cloud-settings: Name of settings file containing credentials for the cloud that packer will build on. (default: packer-cloud-env)