X-Git-Url: https://gerrit.linuxfoundation.org/infra/gitweb?a=blobdiff_plain;f=docs%2Fjjb%2Flf-ci-jobs.rst;h=93ef06748935fa8a407531c1a883c19fc21bcdd5;hb=41c32874d920b94e23fc36c1432a5950ee34cd82;hp=49fb4678b56e860516a1717a75211e1e547a4fb2;hpb=2069fa3b9c656c59fc93c77bf0ba80a18c0ddc00;p=releng%2Fglobal-jjb.git diff --git a/docs/jjb/lf-ci-jobs.rst b/docs/jjb/lf-ci-jobs.rst index 49fb4678..93ef0674 100644 --- a/docs/jjb/lf-ci-jobs.rst +++ b/docs/jjb/lf-ci-jobs.rst @@ -12,6 +12,7 @@ Recommended jobs that should be deployed for CI using Gerrit. :Includes: + - gerrit-jenkins-cfg-merge - gerrit-jjb-deploy-job - gerrit-jjb-merge - gerrit-jjb-verify @@ -23,10 +24,29 @@ Recommended jobs that should be deployed CI using GitHub. :Includes: + - github-jenkins-cfg-merge - github-jjb-deploy-job - github-jjb-merge - github-jjb-verify +{project-name}-info-yaml-jobs +----------------------------- + +Jobs to verify INFO.yaml file changes. + +:Includes: + + - gerrit-info-yaml-verify + +{project-name}-github-info-yaml-jobs +------------------------------------ + +Jobs to verify INFO.yaml file changes using Github. + +:Includes: + + - github-info-yaml-verify + {project-name}-packer-jobs -------------------------- @@ -50,6 +70,47 @@ Jobs related to Packer builds for CI using GitHub. Macros ====== +lf-infra-jjb-parameters +----------------------- + +:Required Parameters: + + :jjb-version: Version of Jenkins Job Builder (JJB) to install and use in + the jjb jobs. + +lf-jenkins-cfg-clouds +--------------------- + +Deploys Jenkins Cloud configuration read from the ``jenkins-clouds`` directory +in ci-management repositories. + +.. note:: + + Requires the jjbini file in Jenkins CFP to contain JJB 2.0 style + config definitions for "production" and "sandbox" systems. + +:Required Parameters: + + :jenkins-silos: Space-separated list of Jenkins silos to update + configuration for as defined in ~/.config/jenkins_jobs/jenkins_jobs.ini + +lf-jenkins-cfg-global-vars +-------------------------- + +Manages the Global Jenkins variables. This macro will clear all exist macros +in Jenkins and replaces them with the ones defined by the +ci-management/jenkins-config/global-vars-SILO.sh script. + +.. note:: + + Requires the jjbini file in Jenkins CFP to contain JJB 2.0 style + config definitions for "production" and "sandbox" systems. + +:Required parameters: + + :jenkins-silos: Space-separated list of Jenkins silos to update + configuration for as defined in ~/.config/jenkins_jobs/jenkins_jobs.ini + lf-infra-jjbini --------------- @@ -62,20 +123,30 @@ Provides jenkins_jobs.ini configuration for Jenkins sandbox. .. todo:: This needs to be consolidated into lf-infra-jjbini when JJB 2.0 is available -lf-packer-verify-file-paths ---------------------------- +lf-packer-common +---------------- -Gerrit file-paths for packer verify jobs. +Common packer configuration. lf-packer-file-paths -------------------- Gerrit file-paths for packer jobs. -lf-packer-common ----------------- +lf-packer-parameters +-------------------- -Common packer configuration. +Parameters useful for packer related tasks. + +:Parameters: + + :packer-version: Version of packer to install / use. + (shell: PACKER_VERSION) + +lf-packer-verify-file-paths +--------------------------- + +Gerrit file-paths for packer verify jobs. Job Templates ============= @@ -90,6 +161,161 @@ Job submits a patch to lock or unlock a project's branch. - gerrit-branch-lock +.. _lf-global-jjb-jenkins-cfg-merge: + +Jenkins Configuration Merge +--------------------------- + +Jenkins job to manage Global Jenkins configuration. + +.. note:: + + Requires the jjbini file in Jenkins CFP to contain JJB 2.0 style + config definitions for "production" and "sandbox" systems. + +:Template names: + + - {project-name}-jenkins-cfg-merge + - gerrit-jenkins-cfg-merge + - github-jenkins-cfg-merge + +:Optional parameters: + + :branch: Git branch to build against. (default: master) + :cron: How often to run the job on a cron schedule. (default: @daily) + :git-url: URL to clone project from. (default: $GIT_URL/$GERRIT_PROJECT) + :jenkins-silos: Space separated list of Jenkins silos to update + configuration for as defined in ~/.config/jenkins_jobs/jenkins_jobs.ini + (default: production sandbox) + +Typically this template is automatically pulled in by the +"{project-name}-ci-jobs" job-group and does not need to be explicitly called if +the job group is being used. + +Miniaml Example: + +.. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-merge-minimal.yaml + :language: yaml + +Full Example: + +.. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-merge-full.yaml + :language: yaml + +.. _jenkins-cfg-envvar: + +Global Environment Variables +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Global Environment Variables are managed via the +``jenkins-config/global-vars-SILO.sh`` file in ci-management. Replace SILO with +the name of the Jenkins silo the variable configuration is for. + +The format for this file is ``KEY=value`` for example:: + + GERRIT_URL=https://git.opendaylight.org/gerrit + GIT_BASE=git://devvexx.opendaylight.org/mirror/$PROJECT + GIT_URL=git://devvexx.opendaylight.org/mirror + JENKINS_HOSTNAME=vex-yul-odl-jenkins-2 + LOGS_SERVER=https://logs.opendaylight.org + NEXUS_URL=https://nexus.opendaylight.org + ODLNEXUSPROXY=https://nexus.opendaylight.org + SILO=sandbox + SONAR_URL=https://sonar.opendaylight.org + +Cloud Configuration +^^^^^^^^^^^^^^^^^^^ + +This configuration requires the OpenStack Cloud plugin in Jenkins and is +currently the only cloud plugin supported. + +OpenStack Cloud plugin version supported: + +* 2.30 +* 2.31 +* 2.32 +* 2.33 +* 2.34 +* 2.35 + +Cloud configuration are managed via a directory structure in ci-management as +follows: + +- jenkins-config/clouds/openstack/ +- jenkins-config/clouds/openstack/cattle/cloud.cfg +- jenkins-config/clouds/openstack/cattle/centos7-builder-2c-2g.cfg +- jenkins-config/clouds/openstack/cattle/centos7-builder-4c-4g.cfg +- jenkins-config/clouds/openstack/cattle/centos7-docker-4c-4g.cfg + +The directory name inside of the "openstack" directory is used as the name of +the cloud configuration. In this case "cattle" is being used as the cloud name. + +The ``cloud.cfg`` file is a special file used to configure the main cloud +configuration in the format ``KEY=value``. + +:Cloud Parameters: + + :CLOUD_URL: API endpoint URL for Keystone. + (default: "") + :CLOUD_IGNORE_SSL: Ignore unverified SSL certificates. (default: false) + :CLOUD_ZONE: OpenStack region to use. (default: "") + :CLOUD_CREDENTIAL_ID: Credential to use for authentication to OpenStack. + (default: "os-cloud") + :INSTANCE_CAP: Total number of instances the cloud will allow spin up. + (default: null) + :SANDBOX_CAP: Total number of instances the clodu will allow to + spin up. This applies to "sandbox" systems and overrides the + INSTANCE_CAP setting. (default: null) + +:Template Parameters: + + .. note:: + + In the case of template definitions of a parameter below is not passed + the one defined in default clouds will be inherited. + + :IMAGE_NAME: The image name to use for this template. + (default: "") + :LABELS: Labels to assign to the vm. (default: FILE_NAME) + :HARDWARE_ID: OpenStack flavor to use. (default: "") + :NETWORK_ID: OpenStack network to use. (default: "") + :USER_DATA_ID: User Data to pass into the instance. + (default: jenkins-init-script) + :INSTANCE_CAP: Total number of instances of this type that can be launched + at one time. When defined in clouds.cfg it defines the total for the + entire cloud. (default: null) + :SANDBOX_CAP: Total number of instances of this type that can be launched + at one time. When defined in clouds.cfg it defines the total for the + entire cloud. This applies to "sandbox" systems and overrides the + INSTANCE_CAP setting. (default: null) + :FLOATING_IP_POOL: Floating ip pool to use. (default: "") + :SECURITY_GROUPS: Security group to use. (default: "default") + :AVAILABILITY_ZONE: OpenStack availability zone to use. (default: "") + :START_TIMEOUT: Number of milliseconds to wait for the agent to be + provisioned and connected. (default: 600000) + :KEY_PAIR_NAME: SSH Public Key Pair to use for authentication. + (default: jenkins) + :NUM_EXECUTORS: Number of executors to enable for the instance. + (default: 1) + :JVM_OPTIONS: JVM Options to pass to Java. (default: "") + :FS_ROOT: File system root for the workspace. (default: "/w") + :RETENTION_TIME: Number of minutes to wait for an idle slave to be used + again before it's removed. If set to -1, the slave will be kept + forever. (default: 0) + +For a live example see the OpenDaylight project jenkins-config directory. +https://github.com/opendaylight/releng-builder/tree/master/jenkins-config + +Troubleshooting +^^^^^^^^^^^^^^^ + +:Cloud Configuration: + + The directory ``groovy-inserts`` contains the groovy script output that is + used to push to Jenkins. In the event of a job failure this file can be + inspected. + + JJB Deploy Job -------------- @@ -193,6 +419,67 @@ Runs `jenkins-jobs test` to validate JJB syntax filter which file modifications will trigger a build. (default defined by lf_jjb_common) +.. _info-yaml-verify: + +Info YAML Verify +---------------- + +Info YAML Verify job validates that INFO.yaml file changes are kept isolated from +other file changes. Verifies INFO.yaml files follow the schema defined in +`global-jjb/info-schema`. + +:Template Names: + - {project-name}-info-yaml-verify + - gerrit-info-yaml-verify + - github-info-yaml-verify + +:Required parameters: + + :build-node: The node to run build on. + :jenkins-ssh-credential: Credential to use for SSH. (Generally should + be configured in defaults.yaml) + +:Optional parameters: + + :branch: Git branch to fetch for the build. (default: master) + :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7) + :build-timeout: Timeout in seconds before aborting build. (default: 10) + :git-url: URL clone project from. (default: $GIT_URL/$PROJECT) + :stream: Keyword that can be used to represent a release code-name. + Often the same as the branch. (default: master) + :submodule-recursive: Whether to checkout submodules recursively. + (default: true) + :gerrit_verify_triggers: Override Gerrit Triggers. + +.. _license-checker: + +License Checker +--------------- + +Job to scan projects for files missing license headers. + +:Template Names: + - {project-name}-license-check + - gerrit-license-check + - github-license-check + +:Optional parameters: + + :file-patterns: Space-separated list of file patterns to scan. + (default: \*.go \*.groovy \*.java \*.py \*.sh) + :spdx-disable: Disable the SPDX-Identifier checker. (default: false) + :lhc-version: Version of LHC to use. (default: 0.2.0) + :license-exclude-paths: Comma-separated list of paths to exclude from the + license checker. The paths used here will be matched using a contains + rule so it is best to be as precise with the path as possible. + For example a path of '/src/generated/' will be searched as + '**/src/generated/**'. + Example: org/opendaylight/yang/gen,protobuff/messages + (default: '') + :licenses-allowed: Comma-separated list of allowed licenses. + (default: Apache-2.0,EPL-1.0,MIT) + +.. _gjjb-packer-merge: Packer Merge ------------ @@ -218,6 +505,7 @@ Packer Merge job runs `packer build` to build system images in the cloud. :Optional parameters: + :cron: Time when the packer image should be rebuilt (default: @monthly) :branch: Git branch to fetch for the build. (default: master) :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7) :build-timeout: Timeout in seconds before aborting build. (default: 10) @@ -233,6 +521,8 @@ Packer Merge job runs `packer build` to build system images in the cloud. :gerrit_verify_triggers: Override Gerrit Triggers. +.. _gjjb-packer-verify: + Packer Verify -------------