8 .. include:: ../job-groups.rst
10 Below is a list of CI job groups:
12 .. literalinclude:: ../../jjb/lf-ci-job-groups.yaml
19 lf-infra-jjb-parameters
20 -----------------------
24 :jjb-cache: Location of Jenkins Job Builder (JJB) cache used for jjb
26 :jjb-version: Version of Jenkins Job Builder (JJB) to install and use in
32 Deploys Jenkins Cloud configuration read from the ``jenkins-clouds`` directory
33 in ci-management repositories.
37 Requires the jjbini file in Jenkins CFP to contain JJB 2.0 style
38 config definitions for "production" and "sandbox" systems.
42 :jenkins-silos: Space-separated list of Jenkins silos to update
43 configuration for as defined in ~/.config/jenkins_jobs/jenkins_jobs.ini
45 lf-jenkins-cfg-global-vars
46 --------------------------
48 Manages the Global Jenkins variables. This macro will clear all exist macros
49 in Jenkins and replaces them with the ones defined by the
50 ci-management/jenkins-config/global-vars-SILO.sh script.
54 Requires the jjbini file in Jenkins CFP to contain JJB 2.0 style
55 config definitions for "production" and "sandbox" systems.
59 :jenkins-silos: Space-separated list of Jenkins silos to update
60 configuration for as defined in ~/.config/jenkins_jobs/jenkins_jobs.ini
65 Provides jenkins_jobs.ini configuration for Jenkins.
67 lf-infra-jjbini-sandbox
68 -----------------------
70 Provides jenkins_jobs.ini configuration for Jenkins sandbox.
72 .. todo:: This needs to be consolidated into lf-infra-jjbini when JJB 2.0 is available
77 Common packer configuration.
82 Gerrit file-paths for packer jobs.
87 Parameters useful for packer related tasks.
91 :packer-version: Version of packer to install / use.
92 (shell: PACKER_VERSION)
94 lf-packer-verify-file-paths
95 ---------------------------
97 Gerrit file-paths for packer verify jobs.
102 Parameters useful for Puppet related tasks.
106 :puppet-lint-version: Version of puppet-lint to install / use.
107 (shell: PUPPET_LINT_VERSION)
112 .. _gerrit-branch-lock:
117 Job submits a patch to lock or unlock a project's branch.
120 - {project-name}-gerrit-branch-lock-{stream}
128 :Required parameters:
130 :build-node: The node to run build on.
131 :jenkins-ssh-credential: Credential to use for SSH. (Generally
132 should be configured in defaults.yaml)
134 :Optional parameters:
136 :branch: Git branch to build against. (default: master)
137 :git-url: URL to clone project from. (default: $GIT_URL/$GERRIT_PROJECT)
138 :stream: Keyword that can be used to represent a release code-name.
139 Often the same as the branch. (default: master)
140 :submodule-timeout: Timeout (in minutes) for checkout operation.
142 :gerrit_merge_triggers: Override Gerrit Triggers.
144 .. _lf-global-jjb-jenkins-cfg-merge:
146 Jenkins Configuration Merge
147 ---------------------------
149 Jenkins job to manage Global Jenkins configuration.
153 Requires the jjbini file in Jenkins CFP to contain JJB 2.0 style
154 config definitions for "production" and "sandbox" systems.
158 - {project-name}-jenkins-cfg-merge
159 - gerrit-jenkins-cfg-merge
160 - github-jenkins-cfg-merge
162 :Optional parameters:
164 :branch: Git branch to build against. (default: master)
165 :cron: How often to run the job on a cron schedule. (default: @daily)
166 :git-url: URL to clone project from. (default: $GIT_URL/$GERRIT_PROJECT)
167 :jenkins-silos: Space separated list of Jenkins silos to update
168 configuration for as defined in ~/.config/jenkins_jobs/jenkins_jobs.ini
169 (default: production sandbox)
171 Typically this template is automatically pulled in by the
172 "{project-name}-ci-jobs" job-group and does not need to be explicitly called if
173 the job group is being used.
177 .. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-merge-minimal.yaml
182 .. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-merge-full.yaml
185 .. _jenkins-cfg-envvar:
187 Global Environment Variables
188 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
190 Global Environment Variables are managed via the
191 ``jenkins-config/global-vars-SILO.sh`` file in ci-management. Replace SILO with
192 the name of the Jenkins silo the variable configuration is for.
194 The format for this file is ``KEY=value`` for example::
196 GERRIT_URL=https://git.opendaylight.org/gerrit
197 GIT_BASE=git://devvexx.opendaylight.org/mirror/$PROJECT
198 GIT_URL=git://devvexx.opendaylight.org/mirror
199 JENKINS_HOSTNAME=vex-yul-odl-jenkins-2
200 LOGS_SERVER=https://logs.opendaylight.org
201 NEXUS_URL=https://nexus.opendaylight.org
202 ODLNEXUSPROXY=https://nexus.opendaylight.org
204 SONAR_URL=https://sonar.opendaylight.org
209 This configuration requires the OpenStack Cloud plugin in Jenkins and is
210 currently the only cloud plugin supported.
212 OpenStack Cloud plugin version supported:
217 Cloud configuration are managed via a directory structure in ci-management as
220 - jenkins-config/clouds/openstack/
221 - jenkins-config/clouds/openstack/cattle/cloud.cfg
222 - jenkins-config/clouds/openstack/cattle/centos7-builder-2c-2g.cfg
223 - jenkins-config/clouds/openstack/cattle/centos7-builder-4c-4g.cfg
224 - jenkins-config/clouds/openstack/cattle/centos7-docker-4c-4g.cfg
226 The directory name inside of the "openstack" directory is used as the name of
227 the cloud configuration. In this case "cattle" is being used as the cloud name.
229 The ``cloud.cfg`` file is a special file used to configure the main cloud
230 configuration in the format ``KEY=value``.
234 :CLOUD_URL: API endpoint URL for Keystone.
236 :CLOUD_IGNORE_SSL: Ignore unverified SSL certificates. (default: false)
237 :CLOUD_ZONE: OpenStack region to use. (default: "")
238 :CLOUD_CREDENTIAL_ID: Credential to use for authentication to OpenStack.
239 (default: "os-cloud")
240 :INSTANCE_CAP: Total number of instances the cloud will allow spin up.
242 :SANDBOX_CAP: Total number of instances the cloud will allow to
243 spin up. This applies to "sandbox" systems and overrides the
244 INSTANCE_CAP setting. (default: null)
246 :Template Parameters:
250 In the case of template definitions of a parameter below is not passed
251 the one defined in default clouds will be inherited.
253 :IMAGE_NAME: The image name to use for this template.
255 :HARDWARE_ID: OpenStack flavor to use. (required)
257 :LABELS: Labels to assign to the vm. (default: FILE_NAME)
258 :NETWORK_ID: OpenStack network to use. (default: "")
259 :USER_DATA_ID: User Data to pass into the instance.
260 (default: jenkins-init-script)
261 :INSTANCE_CAP: Total number of instances of this type that can be launched
262 at one time. When defined in clouds.cfg it defines the total for the
263 entire cloud. (default: null)
264 :SANDBOX_CAP: Total number of instances of this type that can be launched
265 at one time. When defined in clouds.cfg it defines the total for the
266 entire cloud. This applies to "sandbox" systems and overrides the
267 INSTANCE_CAP setting. (default: null)
268 :FLOATING_IP_POOL: Floating ip pool to use. (default: "")
269 :SECURITY_GROUPS: Security group to use. (default: "default")
270 :AVAILABILITY_ZONE: OpenStack availability zone to use. (default: "")
271 :START_TIMEOUT: Number of milliseconds to wait for the agent to be
272 provisioned and connected. (default: 600000)
273 :KEY_PAIR_NAME: SSH Public Key Pair to use for authentication.
275 :NUM_EXECUTORS: Number of executors to enable for the instance.
277 :JVM_OPTIONS: JVM Options to pass to Java. (default: "")
278 :FS_ROOT: File system root for the workspace. (default: "/w")
279 :RETENTION_TIME: Number of minutes to wait for an idle slave to be used
280 again before it's removed. If set to -1, the slave will be kept
281 forever. (default: 0)
282 :CONNECTION_TYPE: The connection type for Jenkins to connect to the build
283 minion. Valid options: JNLP, SSH. (default: "SSH")
285 For a live example see the OpenDaylight project jenkins-config directory.
286 https://github.com/opendaylight/releng-builder/tree/master/jenkins-config
291 :Cloud Configuration:
293 The directory ``groovy-inserts`` contains the groovy script output that is
294 used to push to Jenkins. In the event of a job failure this file can be
297 .. _lf-global-jjb-jenkins-cfg-verify:
299 Jenkins Configuration Verify
300 ----------------------------
302 Jenkins job to verify the Global Jenkins configuration.
304 Requires the ``clouds-yaml`` file to be setup on the Jenkins host.
308 - {project-name}-jenkins-cfg-verify
309 - gerrit-jenkins-cfg-verify
310 - github-jenkins-cfg-verify
312 :Optional parameters:
314 :branch: Git branch to build against. (default: master)
315 :git-url: URL to clone project from. (default: $GIT_URL/$GERRIT_PROJECT)
317 This job is not part of the "{project-name}-ci-jobs" group. It must be called
322 .. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-verify.yaml
326 .. _jenkins-sandbox-cleanup:
328 Jenkins Sandbox Cleanup
329 -----------------------
331 Cleanup Jenkins Sandbox of jobs and views periodically.
335 - {project-name}-jenkins-sandbox-cleanup
336 - gerrit-jenkins-sandbox-cleanup
337 - github-jenkins-sandbox-cleanup
339 :Comment Trigger: NONE
341 :Required parameters:
343 :build-node: The node to run build on.
344 :jenkins-ssh-credential: Credential to use for SSH. (Generally
345 should be configured in defaults.yaml)
347 :Optional parameters:
349 :cron: Schedule to run job. (default: '0 8 * * 6')
357 Deploy jobs to jenkins-sandbox system via code review comment.
359 This job checks out the current code review patch and then runs a
360 ``jenkins-jobs update`` to push a patch defined by the comment.
364 - {project-name}-jjb-deploy-job
365 - gerrit-jjb-deploy-job
366 - github-jjb-deploy-job
368 :Comment Trigger: jjb-deploy JOB_NAME
372 The JJB Deploy Job is configured to trigger only if the Gerrit comment
373 starts with the `jjb-deploy` keyword.
375 Example of a valid command in Gerrit comment that triggers the job:
377 ``jjb-deploy builder-jjb-*``
379 Example of a invalid command in Gerrit comment that would _not_ trigger
382 ``Update the job. jjb-deploy builder-jjb-*``
384 JOB_NAME can include the * wildcard character to push multiple jobs
385 matching the pattern. For example ``jjb-deploy builder-jjb-*`` will push
386 all builder-jjb-* jobs to the sandbox system.
388 :Required parameters:
390 :build-node: The node to run build on.
391 :jenkins-ssh-credential: Credential to use for SSH. (Generally
392 should be configured in defaults.yaml)
394 :Optional parameters:
396 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
397 :gerrit_jjb_deploy_job_triggers: Override Gerrit Triggers.
405 Runs `jenkins-jobs update` to update production job configuration
408 - {project-name}-jjb-merge
412 :Comment Trigger: remerge
414 :Required parameters:
416 :build-node: The node to run build on.
417 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
418 be configured in defaults.yaml)
420 :Optional parameters:
422 :branch: Git branch to fetch for the build. (default: master)
423 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
424 :build-timeout: Timeout in minutes before aborting build. (default: 10)
425 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
426 :jjb-cache: JJB cache location. (default: $HOME/.cache/jenkins_jobs)
427 :jjb-version: JJB version to install. (default: see job-template)
428 :stream: Keyword that can be used to represent a release code-name.
429 Often the same as the branch. (default: master)
430 :submodule-recursive: Whether to checkout submodules recursively.
432 :submodule-timeout: Timeout (in minutes) for checkout operation.
435 :gerrit_merge_triggers: Override Gerrit Triggers.
436 :gerrit_trigger_file_paths: Override file paths which can be used to
437 filter which file modifications will trigger a build.
438 (default defined by lf_jjb_common)
446 Runs `jenkins-jobs test` to validate JJB syntax
449 - {project-name}-jjb-verify
453 :Comment Trigger: recheck|reverify
455 :Required parameters:
457 :build-node: The node to run build on.
458 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
459 be configured in defaults.yaml)
461 :Optional parameters:
463 :branch: Git branch to fetch for the build. (default: master)
464 :build-concurrent: Whether or not to allow this job to run multiple jobs
465 simultaneously. (default: true)
466 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
467 :build-timeout: Timeout in minutes before aborting build. (default: 10)
468 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
469 :jjb-cache: JJB cache location. (default: $HOME/.cache/jenkins_jobs)
470 :jjb-version: JJB version to install. (default: see job-template)
471 :stream: Keyword that can be used to represent a release code-name.
472 Often the same as the branch. (default: master)
473 :submodule-recursive: Whether to checkout submodules recursively.
475 :submodule-timeout: Timeout (in minutes) for checkout operation.
477 :throttle_categories: List of categories to throttle by.
478 :throttle-enabled: Whether or not to enable throttling on the job.
480 :throttle-max-per-node: Max jobs to run on the same node. (default: 1)
481 :throttle-max-total: Max jobs to run across the entire project. - 0
482 means 'unlimited' (default: 0)
483 :throttle-option: Throttle by the project or by list of categories
484 defined in the throttle plugin configuration. (options: 'project',
485 'category'; default: project)
487 :gerrit_verify_triggers: Override Gerrit Triggers.
488 :gerrit_trigger_file_paths: Override file paths which can be used to
489 filter which file modifications will trigger a build.
490 (default defined by lf_jjb_common)
492 .. _jjb-verify-upstream-gjjb:
494 JJB Verify Upstream Global JJB
495 ------------------------------
497 Runs ``jenkins-jobs test`` to validate JJB syntax for upstream global-jjb
498 patches. This job is useful to notify upstream that they may be breaking
502 - {project-name}-jjb-verify-upstream-gjjb
503 - gerrit-jjb-verify-upstream-gjjb
505 :Comment Trigger: recheck|reverify
507 :Required parameters:
509 :build-node: The node to run build on.
510 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
511 be configured in defaults.yaml)
513 :Optional parameters:
515 :branch: Git branch to fetch for the build. (default: master)
516 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
517 :build-timeout: Timeout in minutes before aborting build. (default: 10)
518 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
519 :jjb-cache: JJB cache location. (default: $HOME/.cache/jenkins_jobs)
520 :jjb-version: JJB version to install. (default: see job-template)
521 :stream: Keyword that can be used to represent a release code-name.
522 Often the same as the branch. (default: master)
524 .. _info-yaml-verify:
529 Info YAML Verify job validates that INFO.yaml file changes are kept isolated from
530 other file changes. Verifies INFO.yaml files follow the schema defined in
531 `global-jjb/info-schema`.
534 - {project-name}-info-yaml-verify
535 - gerrit-info-yaml-verify
536 - github-info-yaml-verify
538 :Required parameters:
540 :build-node: The node to run build on.
541 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
542 be configured in defaults.yaml)
544 :Optional parameters:
546 :branch: Git branch to fetch for the build. (default: master)
547 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
548 :build-timeout: Timeout in minutes before aborting build. (default: 10)
549 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
550 :stream: Keyword that can be used to represent a release code-name.
551 Often the same as the branch. (default: master)
552 :submodule-recursive: Whether to checkout submodules recursively.
554 :submodule-timeout: Timeout (in minutes) for checkout operation.
557 :gerrit_verify_triggers: Override Gerrit Triggers.
564 Job to scan projects for files missing license headers.
567 - {project-name}-license-check
568 - gerrit-license-check
569 - github-license-check
571 :Optional parameters:
573 :build-timeout: Timeout in minutes before aborting build. (default: 15)
574 :file-patterns: Space-separated list of file patterns to scan.
575 (default: \*.go \*.groovy \*.java \*.py \*.sh)
576 :spdx-disable: Disable the SPDX-Identifier checker. (default: false)
577 :lhc-version: Version of LHC to use. (default: 0.2.0)
578 :license-exclude-paths: Comma-separated list of paths to exclude from the
579 license checker. The paths used here will be matched using a contains
580 rule so it is best to be as precise with the path as possible.
581 For example a path of '/src/generated/' will be searched as
582 '**/src/generated/**'.
583 Example: org/opendaylight/yang/gen,protobuff/messages
585 :licenses-allowed: Comma-separated list of allowed licenses.
586 (default: Apache-2.0,EPL-1.0,MIT)
587 :project-pattern: The ANT based pattern for Gerrit Trigger to choose which
588 projects to trigger job against. (default: '**')
590 .. _gjjb-openstack-cron:
595 Cron job that runs regularly to perform periodic tasks against OpenStack.
597 This job requires a Config File Provider file named ``clouds-yaml`` available
598 containing the credentials for the cloud.
601 - {project-name}-openstack-cron
602 - gerrit-openstack-cron
603 - github-openstack-cron
605 :Required parameters:
607 :build-node: The node to run build on.
608 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
609 be configured in defaults.yaml)
610 :jenkins-urls: URLs to Jenkins systems to check for active builds.
612 :Optional parameters:
614 :branch: Git branch to fetch for the build. (default: master)
615 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
616 :build-timeout: Timeout in minutes before aborting build. (default: 90)
617 :cron: Time when the packer image should be rebuilt (default: @hourly)
618 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
619 :openstack-cloud: OS_CLOUD setting to pass to openstack client.
621 :openstack-image-cleanup: Whether or not to run the image cleanup script.
623 :openstack-image-cleanup-age: Age in days of image before marking it for
624 removal. (default: 30)
625 :openstack-image-protect: Whether or not to run the image protect script.
627 :openstack-server-cleanup: Whether or not to run the server cleanup script.
629 :openstack-stack-cleanup: Whether or not to run the stack cleanup script.
631 :openstack-volume-cleanup: Whether or not to run the volume cleanup script.
633 :stream: Keyword that can be used to represent a release code-name.
634 Often the same as the branch. (default: master)
635 :submodule-recursive: Whether to checkout submodules recursively.
637 :submodule-timeout: Timeout (in minutes) for checkout operation.
642 .. literalinclude:: ../../.jjb-test/lf-ci-jobs/openstack-cron-minimal.yaml
646 .. literalinclude:: ../../.jjb-test/lf-ci-jobs/openstack-cron-full.yaml
649 .. _gjjb-packer-merge:
654 Packer Merge job runs `packer build` to build system images in the cloud.
657 - {project-name}-packer-merge-{platforms}-{templates}
658 - gerrit-packer-merge
659 - github-packer-merge
661 :Comment Trigger: remerge
663 :Required parameters:
665 :build-node: The node to run build on.
666 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
667 be configured in defaults.yaml)
668 :mvn-settings: The name of settings file containing credentials for
670 :platforms: Platform or distribution to build. Typically json file
671 found in the packer/vars directory. (Example: centos)
672 :template: System template to build. Typically shell script found in
673 the packer/provision directory. (Example: java-builder)
675 :Optional parameters:
677 :cron: Time when the packer image should be rebuilt (default: @monthly)
678 :branch: Git branch to fetch for the build. (default: master)
679 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
680 :build-timeout: Timeout in minutes before aborting build. (default: 90)
681 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
682 :openstack: Packer template uses an OpenStack builder (default: true).
683 :openstack-cloud: Sets OS_CLOUD variable to the value of this parameter.
685 :packer-cloud-settings: Name of settings file containing credentials
686 for the cloud that packer will build on. (default: packer-cloud-env)
687 :packer-version: Version of packer to install / use in build. (default: 1.0.2)
688 :stream: Keyword that can be used to represent a release code-name.
689 Often the same as the branch. (default: master)
690 :submodule-recursive: Whether to checkout submodules recursively.
692 :submodule-timeout: Timeout (in minutes) for checkout operation.
695 :gerrit_verify_triggers: Override Gerrit Triggers.
697 Test an in-progress patch
698 ^^^^^^^^^^^^^^^^^^^^^^^^^
700 To test an in-progress patch from a GitHub Pull Request. Upload this
701 job to the :doc:`Jenkins Sandbox <lfdocs:jenkins-sandbox>`. Then when manually
702 building the job replace the GERRIT_REFSPEC parameter with the GitHub Pull
703 Request number of the patch you would like to test.
709 GERRIT_REFSPEC: origin/pr/49/merge
711 .. _gjjb-packer-verify:
716 Packer Verify job runs `packer validate` to verify packer configuration.
719 - {project-name}-packer-verify
720 - gerrit-packer-verify
721 - github-packer-verify
723 :Comment Trigger: recheck|reverify
725 :Required parameters:
727 :build-node: The node to run build on.
728 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
729 be configured in defaults.yaml)
730 :mvn-settings: The name of settings file containing credentials for
733 :Optional parameters:
735 :branch: Git branch to fetch for the build. (default: master)
736 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
737 :build-timeout: Timeout in minutes before aborting build. (default: 10)
738 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
739 :openstack: Packer template uses an OpenStack builder (default: true).
740 :openstack-cloud: Sets OS_CLOUD variable to the value of this parameter.
742 :packer-cloud-settings: Name of settings file containing credentials
743 for the cloud that packer will build on. (default: packer-cloud-env)
744 :packer-version: Version of packer to install / use in build. (default: 1.0.2)
745 :stream: Keyword that can be used to represent a release code-name.
746 Often the same as the branch. (default: master)
747 :submodule-recursive: Whether to checkout submodules recursively.
749 :submodule-timeout: Timeout (in minutes) for checkout operation.
752 :gerrit_verify_triggers: Override Gerrit Triggers.
753 :gerrit_trigger_file_paths: Override file paths which can be used to
754 filter which file modifications will trigger a build.
760 Runs puppet-lint in the ``puppet-dir`` directory. puppet-lint runs recursively,
761 the base directory is usually the best place to run from.
765 - {project-name}-puppet-verify
766 - gerrit-puppet-verify
767 - github-puppet-verify
769 :Comment Trigger: recheck|reverify
771 :Required Parameters:
773 :build-node: The node to run build on.
774 :jenkins-ssh-credential: Credential to use for SSH. (Generally set
777 :Optional Parameters:
779 :branch: The branch to build against. (default: master)
780 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
781 :build-timeout: Timeout in minutes before aborting build. (default: 15)
782 :gerrit_trigger_file_paths: Override file paths which used to filter which
783 file modifications will trigger a build. Refer to JJB documentation for
785 https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit
786 :git-url: URL clone project from. (default: $GIT_URL/$GERRIT_PROJECT)
787 :puppet-dir: Directory containing the project's puppet module(s) relative
790 :puppet-lint-version: Version of puppet-lint to use for testing.
792 :stream: Keyword representing a release code-name.
793 Often the same as the branch. (default: master)
794 :submodule-recursive: Whether to checkout submodules recursively.
796 :submodule-timeout: Timeout (in minutes) for checkout operation.