8 .. include:: ../job-groups.rst
10 Below is a list of CI job groups:
12 .. literalinclude:: ../../jjb/lf-ci-job-groups.yaml
19 lf-infra-jjb-parameters
20 -----------------------
24 :jjb-cache: Location of Jenkins Job Builder (JJB) cache used for jjb
26 :jjb-version: Version of Jenkins Job Builder (JJB) to install and use in
32 Deploys Jenkins Cloud configuration read from the ``jenkins-clouds`` directory
33 in ci-management repositories.
37 Requires the jjbini file in Jenkins CFP to contain JJB 2.0 style
38 config definitions for "production" and "sandbox" systems.
42 :jenkins-silos: Space-separated list of Jenkins silos to update
43 configuration for as defined in ~/.config/jenkins_jobs/jenkins_jobs.ini
45 lf-jenkins-cfg-global-vars
46 --------------------------
48 Manages the Global Jenkins variables. This macro will clear all exist macros
49 in Jenkins and replaces them with the ones defined by the
50 ci-management/jenkins-config/global-vars-SILO.sh script.
54 Requires the jjbini file in Jenkins CFP to contain JJB 2.0 style
55 config definitions for "production" and "sandbox" systems.
59 :jenkins-silos: Space-separated list of Jenkins silos to update
60 configuration for as defined in ~/.config/jenkins_jobs/jenkins_jobs.ini
65 Provides jenkins_jobs.ini configuration for Jenkins.
67 lf-infra-jjbini-sandbox
68 -----------------------
70 Provides jenkins_jobs.ini configuration for Jenkins sandbox.
72 .. todo:: This needs to be consolidated into lf-infra-jjbini when JJB 2.0 is available
77 Common packer configuration.
82 Gerrit file-paths for packer jobs.
87 Parameters useful for packer related tasks.
91 :packer-version: Version of packer to install / use.
92 (shell: PACKER_VERSION)
94 lf-packer-verify-file-paths
95 ---------------------------
97 Gerrit file-paths for packer verify jobs.
102 .. _gerrit-branch-lock:
107 Job submits a patch to lock or unlock a project's branch.
110 - {project-name}-gerrit-branch-lock-{stream}
118 :Required parameters:
120 :build-node: The node to run build on.
121 :jenkins-ssh-credential: Credential to use for SSH. (Generally
122 should be configured in defaults.yaml)
124 :Optional parameters:
126 :branch: Git branch to build against. (default: master)
127 :git-url: URL to clone project from. (default: $GIT_URL/$GERRIT_PROJECT)
128 :stream: Keyword that can be used to represent a release code-name.
129 Often the same as the branch. (default: master)
130 :submodule-timeout: Timeout (in minutes) for checkout operation.
132 :gerrit_merge_triggers: Override Gerrit Triggers.
134 .. _lf-global-jjb-jenkins-cfg-merge:
136 Jenkins Configuration Merge
137 ---------------------------
139 Jenkins job to manage Global Jenkins configuration.
143 Requires the jjbini file in Jenkins CFP to contain JJB 2.0 style
144 config definitions for "production" and "sandbox" systems.
148 - {project-name}-jenkins-cfg-merge
149 - gerrit-jenkins-cfg-merge
150 - github-jenkins-cfg-merge
152 :Optional parameters:
154 :branch: Git branch to build against. (default: master)
155 :cron: How often to run the job on a cron schedule. (default: @daily)
156 :git-url: URL to clone project from. (default: $GIT_URL/$GERRIT_PROJECT)
157 :jenkins-silos: Space separated list of Jenkins silos to update
158 configuration for as defined in ~/.config/jenkins_jobs/jenkins_jobs.ini
159 (default: production sandbox)
161 Typically this template is automatically pulled in by the
162 "{project-name}-ci-jobs" job-group and does not need to be explicitly called if
163 the job group is being used.
167 .. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-merge-minimal.yaml
172 .. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-merge-full.yaml
175 .. _jenkins-cfg-envvar:
177 Global Environment Variables
178 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
180 Global Environment Variables are managed via the
181 ``jenkins-config/global-vars-SILO.sh`` file in ci-management. Replace SILO with
182 the name of the Jenkins silo the variable configuration is for.
184 The format for this file is ``KEY=value`` for example::
186 GERRIT_URL=https://git.opendaylight.org/gerrit
187 GIT_BASE=git://devvexx.opendaylight.org/mirror/$PROJECT
188 GIT_URL=git://devvexx.opendaylight.org/mirror
189 JENKINS_HOSTNAME=vex-yul-odl-jenkins-2
190 LOGS_SERVER=https://logs.opendaylight.org
191 NEXUS_URL=https://nexus.opendaylight.org
192 ODLNEXUSPROXY=https://nexus.opendaylight.org
194 SONAR_URL=https://sonar.opendaylight.org
199 This configuration requires the OpenStack Cloud plugin in Jenkins and is
200 currently the only cloud plugin supported.
202 OpenStack Cloud plugin version supported:
207 Cloud configuration are managed via a directory structure in ci-management as
210 - jenkins-config/clouds/openstack/
211 - jenkins-config/clouds/openstack/cattle/cloud.cfg
212 - jenkins-config/clouds/openstack/cattle/centos7-builder-2c-2g.cfg
213 - jenkins-config/clouds/openstack/cattle/centos7-builder-4c-4g.cfg
214 - jenkins-config/clouds/openstack/cattle/centos7-docker-4c-4g.cfg
216 The directory name inside of the "openstack" directory is used as the name of
217 the cloud configuration. In this case "cattle" is being used as the cloud name.
219 The ``cloud.cfg`` file is a special file used to configure the main cloud
220 configuration in the format ``KEY=value``.
224 :CLOUD_URL: API endpoint URL for Keystone.
226 :CLOUD_IGNORE_SSL: Ignore unverified SSL certificates. (default: false)
227 :CLOUD_ZONE: OpenStack region to use. (default: "")
228 :CLOUD_CREDENTIAL_ID: Credential to use for authentication to OpenStack.
229 (default: "os-cloud")
230 :INSTANCE_CAP: Total number of instances the cloud will allow spin up.
232 :SANDBOX_CAP: Total number of instances the cloud will allow to
233 spin up. This applies to "sandbox" systems and overrides the
234 INSTANCE_CAP setting. (default: null)
236 :Template Parameters:
240 In the case of template definitions of a parameter below is not passed
241 the one defined in default clouds will be inherited.
243 :IMAGE_NAME: The image name to use for this template.
245 :HARDWARE_ID: OpenStack flavor to use. (required)
247 :LABELS: Labels to assign to the vm. (default: FILE_NAME)
248 :NETWORK_ID: OpenStack network to use. (default: "")
249 :USER_DATA_ID: User Data to pass into the instance.
250 (default: jenkins-init-script)
251 :INSTANCE_CAP: Total number of instances of this type that can be launched
252 at one time. When defined in clouds.cfg it defines the total for the
253 entire cloud. (default: null)
254 :SANDBOX_CAP: Total number of instances of this type that can be launched
255 at one time. When defined in clouds.cfg it defines the total for the
256 entire cloud. This applies to "sandbox" systems and overrides the
257 INSTANCE_CAP setting. (default: null)
258 :FLOATING_IP_POOL: Floating ip pool to use. (default: "")
259 :SECURITY_GROUPS: Security group to use. (default: "default")
260 :AVAILABILITY_ZONE: OpenStack availability zone to use. (default: "")
261 :START_TIMEOUT: Number of milliseconds to wait for the agent to be
262 provisioned and connected. (default: 600000)
263 :KEY_PAIR_NAME: SSH Public Key Pair to use for authentication.
265 :NUM_EXECUTORS: Number of executors to enable for the instance.
267 :JVM_OPTIONS: JVM Options to pass to Java. (default: "")
268 :FS_ROOT: File system root for the workspace. (default: "/w")
269 :RETENTION_TIME: Number of minutes to wait for an idle slave to be used
270 again before it's removed. If set to -1, the slave will be kept
271 forever. (default: 0)
272 :CONNECTION_TYPE: The connection type for Jenkins to connect to the build
273 minion. Valid options: JNLP, SSH. (default: "SSH")
275 For a live example see the OpenDaylight project jenkins-config directory.
276 https://github.com/opendaylight/releng-builder/tree/master/jenkins-config
281 :Cloud Configuration:
283 The directory ``groovy-inserts`` contains the groovy script output that is
284 used to push to Jenkins. In the event of a job failure this file can be
287 .. _lf-global-jjb-jenkins-cfg-verify:
289 Jenkins Configuration Verify
290 ----------------------------
292 Jenkins job to verify the Global Jenkins configuration.
294 Requires the ``clouds-yaml`` file to be setup on the Jenkins host.
298 - {project-name}-jenkins-cfg-verify
299 - gerrit-jenkins-cfg-verify
300 - github-jenkins-cfg-verify
302 :Optional parameters:
304 :branch: Git branch to build against. (default: master)
305 :git-url: URL to clone project from. (default: $GIT_URL/$GERRIT_PROJECT)
307 This job is not part of the "{project-name}-ci-jobs" group. It must be called
312 .. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-verify.yaml
315 Jenkins Sandbox Cleanup
316 -----------------------
318 Cleanup Jenkins Sandbox of jobs and views periodically.
322 - {project-name}-jenkins-sandbox-cleanup
323 - gerrit-jenkins-sandbox-cleanup
324 - github-jenkins-sandbox-cleanup
326 :Comment Trigger: NONE
328 :Required parameters:
330 :build-node: The node to run build on.
331 :jenkins-ssh-credential: Credential to use for SSH. (Generally
332 should be configured in defaults.yaml)
334 :Optional parameters:
336 :cron: Schedule to run job. (default: '0 8 * * 6')
342 Deploy jobs to jenkins-sandbox system via code review comment
344 This job checks out the current code review patch and then runs a
345 `jenkins-jobs update` to push a patch defined by the comment.
349 - {project-name}-jjb-deploy-job
350 - gerrit-jjb-deploy-job
351 - github-jjb-deploy-job
353 :Comment Trigger: jjb-deploy JOB_NAME
357 JOB_NAME can include the * wildcard character to push multiple jobs
358 matching the pattern. For example `jjb-deploy builder-jjb-*`` will push
359 all builder-jjb-* jobs to the sandbox system.
361 :Required parameters:
363 :build-node: The node to run build on.
364 :jenkins-ssh-credential: Credential to use for SSH. (Generally
365 should be configured in defaults.yaml)
367 :Optional parameters:
369 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
370 :gerrit_jjb_deploy_job_triggers: Override Gerrit Triggers.
376 Runs `jenkins-jobs update` to update production job configuration
379 - {project-name}-jjb-merge
383 :Comment Trigger: remerge
385 :Required parameters:
387 :build-node: The node to run build on.
388 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
389 be configured in defaults.yaml)
391 :Optional parameters:
393 :branch: Git branch to fetch for the build. (default: master)
394 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
395 :build-timeout: Timeout in minutes before aborting build. (default: 10)
396 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
397 :jjb-cache: JJB cache location. (default: $HOME/.cache/jenkins_jobs)
398 :jjb-version: JJB version to install. (default: see job-template)
399 :stream: Keyword that can be used to represent a release code-name.
400 Often the same as the branch. (default: master)
401 :submodule-recursive: Whether to checkout submodules recursively.
403 :submodule-timeout: Timeout (in minutes) for checkout operation.
406 :gerrit_merge_triggers: Override Gerrit Triggers.
407 :gerrit_trigger_file_paths: Override file paths which can be used to
408 filter which file modifications will trigger a build.
409 (default defined by lf_jjb_common)
415 Runs `jenkins-jobs test` to validate JJB syntax
418 - {project-name}-jjb-verify
422 :Comment Trigger: recheck|reverify
424 :Required parameters:
426 :build-node: The node to run build on.
427 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
428 be configured in defaults.yaml)
430 :Optional parameters:
432 :branch: Git branch to fetch for the build. (default: master)
433 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
434 :build-timeout: Timeout in minutes before aborting build. (default: 10)
435 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
436 :jjb-cache: JJB cache location. (default: $HOME/.cache/jenkins_jobs)
437 :jjb-version: JJB version to install. (default: see job-template)
438 :stream: Keyword that can be used to represent a release code-name.
439 Often the same as the branch. (default: master)
440 :submodule-recursive: Whether to checkout submodules recursively.
442 :submodule-timeout: Timeout (in minutes) for checkout operation.
445 :gerrit_verify_triggers: Override Gerrit Triggers.
446 :gerrit_trigger_file_paths: Override file paths which can be used to
447 filter which file modifications will trigger a build.
448 (default defined by lf_jjb_common)
450 .. _jjb-verify-upstream-gjjb:
452 JJB Verify Upstream Global JJB
453 ------------------------------
455 Runs ``jenkins-jobs test`` to validate JJB syntax for upstream global-jjb
456 patches. This job is useful to notify upstream that they may be breaking
460 - {project-name}-jjb-verify-upstream-gjjb
461 - gerrit-jjb-verify-upstream-gjjb
463 :Comment Trigger: recheck|reverify
465 :Required parameters:
467 :build-node: The node to run build on.
468 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
469 be configured in defaults.yaml)
471 :Optional parameters:
473 :branch: Git branch to fetch for the build. (default: master)
474 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
475 :build-timeout: Timeout in minutes before aborting build. (default: 10)
476 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
477 :jjb-cache: JJB cache location. (default: $HOME/.cache/jenkins_jobs)
478 :jjb-version: JJB version to install. (default: see job-template)
479 :stream: Keyword that can be used to represent a release code-name.
480 Often the same as the branch. (default: master)
482 .. _info-yaml-verify:
487 Info YAML Verify job validates that INFO.yaml file changes are kept isolated from
488 other file changes. Verifies INFO.yaml files follow the schema defined in
489 `global-jjb/info-schema`.
492 - {project-name}-info-yaml-verify
493 - gerrit-info-yaml-verify
494 - github-info-yaml-verify
496 :Required parameters:
498 :build-node: The node to run build on.
499 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
500 be configured in defaults.yaml)
502 :Optional parameters:
504 :branch: Git branch to fetch for the build. (default: master)
505 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
506 :build-timeout: Timeout in minutes before aborting build. (default: 10)
507 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
508 :stream: Keyword that can be used to represent a release code-name.
509 Often the same as the branch. (default: master)
510 :submodule-recursive: Whether to checkout submodules recursively.
512 :submodule-timeout: Timeout (in minutes) for checkout operation.
515 :gerrit_verify_triggers: Override Gerrit Triggers.
522 Job to scan projects for files missing license headers.
525 - {project-name}-license-check
526 - gerrit-license-check
527 - github-license-check
529 :Optional parameters:
531 :build-timeout: Timeout in minutes before aborting build. (default: 15)
532 :file-patterns: Space-separated list of file patterns to scan.
533 (default: \*.go \*.groovy \*.java \*.py \*.sh)
534 :spdx-disable: Disable the SPDX-Identifier checker. (default: false)
535 :lhc-version: Version of LHC to use. (default: 0.2.0)
536 :license-exclude-paths: Comma-separated list of paths to exclude from the
537 license checker. The paths used here will be matched using a contains
538 rule so it is best to be as precise with the path as possible.
539 For example a path of '/src/generated/' will be searched as
540 '**/src/generated/**'.
541 Example: org/opendaylight/yang/gen,protobuff/messages
543 :licenses-allowed: Comma-separated list of allowed licenses.
544 (default: Apache-2.0,EPL-1.0,MIT)
545 :project-pattern: The ANT based pattern for Gerrit Trigger to choose which
546 projects to trigger job against. (default: '**')
548 .. _gjjb-openstack-cron:
553 Cron job that runs regularly to perform periodic tasks against OpenStack.
555 This job requires a Config File Provider file named ``clouds-yaml`` available
556 containing the credentials for the cloud.
559 - {project-name}-openstack-cron
560 - gerrit-openstack-cron
561 - github-openstack-cron
563 :Required parameters:
565 :build-node: The node to run build on.
566 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
567 be configured in defaults.yaml)
568 :jenkins-urls: URLs to Jenkins systems to check for active builds.
570 :Optional parameters:
572 :branch: Git branch to fetch for the build. (default: master)
573 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
574 :build-timeout: Timeout in minutes before aborting build. (default: 90)
575 :cron: Time when the packer image should be rebuilt (default: @hourly)
576 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
577 :openstack-cloud: OS_CLOUD setting to pass to openstack client.
579 :openstack-image-cleanup: Whether or not to run the image cleanup script.
581 :openstack-image-cleanup-age: Age in days of image before marking it for
582 removal. (default: 30)
583 :openstack-image-protect: Whether or not to run the image protect script.
585 :openstack-server-cleanup: Whether or not to run the server cleanup script.
587 :openstack-stack-cleanup: Whether or not to run the stack cleanup script.
589 :openstack-volume-cleanup: Whether or not to run the volume cleanup script.
591 :stream: Keyword that can be used to represent a release code-name.
592 Often the same as the branch. (default: master)
593 :submodule-recursive: Whether to checkout submodules recursively.
595 :submodule-timeout: Timeout (in minutes) for checkout operation.
600 .. literalinclude:: ../../.jjb-test/lf-ci-jobs/openstack-cron-minimal.yaml
604 .. literalinclude:: ../../.jjb-test/lf-ci-jobs/openstack-cron-full.yaml
607 .. _gjjb-packer-merge:
612 Packer Merge job runs `packer build` to build system images in the cloud.
615 - {project-name}-packer-merge-{platforms}-{templates}
616 - gerrit-packer-merge
617 - github-packer-merge
619 :Comment Trigger: remerge
621 :Required parameters:
623 :build-node: The node to run build on.
624 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
625 be configured in defaults.yaml)
626 :mvn-settings: The name of settings file containing credentials for
628 :platforms: Platform or distribution to build. Typically json file
629 found in the packer/vars directory. (Example: centos)
630 :template: System template to build. Typically shell script found in
631 the packer/provision directory. (Example: java-builder)
633 :Optional parameters:
635 :cron: Time when the packer image should be rebuilt (default: @monthly)
636 :branch: Git branch to fetch for the build. (default: master)
637 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
638 :build-timeout: Timeout in minutes before aborting build. (default: 90)
639 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
640 :openstack: Packer template uses an OpenStack builder (default: true).
641 :openstack-cloud: Sets OS_CLOUD variable to the value of this parameter.
643 :packer-cloud-settings: Name of settings file containing credentials
644 for the cloud that packer will build on. (default: packer-cloud-env)
645 :packer-version: Version of packer to install / use in build. (default: 1.0.2)
646 :stream: Keyword that can be used to represent a release code-name.
647 Often the same as the branch. (default: master)
648 :submodule-recursive: Whether to checkout submodules recursively.
650 :submodule-timeout: Timeout (in minutes) for checkout operation.
653 :gerrit_verify_triggers: Override Gerrit Triggers.
656 .. _gjjb-packer-verify:
661 Packer Verify job runs `packer validate` to verify packer configuration.
664 - {project-name}-packer-verify
665 - gerrit-packer-verify
666 - github-packer-verify
668 :Comment Trigger: recheck|reverify
670 :Required parameters:
672 :build-node: The node to run build on.
673 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
674 be configured in defaults.yaml)
675 :mvn-settings: The name of settings file containing credentials for
678 :Optional parameters:
680 :branch: Git branch to fetch for the build. (default: master)
681 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
682 :build-timeout: Timeout in minutes before aborting build. (default: 10)
683 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
684 :openstack: Packer template uses an OpenStack builder (default: true).
685 :openstack-cloud: Sets OS_CLOUD variable to the value of this parameter.
687 :packer-cloud-settings: Name of settings file containing credentials
688 for the cloud that packer will build on. (default: packer-cloud-env)
689 :packer-version: Version of packer to install / use in build. (default: 1.0.2)
690 :stream: Keyword that can be used to represent a release code-name.
691 Often the same as the branch. (default: master)
692 :submodule-recursive: Whether to checkout submodules recursively.
694 :submodule-timeout: Timeout (in minutes) for checkout operation.
697 :gerrit_verify_triggers: Override Gerrit Triggers.
698 :gerrit_trigger_file_paths: Override file paths which can be used to
699 filter which file modifications will trigger a build.