Job Groups
==========
-{project-name}-ci-jobs
-----------------------
+.. include:: ../job-groups.rst
-Recommended jobs that should be deployed for CI using Gerrit.
+Below is a list of CI job groups:
-:Includes:
-
- - gerrit-jenkins-cfg-merge
- - gerrit-jenkins-sandbox-cleanup
- - gerrit-jjb-deploy-job
- - gerrit-jjb-merge
- - gerrit-jjb-verify
-
-{project-name}-github-ci-jobs
------------------------------
-
-Recommended jobs that should be deployed CI using GitHub.
-
-:Includes:
-
- - github-jenkins-cfg-merge
- - github-jenkins-sandbox-cleanup
- - github-jjb-deploy-job
- - github-jjb-merge
- - github-jjb-verify
-
-{project-name}-info-yaml-jobs
------------------------------
-
-Jobs to verify INFO.yaml file changes.
-
-:Includes:
-
- - gerrit-info-yaml-verify
-
-{project-name}-github-info-yaml-jobs
-------------------------------------
-
-Jobs to verify INFO.yaml file changes using Github.
-
-:Includes:
-
- - github-info-yaml-verify
-
-{project-name}-packer-jobs
---------------------------
-
-Jobs related to Packer builds for CI using Gerrit.
-
-:Includes:
-
- - gerrit-packer-merge
- - gerrit-packer-verify
-
-{project-name}-github-packer-jobs
----------------------------------
-
-Jobs related to Packer builds for CI using GitHub.
-
-:Includes:
+.. literalinclude:: ../../jjb/lf-ci-job-groups.yaml
+ :language: yaml
- - github-packer-merge
- - github-packer-verify
Macros
======
:Required Parameters:
+ :jjb-cache: Location of Jenkins Job Builder (JJB) cache used for jjb
+ jobs.
:jjb-version: Version of Jenkins Job Builder (JJB) to install and use in
the jjb jobs.
Gerrit file-paths for packer verify jobs.
+lf-puppet-parameters
+--------------------
+
+Parameters useful for Puppet related tasks.
+
+:Parameters:
+
+ :puppet-lint-version: Version of puppet-lint to install / use.
+ (shell: PUPPET_LINT_VERSION)
+
Job Templates
=============
+.. _gerrit-branch-lock:
+
Gerrit Branch Lock
------------------
- {project-name}-gerrit-branch-lock-{stream}
- gerrit-branch-lock
+:Comment Trigger:
+
+ * lock branch
+ * unlock branch
+
+:Required parameters:
+
+ :build-node: The node to run build on.
+ :jenkins-ssh-credential: Credential to use for SSH. (Generally
+ should be configured in defaults.yaml)
+
+:Optional parameters:
+
+ :branch: Git branch to build against. (default: master)
+ :git-url: URL to clone project from. (default: $GIT_URL/$GERRIT_PROJECT)
+ :stream: Keyword that can be used to represent a release code-name.
+ Often the same as the branch. (default: master)
+ :submodule-timeout: Timeout (in minutes) for checkout operation.
+ (default: 10)
+ :gerrit_merge_triggers: Override Gerrit Triggers.
.. _lf-global-jjb-jenkins-cfg-merge:
"{project-name}-ci-jobs" job-group and does not need to be explicitly called if
the job group is being used.
-Miniaml Example:
+Minimal Example:
.. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-merge-minimal.yaml
:language: yaml
(default: "os-cloud")
:INSTANCE_CAP: Total number of instances the cloud will allow spin up.
(default: null)
- :SANDBOX_CAP: Total number of instances the clodu will allow to
+ :SANDBOX_CAP: Total number of instances the cloud will allow to
spin up. This applies to "sandbox" systems and overrides the
INSTANCE_CAP setting. (default: null)
the one defined in default clouds will be inherited.
:IMAGE_NAME: The image name to use for this template.
- (default: "")
+ (required)
+ :HARDWARE_ID: OpenStack flavor to use. (required)
+
:LABELS: Labels to assign to the vm. (default: FILE_NAME)
- :HARDWARE_ID: OpenStack flavor to use. (default: "")
:NETWORK_ID: OpenStack network to use. (default: "")
:USER_DATA_ID: User Data to pass into the instance.
(default: jenkins-init-script)
used to push to Jenkins. In the event of a job failure this file can be
inspected.
+ .. _lf-global-jjb-jenkins-cfg-verify:
+
+Jenkins Configuration Verify
+----------------------------
+
+Jenkins job to verify the Global Jenkins configuration.
+
+Requires the ``clouds-yaml`` file to be setup on the Jenkins host.
+
+:Template names:
+
+ - {project-name}-jenkins-cfg-verify
+ - gerrit-jenkins-cfg-verify
+ - github-jenkins-cfg-verify
+
+:Optional parameters:
+
+ :branch: Git branch to build against. (default: master)
+ :git-url: URL to clone project from. (default: $GIT_URL/$GERRIT_PROJECT)
+
+This job is not part of the "{project-name}-ci-jobs" group. It must be called
+explicitly.
+
+Example:
+
+.. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-verify.yaml
+ :language: yaml
+
+
+.. _jenkins-sandbox-cleanup:
Jenkins Sandbox Cleanup
-----------------------
:cron: Schedule to run job. (default: '0 8 * * 6')
+.. _jjb-deploy:
+
JJB Deploy Job
--------------
-Deploy jobs to jenkins-sandbox system via code review comment
+Deploy jobs to jenkins-sandbox system via code review comment.
This job checks out the current code review patch and then runs a
-`jenkins-jobs update` to push a patch defined by the comment.
+``jenkins-jobs update`` to push a patch defined by the comment.
:Template names:
.. note::
JOB_NAME can include the * wildcard character to push multiple jobs
- matching the pattern. For example `jjb-deploy builder-jjb-*`` will push
+ matching the pattern. For example ``jjb-deploy builder-jjb-*`` will push
all builder-jjb-* jobs to the sandbox system.
:Required parameters:
:gerrit_jjb_deploy_job_triggers: Override Gerrit Triggers.
+.. _jjb-merge:
+
JJB Merge
---------
:build-node: The node to run build on.
:jenkins-ssh-credential: Credential to use for SSH. (Generally should
be configured in defaults.yaml)
- :mvn-settings: The name of settings file containing credentials for
- the project.
:Optional parameters:
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
:build-timeout: Timeout in minutes before aborting build. (default: 10)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+ :jjb-cache: JJB cache location. (default: $HOME/.cache/jenkins_jobs)
:jjb-version: JJB version to install. (default: see job-template)
:stream: Keyword that can be used to represent a release code-name.
Often the same as the branch. (default: master)
:submodule-recursive: Whether to checkout submodules recursively.
(default: true)
+ :submodule-timeout: Timeout (in minutes) for checkout operation.
+ (default: 10)
:gerrit_merge_triggers: Override Gerrit Triggers.
:gerrit_trigger_file_paths: Override file paths which can be used to
(default defined by lf_jjb_common)
+.. _jjb-verify:
+
JJB Verify
----------
:build-node: The node to run build on.
:jenkins-ssh-credential: Credential to use for SSH. (Generally should
be configured in defaults.yaml)
- :mvn-settings: The name of settings file containing credentials for
- the project.
:Optional parameters:
:branch: Git branch to fetch for the build. (default: master)
+ :build-concurrent: Whether or not to allow this job to run multiple jobs
+ simultaneously. (default: true)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
:build-timeout: Timeout in minutes before aborting build. (default: 10)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+ :jjb-cache: JJB cache location. (default: $HOME/.cache/jenkins_jobs)
:jjb-version: JJB version to install. (default: see job-template)
:stream: Keyword that can be used to represent a release code-name.
Often the same as the branch. (default: master)
:submodule-recursive: Whether to checkout submodules recursively.
(default: true)
+ :submodule-timeout: Timeout (in minutes) for checkout operation.
+ (default: 10)
:gerrit_verify_triggers: Override Gerrit Triggers.
:gerrit_trigger_file_paths: Override file paths which can be used to
filter which file modifications will trigger a build.
(default defined by lf_jjb_common)
+.. _jjb-verify-upstream-gjjb:
+
+JJB Verify Upstream Global JJB
+------------------------------
+
+Runs ``jenkins-jobs test`` to validate JJB syntax for upstream global-jjb
+patches. This job is useful to notify upstream that they may be breaking
+project level jobs.
+
+:Template Names:
+ - {project-name}-jjb-verify-upstream-gjjb
+ - gerrit-jjb-verify-upstream-gjjb
+
+:Comment Trigger: recheck|reverify
+
+:Required parameters:
+
+ :build-node: The node to run build on.
+ :jenkins-ssh-credential: Credential to use for SSH. (Generally should
+ be configured in defaults.yaml)
+
+:Optional parameters:
+
+ :branch: Git branch to fetch for the build. (default: master)
+ :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
+ :build-timeout: Timeout in minutes before aborting build. (default: 10)
+ :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+ :jjb-cache: JJB cache location. (default: $HOME/.cache/jenkins_jobs)
+ :jjb-version: JJB version to install. (default: see job-template)
+ :stream: Keyword that can be used to represent a release code-name.
+ Often the same as the branch. (default: master)
+
.. _info-yaml-verify:
Info YAML Verify
Often the same as the branch. (default: master)
:submodule-recursive: Whether to checkout submodules recursively.
(default: true)
+ :submodule-timeout: Timeout (in minutes) for checkout operation.
+ (default: 10)
+
:gerrit_verify_triggers: Override Gerrit Triggers.
.. _license-checker:
:project-pattern: The ANT based pattern for Gerrit Trigger to choose which
projects to trigger job against. (default: '**')
+.. _gjjb-openstack-cron:
+
+OpenStack Cron
+--------------
+
+Cron job that runs regularly to perform periodic tasks against OpenStack.
+
+This job requires a Config File Provider file named ``clouds-yaml`` available
+containing the credentials for the cloud.
+
+:Template Names:
+ - {project-name}-openstack-cron
+ - gerrit-openstack-cron
+ - github-openstack-cron
+
+:Required parameters:
+
+ :build-node: The node to run build on.
+ :jenkins-ssh-credential: Credential to use for SSH. (Generally should
+ be configured in defaults.yaml)
+ :jenkins-urls: URLs to Jenkins systems to check for active builds.
+
+:Optional parameters:
+
+ :branch: Git branch to fetch for the build. (default: master)
+ :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
+ :build-timeout: Timeout in minutes before aborting build. (default: 90)
+ :cron: Time when the packer image should be rebuilt (default: @hourly)
+ :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+ :openstack-cloud: OS_CLOUD setting to pass to openstack client.
+ (default: vex)
+ :openstack-image-cleanup: Whether or not to run the image cleanup script.
+ (default: true)
+ :openstack-image-cleanup-age: Age in days of image before marking it for
+ removal. (default: 30)
+ :openstack-image-protect: Whether or not to run the image protect script.
+ (default: true)
+ :openstack-server-cleanup: Whether or not to run the server cleanup script.
+ (default: true)
+ :openstack-stack-cleanup: Whether or not to run the stack cleanup script.
+ (default: true)
+ :openstack-volume-cleanup: Whether or not to run the volume cleanup script.
+ (default: true)
+ :stream: Keyword that can be used to represent a release code-name.
+ Often the same as the branch. (default: master)
+ :submodule-recursive: Whether to checkout submodules recursively.
+ (default: true)
+ :submodule-timeout: Timeout (in minutes) for checkout operation.
+ (default: 10)
+
+Minimal Example:
+
+.. literalinclude:: ../../.jjb-test/lf-ci-jobs/openstack-cron-minimal.yaml
+
+Full Example:
+
+.. literalinclude:: ../../.jjb-test/lf-ci-jobs/openstack-cron-full.yaml
+
+
.. _gjjb-packer-merge:
Packer Merge
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
:build-timeout: Timeout in minutes before aborting build. (default: 90)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+ :openstack: Packer template uses an OpenStack builder (default: true).
+ :openstack-cloud: Sets OS_CLOUD variable to the value of this parameter.
+ (default: vex).
:packer-cloud-settings: Name of settings file containing credentials
for the cloud that packer will build on. (default: packer-cloud-env)
:packer-version: Version of packer to install / use in build. (default: 1.0.2)
Often the same as the branch. (default: master)
:submodule-recursive: Whether to checkout submodules recursively.
(default: true)
+ :submodule-timeout: Timeout (in minutes) for checkout operation.
+ (default: 10)
:gerrit_verify_triggers: Override Gerrit Triggers.
+Test an in-progress patch
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To test an in-progress patch from a GitHub Pull Request. Upload this
+job to the :doc:`Jenkins Sandbox <lfdocs:jenkins-sandbox>`. Then when manually
+building the job replace the GERRIT_REFSPEC parameter with the GitHub Pull
+Request number of the patch you would like to test.
+
+Example GitHub:
+
+.. code-block:: none
+
+ GERRIT_REFSPEC: origin/pr/49/merge
.. _gjjb-packer-verify:
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
:build-timeout: Timeout in minutes before aborting build. (default: 10)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+ :openstack: Packer template uses an OpenStack builder (default: true).
+ :openstack-cloud: Sets OS_CLOUD variable to the value of this parameter.
+ (default: vex).
:packer-cloud-settings: Name of settings file containing credentials
for the cloud that packer will build on. (default: packer-cloud-env)
:packer-version: Version of packer to install / use in build. (default: 1.0.2)
Often the same as the branch. (default: master)
:submodule-recursive: Whether to checkout submodules recursively.
(default: true)
+ :submodule-timeout: Timeout (in minutes) for checkout operation.
+ (default: 10)
:gerrit_verify_triggers: Override Gerrit Triggers.
:gerrit_trigger_file_paths: Override file paths which can be used to
filter which file modifications will trigger a build.
+
+
+Puppet Verify
+-------------
+
+Runs puppet-lint in the ``puppet-dir`` directory. puppet-lint runs recursively,
+so the base directory is usually the best place to run from.
+
+:Template Names:
+
+ - {project-name}-puppet-verify
+ - gerrit-puppet-verify
+ - github-puppet-verify
+
+:Comment Trigger: recheck|reverify
+
+:Required Parameters:
+
+ :build-node: The node to run build on.
+ :jenkins-ssh-credential: Credential to use for SSH. (Generally set
+ in defaults.yaml)
+
+:Optional Parameters:
+
+ :branch: The branch to build against. (default: master)
+ :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
+ :build-timeout: Timeout in minutes before aborting build. (default: 15)
+ :gerrit_trigger_file_paths: Override file paths which used to filter which
+ file modifications will trigger a build. Refer to JJB documentation for
+ "file-path" details.
+ https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit
+ :git-url: URL clone project from. (default: $GIT_URL/$GERRIT_PROJECT)
+ :puppet-dir: Directory containing the project's puppet module(s) relative
+ to the workspace.
+ (default: '')
+ :puppet-lint-version: Version of puppet-lint to use for testing.
+ (default: 2.3.6)
+ :stream: Keyword representing a release code-name.
+ Often the same as the branch. (default: master)
+ :submodule-recursive: Whether to checkout submodules recursively.
+ (default: true)
+ :submodule-timeout: Timeout (in minutes) for checkout operation.
+ (default: 10)