Add Sonar plug-in configurations to docs

[releng/global-jjb.git] / docs / jjb / lf-ci-jobs.rst

diff --git a/docs/jjb/lf-ci-jobs.rst b/docs/jjb/lf-ci-jobs.rst
index 25f9d11..9883dd7 100644 (file)
--- a/docs/jjb/lf-ci-jobs.rst
+++ b/docs/jjb/lf-ci-jobs.rst
@@ -64,13 +64,6 @@ lf-infra-jjbini
 
 Provides jenkins_jobs.ini configuration for Jenkins.
 
 
 Provides jenkins_jobs.ini configuration for Jenkins.
 

-lf-infra-jjbini-sandbox
------------------------
-
-Provides jenkins_jobs.ini configuration for Jenkins sandbox.
-
-.. todo:: This needs to be consolidated into lf-infra-jjbini when JJB 2.0 is available
-

 lf-packer-common
 ----------------
 
 lf-packer-common
 ----------------
 


@@ -96,6 +89,16 @@ lf-packer-verify-file-paths
 
 Gerrit file-paths for packer verify jobs.
 
 
 Gerrit file-paths for packer verify jobs.
 

+lf-puppet-parameters
+--------------------
+
+Parameters useful for Puppet related tasks.
+
+:Parameters:
+
+    :puppet-lint-version: Version of puppet-lint to install / use.
+        (shell: PUPPET_LINT_VERSION)
+

 Job Templates
 =============
 
 Job Templates
 =============
 


@@ -129,6 +132,8 @@ Job submits a patch to lock or unlock a project's branch.
         Often the same as the branch. (default: master)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)
         Often the same as the branch. (default: master)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)

+    :submodule-disable: Disable submodule checkout operation.
+        (default: false)

     :gerrit_merge_triggers: Override Gerrit Triggers.
 
 .. _lf-global-jjb-jenkins-cfg-merge:
     :gerrit_merge_triggers: Override Gerrit Triggers.
 
 .. _lf-global-jjb-jenkins-cfg-merge:


@@ -284,6 +289,36 @@ Troubleshooting
     used to push to Jenkins. In the event of a job failure this file can be
     inspected.
 
     used to push to Jenkins. In the event of a job failure this file can be
     inspected.
 

+ .. _lf-global-jjb-jenkins-cfg-verify:
+
+Jenkins Configuration Verify
+----------------------------
+
+Jenkins job to verify the Global Jenkins configuration.
+
+Requires the ``clouds-yaml`` file to be setup on the Jenkins host.
+
+:Template names:
+
+    - {project-name}-jenkins-cfg-verify
+    - gerrit-jenkins-cfg-verify
+    - github-jenkins-cfg-verify
+
+:Optional parameters:
+
+    :branch: Git branch to build against. (default: master)
+    :git-url: URL to clone project from. (default: $GIT_URL/$GERRIT_PROJECT)
+
+This job is not part of the "{project-name}-ci-jobs" group. It must be called
+explicitly.
+
+Example:
+
+.. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-verify.yaml
+   :language: yaml
+
+
+.. _jenkins-sandbox-cleanup:

 
 Jenkins Sandbox Cleanup
 -----------------------
 
 Jenkins Sandbox Cleanup
 -----------------------


@@ -309,13 +344,15 @@ Cleanup Jenkins Sandbox of jobs and views periodically.
     :cron: Schedule to run job. (default: '0 8 * * 6')
 
 
     :cron: Schedule to run job. (default: '0 8 * * 6')
 
 

+.. _jjb-deploy:
+

 JJB Deploy Job
 --------------
 
 JJB Deploy Job
 --------------
 

-Deploy jobs to jenkins-sandbox system via code review comment
+Deploy jobs to jenkins-sandbox system via code review comment.

 
 This job checks out the current code review patch and then runs a
 
 This job checks out the current code review patch and then runs a

-`jenkins-jobs update` to push a patch defined by the comment.
+``jenkins-jobs update`` to push a patch defined by the comment.

 
 :Template names:
 
 
 :Template names:
 


@@ -327,8 +364,20 @@ This job checks out the current code review patch and then runs a
 
     .. note::
 
 
     .. note::
 

+       The JJB Deploy Job is configured to trigger only if the Gerrit comment
+       starts with the `jjb-deploy` keyword.
+
+       Example of a valid command in Gerrit comment that triggers the job:
+
+       ``jjb-deploy builder-jjb-*``
+
+       Example of a invalid command in Gerrit comment that would _not_ trigger
+       the job:
+
+       ``Update the job. jjb-deploy builder-jjb-*``
+

        JOB_NAME can include the * wildcard character to push multiple jobs
        JOB_NAME can include the * wildcard character to push multiple jobs

-       matching the pattern. For example `jjb-deploy builder-jjb-*`` will push
+       matching the pattern. For example ``jjb-deploy builder-jjb-*`` will push

        all builder-jjb-* jobs to the sandbox system.
 
 :Required parameters:
        all builder-jjb-* jobs to the sandbox system.
 
 :Required parameters:


@@ -343,6 +392,8 @@ This job checks out the current code review patch and then runs a
     :gerrit_jjb_deploy_job_triggers: Override Gerrit Triggers.
 
 
     :gerrit_jjb_deploy_job_triggers: Override Gerrit Triggers.
 
 

+.. _jjb-merge:
+

 JJB Merge
 ---------
 
 JJB Merge
 ---------
 


@@ -368,6 +419,8 @@ Runs `jenkins-jobs update` to update production job configuration
     :build-timeout: Timeout in minutes before aborting build. (default: 10)
     :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
     :jjb-cache: JJB cache location. (default: $HOME/.cache/jenkins_jobs)
     :build-timeout: Timeout in minutes before aborting build. (default: 10)
     :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
     :jjb-cache: JJB cache location. (default: $HOME/.cache/jenkins_jobs)

+    :jjb-workers: Number of threads to run **update** with. Set to 0 by default
+        which is equivalent to the number of available CPU cores. (default: 0)

     :jjb-version: JJB version to install. (default: see job-template)
     :stream: Keyword that can be used to represent a release code-name.
         Often the same as the branch. (default: master)
     :jjb-version: JJB version to install. (default: see job-template)
     :stream: Keyword that can be used to represent a release code-name.
         Often the same as the branch. (default: master)


@@ -375,6 +428,8 @@ Runs `jenkins-jobs update` to update production job configuration
         (default: true)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)
         (default: true)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)

+    :submodule-disable: Disable submodule checkout operation.
+        (default: false)

 
     :gerrit_merge_triggers: Override Gerrit Triggers.
     :gerrit_trigger_file_paths: Override file paths which can be used to
 
     :gerrit_merge_triggers: Override Gerrit Triggers.
     :gerrit_trigger_file_paths: Override file paths which can be used to


@@ -382,6 +437,8 @@ Runs `jenkins-jobs update` to update production job configuration
         (default defined by lf_jjb_common)
 
 
         (default defined by lf_jjb_common)
 
 

+.. _jjb-verify:
+

 JJB Verify
 ----------
 
 JJB Verify
 ----------
 


@@ -403,6 +460,8 @@ Runs `jenkins-jobs test` to validate JJB syntax
 :Optional parameters:
 
     :branch: Git branch to fetch for the build. (default: master)
 :Optional parameters:
 
     :branch: Git branch to fetch for the build. (default: master)

+    :build-concurrent: Whether or not to allow this job to run multiple jobs
+        simultaneously. (default: true)

     :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
     :build-timeout: Timeout in minutes before aborting build. (default: 10)
     :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
     :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
     :build-timeout: Timeout in minutes before aborting build. (default: 10)
     :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)


@@ -414,6 +473,17 @@ Runs `jenkins-jobs test` to validate JJB syntax
         (default: true)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)
         (default: true)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)

+    :submodule-disable: Disable submodule checkout operation.
+        (default: false)
+    :throttle_categories: List of categories to throttle by.
+    :throttle-enabled: Whether or not to enable throttling on the job.
+        (default: true)
+    :throttle-max-per-node: Max jobs to run on the same node. (default: 1)
+    :throttle-max-total: Max jobs to run across the entire project. - 0
+        means 'unlimited' (default: 0)
+    :throttle-option: Throttle by the project or by list of categories
+        defined in the throttle plugin configuration. (options: 'project',
+        'category'; default: project)

 
     :gerrit_verify_triggers: Override Gerrit Triggers.
     :gerrit_trigger_file_paths: Override file paths which can be used to
 
     :gerrit_verify_triggers: Override Gerrit Triggers.
     :gerrit_trigger_file_paths: Override file paths which can be used to


@@ -459,7 +529,7 @@ Info YAML Verify
 
 Info YAML Verify job validates that INFO.yaml file changes are kept isolated from
 other file changes. Verifies INFO.yaml files follow the schema defined in
 
 Info YAML Verify job validates that INFO.yaml file changes are kept isolated from
 other file changes. Verifies INFO.yaml files follow the schema defined in

-`global-jjb/info-schema`.
+`lfit/releng-global-jjb/schema/info-schema.yaml`.

 
 :Template Names:
     - {project-name}-info-yaml-verify
 
 :Template Names:
     - {project-name}-info-yaml-verify


@@ -484,6 +554,8 @@ other file changes. Verifies INFO.yaml files follow the schema defined in
         (default: true)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)
         (default: true)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)

+    :submodule-disable: Disable submodule checkout operation.
+        (default: false)

 
     :gerrit_verify_triggers: Override Gerrit Triggers.
 
 
     :gerrit_verify_triggers: Override Gerrit Triggers.
 


@@ -538,22 +610,46 @@ containing the credentials for the cloud.
     :build-node: The node to run build on.
     :jenkins-ssh-credential: Credential to use for SSH. (Generally should
         be configured in defaults.yaml)
     :build-node: The node to run build on.
     :jenkins-ssh-credential: Credential to use for SSH. (Generally should
         be configured in defaults.yaml)

+    :jenkins-urls: URLs to Jenkins systems to check for active builds.

 
 :Optional parameters:
 
     :branch: Git branch to fetch for the build. (default: master)
     :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
     :build-timeout: Timeout in minutes before aborting build. (default: 90)
 
 :Optional parameters:
 
     :branch: Git branch to fetch for the build. (default: master)
     :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
     :build-timeout: Timeout in minutes before aborting build. (default: 90)

-    :cron: Time when the packer image should be rebuilt (default: @daily)
+    :cron: Time when the packer image should be rebuilt (default: @hourly)

     :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
     :openstack-cloud: OS_CLOUD setting to pass to openstack client.
         (default: vex)
     :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
     :openstack-cloud: OS_CLOUD setting to pass to openstack client.
         (default: vex)

+    :openstack-image-cleanup: Whether or not to run the image cleanup script.
+        (default: true)
+    :openstack-image-cleanup-age: Age in days of image before marking it for
+        removal. (default: 30)
+    :openstack-image-protect: Whether or not to run the image protect script.
+        (default: true)
+    :openstack-server-cleanup: Whether or not to run the server cleanup script.
+        (default: true)
+    :openstack-stack-cleanup: Whether or not to run the stack cleanup script.
+        (default: true)
+    :openstack-volume-cleanup: Whether or not to run the volume cleanup script.
+        (default: true)

     :stream: Keyword that can be used to represent a release code-name.
         Often the same as the branch. (default: master)
     :submodule-recursive: Whether to checkout submodules recursively.
         (default: true)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)
     :stream: Keyword that can be used to represent a release code-name.
         Often the same as the branch. (default: master)
     :submodule-recursive: Whether to checkout submodules recursively.
         (default: true)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)

+    :submodule-disable: Disable submodule checkout operation.
+        (default: false)
+
+Minimal Example:
+
+.. literalinclude:: ../../.jjb-test/lf-ci-jobs/openstack-cron-minimal.yaml
+
+Full Example:
+
+.. literalinclude:: ../../.jjb-test/lf-ci-jobs/openstack-cron-full.yaml
+

 
 .. _gjjb-packer-merge:
 
 
 .. _gjjb-packer-merge:
 


@@ -600,9 +696,24 @@ Packer Merge job runs `packer build` to build system images in the cloud.
         (default: true)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)
         (default: true)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)

+    :submodule-disable: Disable submodule checkout operation.
+        (default: false)

 
     :gerrit_verify_triggers: Override Gerrit Triggers.
 
 
     :gerrit_verify_triggers: Override Gerrit Triggers.
 

+Test an in-progress patch
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To test an in-progress patch from a GitHub Pull Request. Upload this
+job to the :doc:`Jenkins Sandbox <lfdocs:jenkins-sandbox>`. Then when manually
+building the job replace the GERRIT_REFSPEC parameter with the GitHub Pull
+Request number of the patch you would like to test.
+
+Example GitHub:
+
+.. code-block:: none
+
+   GERRIT_REFSPEC: origin/pr/49/merge

 
 .. _gjjb-packer-verify:
 
 
 .. _gjjb-packer-verify:
 


@@ -644,7 +755,86 @@ Packer Verify job runs `packer validate` to verify packer configuration.
         (default: true)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)
         (default: true)
     :submodule-timeout: Timeout (in minutes) for checkout operation.
         (default: 10)

+    :submodule-disable: Disable submodule checkout operation.
+        (default: false)

 
     :gerrit_verify_triggers: Override Gerrit Triggers.
     :gerrit_trigger_file_paths: Override file paths which can be used to
         filter which file modifications will trigger a build.
 
     :gerrit_verify_triggers: Override Gerrit Triggers.
     :gerrit_trigger_file_paths: Override file paths which can be used to
         filter which file modifications will trigger a build.

+
+
+Puppet Verify
+-------------
+
+Runs puppet-lint in the ``puppet-dir`` directory. puppet-lint runs recursively,
+the base directory is usually the best place to run from.
+
+:Template Names:
+
+    - {project-name}-puppet-verify
+    - gerrit-puppet-verify
+    - github-puppet-verify
+
+:Comment Trigger: recheck|reverify
+
+:Required Parameters:
+
+    :build-node: The node to run build on.
+    :jenkins-ssh-credential: Credential to use for SSH. (Generally set
+        in defaults.yaml)
+
+:Optional Parameters:
+
+    :branch: The branch to build against. (default: master)
+    :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
+    :build-timeout: Timeout in minutes before aborting build. (default: 15)
+    :gerrit_trigger_file_paths: Override file paths which used to filter which
+        file modifications will trigger a build. Refer to JJB documentation for
+        "file-path" details.
+        https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit
+    :git-url: URL clone project from. (default: $GIT_URL/$GERRIT_PROJECT)
+    :puppet-dir: Directory containing the project's puppet module(s) relative
+        to the workspace.
+        (default: '')
+    :puppet-lint-version: Version of puppet-lint to use for testing.
+        (default: 2.3.6)
+    :stream: Keyword representing a release code-name.
+        Often the same as the branch. (default: master)
+    :submodule-recursive: Whether to checkout submodules recursively.
+        (default: true)
+    :submodule-timeout: Timeout (in minutes) for checkout operation.
+        (default: 10)
+    :submodule-disable: Disable submodule checkout operation.
+        (default: false)
+
+
+Sonar
+-----
+
+Runs Jenkins Sonarqube plug-in to review for bugs, code smells,
+and security vulnerabilities.
+
+Requires ``SonarQube Scanner for Jenkins``
+
+Plug-in configurations
+    Manage Jenkins --> Configure System --> SonarQube servers
+        - Name: Sonar (fixed)
+        - Server URL: https://sonar.server.org/
+        - Server authentication token: none
+
+    Manage Jenkins --> Global Tool Configuration --> SonarQube Scanner
+        - Name: SonarQube Scanner (fixed)
+        - Install automatically
+        - Select latest version
+
+:Template Names:
+
+    - {project-name}-sonar
+    - gerrit-sonar
+    - github-sonar
+
+:Optional Parameters:
+    :sonar-task: Sonar task to run. (default: "")
+    :sonar-properties: Sonar configuration properties. (default: "")
+    :sonar-java-opts: JVM options. (default: "")
+    :sonar-additional-args: Additional command line arguments. (default: "")