Refactor job-groups documentation

Improve the way job-groups are documented so that it stays properly
in sync. This update recommends projects define their own job-groups
and using the ones defined in global-jjb only as a starting point.

Update ensure-documented to ignore job-groups since they should be
self documenting now in the new format.

Change-Id: I945735ecd50596605139c6522903a49350a1ea10
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

diff --git a/docs/jjb/lf-ci-jobs.rst b/docs/jjb/lf-ci-jobs.rst
index 18cd0a5..0ac86d9 100644 (file)
--- a/docs/jjb/lf-ci-jobs.rst
+++ b/docs/jjb/lf-ci-jobs.rst
@@ -5,69 +5,13 @@ CI Jobs
 Job Groups
 ==========
 
-{project-name}-ci-jobs
-----------------------
+.. include:: ../job-groups.rst
 
-Recommended jobs that should be deployed for CI using Gerrit.
+Below is a list of CI job groups:
 
-:Includes:
-
-    - gerrit-jenkins-cfg-merge
-    - gerrit-jenkins-sandbox-cleanup
-    - gerrit-jjb-deploy-job
-    - gerrit-jjb-merge
-    - gerrit-jjb-verify
-
-{project-name}-github-ci-jobs
------------------------------
-
-Recommended jobs that should be deployed CI using GitHub.
-
-:Includes:
-
-    - github-jenkins-cfg-merge
-    - github-jenkins-sandbox-cleanup
-    - github-jjb-deploy-job
-    - github-jjb-merge
-    - github-jjb-verify
-
-{project-name}-info-yaml-jobs
------------------------------
-
-Jobs to verify INFO.yaml file changes.
-
-:Includes:
-
-    - gerrit-info-yaml-verify
-
-{project-name}-github-info-yaml-jobs
-------------------------------------
-
-Jobs to verify INFO.yaml file changes using Github.
-
-:Includes:
-
-    - github-info-yaml-verify
-
-{project-name}-packer-jobs
---------------------------
-
-Jobs related to Packer builds for CI using Gerrit.
-
-:Includes:
-
-    - gerrit-packer-merge
-    - gerrit-packer-verify
-
-{project-name}-github-packer-jobs
----------------------------------
-
-Jobs related to Packer builds for CI using GitHub.
-
-:Includes:
+.. literalinclude:: ../../jjb/lf-ci-job-groups.yaml
+   :language: yaml
 
-    - github-packer-merge
-    - github-packer-verify
 
 Macros
 ======

diff --git a/docs/jjb/lf-maven-jobs.rst b/docs/jjb/lf-maven-jobs.rst
index 02de9f1..74a3e7e 100644 (file)
--- a/docs/jjb/lf-maven-jobs.rst
+++ b/docs/jjb/lf-maven-jobs.rst
@@ -5,48 +5,12 @@ Maven Jobs
 Job Groups
 ==========
 
-{project-name}-maven-jobs
--------------------------
+.. include:: ../job-groups.rst
 
-Jobs for Maven projects using Gerrit.
+Below is a list of Maven job groups:
 
-:Includes:
-
-    - gerrit-maven-clm
-    - gerrit-maven-stage
-    - gerrit-maven-verify
-    - gerrit-maven-verify-dependencies
-
-{project-name}-github-maven-jobs
---------------------------------
-
-Jobs for Maven projects using GitHub.
-
-:Includes:
-
-    - github-maven-clm
-    - github-maven-stage
-    - github-maven-verify
-
-{project-name}-maven-javadoc-jobs
----------------------------------
-
-Jobs for Maven projects to generate javadoc using Gerrit.
-
-:Includes:
-
-    - gerrit-maven-javadoc-publish
-    - gerrit-maven-javadoc-verify
-
-{project-name}-github-maven-javadoc-jobs
-----------------------------------------
-
-Jobs for Maven projects to generate javadoc using GitHub.
-
-:Includes:
-
-    - github-maven-javadoc-publish
-    - github-maven-javadoc-verify
+.. literalinclude:: ../../jjb/lf-maven-job-groups.yaml
+   :language: yaml
 
 
 Macros

diff --git a/docs/jjb/lf-node-jobs.rst b/docs/jjb/lf-node-jobs.rst
index d3feb1a..dd57e46 100644 (file)
--- a/docs/jjb/lf-node-jobs.rst
+++ b/docs/jjb/lf-node-jobs.rst
@@ -5,23 +5,13 @@ NodeJS Jobs
 Job Groups
 ==========
 
-{project-name}-node-jobs
-------------------------
+.. include:: ../job-groups.rst
 
-Jobs for NodeJS projects using Gerrit.
+Below is a list of Node job groups:
 
-:Includes:
+.. literalinclude:: ../../jjb/lf-node-job-groups.yaml
+   :language: yaml
 
-    - gerrit-node-verify
-
-{project-name}-github-node-jobs
--------------------------------
-
-Jobs for NodeJS projects using GitHub.
-
-:Includes:
-
-    - github-node-verify
 
 Job Templates
 =============

diff --git a/docs/jjb/lf-python-jobs.rst b/docs/jjb/lf-python-jobs.rst
index deb08c2..76b4548 100644 (file)
--- a/docs/jjb/lf-python-jobs.rst
+++ b/docs/jjb/lf-python-jobs.rst
@@ -5,23 +5,12 @@ Python Jobs
 Job Groups
 ==========
 
-{project-name}-python-jobs
---------------------------
+.. include:: ../job-groups.rst
 
-Jobs for Python projects using Gerrit.
+Below is a list of Maven job groups:
 
-:Includes:
-
-    - gerrit-tox-verify
-
-{project-name}-github-python-jobs
----------------------------------
-
-Jobs for Python projects using GitHub.
-
-:Includes:
-
-    - github-tox-verify
+.. literalinclude:: ../../jjb/lf-python-job-groups.yaml
+   :language: yaml
 
 
 Macros

diff --git a/docs/jjb/lf-rtd-jobs.rst b/docs/jjb/lf-rtd-jobs.rst
index 0cd94ed..ca0aac2 100644 (file)
--- a/docs/jjb/lf-rtd-jobs.rst
+++ b/docs/jjb/lf-rtd-jobs.rst
@@ -7,25 +7,12 @@ ReadTheDocs Jobs
 Job Groups
 ==========
 
-{project-name}-rtd-jobs
------------------------
+.. include:: ../job-groups.rst
 
-Jobs to deploy for a project producing ReadTheDocs using Gerrit.
+Below is a list of Maven job groups:
 
-:Includes:
-
-    - gerrit-rtd-merge
-    - gerrit-rtd-verify
-
-{project-name}-github-rtd-jobs
-------------------------------
-
-Jobs to deploy for a project producing ReadTheDocs using GitHub.
-
-:Includes:
-
-    - github-rtd-merge
-    - github-rtd-verify
+.. literalinclude:: ../../jjb/lf-rtd-job-groups.yaml
+   :language: yaml
 
 
 Macros

diff --git a/docs/job-groups.rst b/docs/job-groups.rst
new file mode 100644 (file)
index 0000000..d9ce2ff
--- /dev/null
+++ b/docs/job-groups.rst
@@ -0,0 +1,34 @@
+Job groups are a great tool to configure categories of jobs together at the
+same time. Below the example are some starting point job-groups but we
+recommend creating your own to ensure that the jobs configured reflect the
+project's needs.
+
+An example project:
+
+.. code-block:: yaml
+
+    - job-group:
+        name: odl-maven-jobs
+
+        jobs:
+          - gerrit-maven-clm
+          - gerrit-maven-merge
+          - gerrit-maven-release
+          - gerrit-maven-verify
+          - gerrit-maven-verify-dependencies:
+              build-timeout: 180
+
+        mvn-version: mvn35
+
+    - project:
+        name: aaa
+        jobs:
+          - odl-maven-jobs
+
+In this example we are using the job-group to assign a list of common jobs to
+the aaa project. The job-group also hardcodes ``mvn-version`` to *mvn35* and
+``build-timeout`` to *180* for all projects using this job-group.
+
+A benefit of this method is for example disabling entire category of jobs by
+modifying the job-group, insert ``disable-jobs: true`` parameter
+against the jobs to disable.

diff --git a/ensure-documented.sh b/ensure-documented.sh
index 434e06c..12d4927 100755 (executable)
--- a/ensure-documented.sh
+++ b/ensure-documented.sh
@@ -16,7 +16,6 @@ mapfile -t jjb_files < <(find jjb -name "*.yaml")
 undocumented_count=0
 for file in "${jjb_files[@]}"; do
     mapfile -t docs_interests < <(grep -e '\- builder:' \
-         -e '\- job-group:' \
          -e '\- job-template:' \
          -e '\- parameter:' \
          -e '\- property:' \

diff --git a/jjb/lf-ci-job-groups.yaml b/jjb/lf-ci-job-groups.yaml
new file mode 100644 (file)
index 0000000..2dcff4c
--- /dev/null
+++ b/jjb/lf-ci-job-groups.yaml
@@ -0,0 +1,46 @@
+---
+- job-group:
+    name: '{project-name}-ci-jobs'
+
+    jobs:
+      - gerrit-jenkins-cfg-merge
+      - gerrit-jenkins-sandbox-cleanup
+      - gerrit-jjb-deploy-job
+      - gerrit-jjb-merge
+      - gerrit-jjb-verify
+
+- job-group:
+    name: '{project-name}-github-ci-jobs'
+
+    jobs:
+      - github-jenkins-cfg-merge
+      - github-jenkins-sandbox-cleanup
+      - github-jjb-deploy-job
+      - github-jjb-merge
+      - github-jjb-verify
+
+- job-group:
+    name: '{project-name}-info-yaml-jobs'
+
+    jobs:
+      - gerrit-info-yaml-verify
+
+- job-group:
+    name: '{project-name}-github-info-yaml-jobs'
+
+    jobs:
+      - github-info-yaml-verify
+
+- job-group:
+    name: '{project-name}-packer-jobs'
+
+    jobs:
+      - gerrit-packer-merge
+      - gerrit-packer-verify
+
+- job-group:
+    name: '{project-name}-github-packer-jobs'
+
+    jobs:
+      - github-packer-merge
+      - github-packer-verify

diff --git a/jjb/lf-ci-jobs.yaml b/jjb/lf-ci-jobs.yaml
index f9aff90..ec3e48e 100644 (file)
--- a/jjb/lf-ci-jobs.yaml
+++ b/jjb/lf-ci-jobs.yaml
@@ -1,49 +1,4 @@
 ---
-- job-group:
-    name: '{project-name}-ci-jobs'
-
-    jobs:
-      - gerrit-jenkins-cfg-merge
-      - gerrit-jenkins-sandbox-cleanup
-      - gerrit-jjb-deploy-job
-      - gerrit-jjb-merge
-      - gerrit-jjb-verify
-
-- job-group:
-    name: '{project-name}-github-ci-jobs'
-
-    jobs:
-      - github-jenkins-cfg-merge
-      - github-jenkins-sandbox-cleanup
-      - github-jjb-deploy-job
-      - github-jjb-merge
-      - github-jjb-verify
-
-- job-group:
-    name: '{project-name}-info-yaml-jobs'
-
-    jobs:
-      - gerrit-info-yaml-verify
-
-- job-group:
-    name: '{project-name}-github-info-yaml-jobs'
-
-    jobs:
-      - github-info-yaml-verify
-
-- job-group:
-    name: '{project-name}-packer-jobs'
-
-    jobs:
-      - gerrit-packer-merge
-      - gerrit-packer-verify
-
-- job-group:
-    name: '{project-name}-github-packer-jobs'
-
-    jobs:
-      - github-packer-merge
-      - github-packer-verify
 
 ####################
 # COMMON FUNCTIONS #

diff --git a/jjb/lf-maven-job-groups.yaml b/jjb/lf-maven-job-groups.yaml
new file mode 100644 (file)
index 0000000..f86b934
--- /dev/null
+++ b/jjb/lf-maven-job-groups.yaml
@@ -0,0 +1,45 @@
+---
+- job-group:
+    name: '{project-name}-maven-jobs'
+
+    # This job group contains all the recommended jobs that should be deployed
+    # for any project ci.
+
+    jobs:
+      - gerrit-maven-clm
+      - gerrit-maven-merge
+      - gerrit-maven-stage
+      - gerrit-maven-verify
+      - gerrit-maven-verify-dependencies
+
+- job-group:
+    name: '{project-name}-github-maven-jobs'
+
+    # This job group contains all the recommended jobs that should be deployed
+    # for any project ci that is using github.
+
+    jobs:
+      - github-maven-clm
+      - github-maven-merge
+      - github-maven-stage
+      - github-maven-verify
+
+- job-group:
+    name: '{project-name}-maven-javadoc-jobs'
+
+    # This job group contains all the recommended jobs that should be deployed
+    # for any project ci.
+
+    jobs:
+      - gerrit-maven-javadoc-publish
+      - gerrit-maven-javadoc-verify
+
+- job-group:
+    name: '{project-name}-github-maven-javadoc-jobs'
+
+    # This job group contains all the recommended jobs that should be deployed
+    # for any project ci.
+
+    jobs:
+      - github-maven-javadoc-publish
+      - github-maven-javadoc-verify

diff --git a/jjb/lf-maven-jobs.yaml b/jjb/lf-maven-jobs.yaml
index e879831..c89ba5b 100644 (file)
--- a/jjb/lf-maven-jobs.yaml
+++ b/jjb/lf-maven-jobs.yaml
@@ -1,51 +1,6 @@
 ---
 # This file contains job templates for Maven projects.
 
-- job-group:
-    name: '{project-name}-maven-jobs'
-
-    # This job group contains all the recommended jobs that should be deployed
-    # for any project ci.
-
-    jobs:
-      - gerrit-maven-clm
-      - gerrit-maven-merge
-      - gerrit-maven-stage
-      - gerrit-maven-verify
-      - gerrit-maven-verify-dependencies
-
-- job-group:
-    name: '{project-name}-github-maven-jobs'
-
-    # This job group contains all the recommended jobs that should be deployed
-    # for any project ci that is using github.
-
-    jobs:
-      - github-maven-clm
-      - github-maven-merge
-      - github-maven-stage
-      - github-maven-verify
-
-- job-group:
-    name: '{project-name}-maven-javadoc-jobs'
-
-    # This job group contains all the recommended jobs that should be deployed
-    # for any project ci.
-
-    jobs:
-      - gerrit-maven-javadoc-publish
-      - gerrit-maven-javadoc-verify
-
-- job-group:
-    name: '{project-name}-github-maven-javadoc-jobs'
-
-    # This job group contains all the recommended jobs that should be deployed
-    # for any project ci.
-
-    jobs:
-      - github-maven-javadoc-publish
-      - github-maven-javadoc-verify
-
 ##########
 # Macros #
 ##########

diff --git a/jjb/lf-node-job-groups.yaml b/jjb/lf-node-job-groups.yaml
new file mode 100644 (file)
index 0000000..a1db3b4
--- /dev/null
+++ b/jjb/lf-node-job-groups.yaml
@@ -0,0 +1,20 @@
+---
+- job-group:
+    name: '{project-name}-github-node-jobs'
+
+    # Job group containing recommended jobs to deploy for a Node Project.
+
+    node-version: 6.11.4
+
+    jobs:
+      - github-node-verify
+
+- job-group:
+    name: '{project-name}-node-jobs'
+
+    # Job group containing recommended jobs to deploy for a Node Project.
+
+    node-version: 6.11.4
+
+    jobs:
+      - gerrit-node-verify

diff --git a/jjb/lf-node-jobs.yaml b/jjb/lf-node-jobs.yaml
index 77eb800..c6bcc9b 100644 (file)
--- a/jjb/lf-node-jobs.yaml
+++ b/jjb/lf-node-jobs.yaml
@@ -1,23 +1,4 @@
 ---
-- job-group:
-    name: '{project-name}-github-node-jobs'
-
-    # Job group containing recommended jobs to deploy for a Node Project.
-
-    node-version: 6.11.4
-
-    jobs:
-      - github-node-verify
-
-- job-group:
-    name: '{project-name}-node-jobs'
-
-    # Job group containing recommended jobs to deploy for a Node Project.
-
-    node-version: 6.11.4
-
-    jobs:
-      - gerrit-node-verify
 
 ###############
 # Node Verify #

diff --git a/jjb/lf-python-job-groups.yaml b/jjb/lf-python-job-groups.yaml
new file mode 100644 (file)
index 0000000..3851a13
--- /dev/null
+++ b/jjb/lf-python-job-groups.yaml
@@ -0,0 +1,20 @@
+---
+- job-group:
+    name: '{project-name}-python-jobs'
+
+    # This job group contains all the recommended jobs that should be deployed
+    # for any project ci that is using Gerrit.
+
+    jobs:
+      - gerrit-python-xc-clm
+      - gerrit-tox-verify
+
+- job-group:
+    name: '{project-name}-github-python-jobs'
+
+    # This job group contains all the recommended jobs that should be deployed
+    # for any project ci that is using GitHub.
+
+    jobs:
+      - github-python-xc-clm
+      - github-tox-verify

diff --git a/jjb/lf-python-jobs.yaml b/jjb/lf-python-jobs.yaml
index ef4fc70..2c30a46 100644 (file)
--- a/jjb/lf-python-jobs.yaml
+++ b/jjb/lf-python-jobs.yaml
@@ -1,23 +1,4 @@
 ---
-- job-group:
-    name: '{project-name}-python-jobs'
-
-    # This job group contains all the recommended jobs that should be deployed
-    # for any project ci that is using Gerrit.
-
-    jobs:
-      - gerrit-python-xc-clm
-      - gerrit-tox-verify
-
-- job-group:
-    name: '{project-name}-github-python-jobs'
-
-    # This job group contains all the recommended jobs that should be deployed
-    # for any project ci that is using GitHub.
-
-    jobs:
-      - github-python-xc-clm
-      - github-tox-verify
 
 ##########
 # Macros #

diff --git a/jjb/lf-rtd-job-groups.yaml b/jjb/lf-rtd-job-groups.yaml
new file mode 100644 (file)
index 0000000..1a42a1b
--- /dev/null
+++ b/jjb/lf-rtd-job-groups.yaml
@@ -0,0 +1,14 @@
+---
+- job-group:
+    name: '{project-name}-rtd-jobs'
+
+    jobs:
+      - gerrit-rtd-merge
+      - gerrit-rtd-verify
+
+- job-group:
+    name: '{project-name}-github-rtd-jobs'
+
+    jobs:
+      - github-rtd-merge
+      - github-rtd-verify

diff --git a/jjb/lf-rtd-jobs.yaml b/jjb/lf-rtd-jobs.yaml
index 1640918..b7ddf61 100644 (file)
--- a/jjb/lf-rtd-jobs.yaml
+++ b/jjb/lf-rtd-jobs.yaml
@@ -1,17 +1,4 @@
 ---
-- job-group:
-    name: '{project-name}-rtd-jobs'
-
-    jobs:
-      - gerrit-rtd-merge
-      - gerrit-rtd-verify
-
-- job-group:
-    name: '{project-name}-github-rtd-jobs'
-
-    jobs:
-      - github-rtd-merge
-      - github-rtd-verify
 
 ####################
 # COMMON FUNCTIONS #