RTD Allow projects to change their default-version

Allow docs release via jjb config
Changes the default landing page from /latest/ to /stable/

by default we se to /latest/ (a no-op)
And on release change to stable via jjb config.

read the docs, by design creates /latest/ and /stable/ landing points
for the user.

If read the docs sees a tag or a new branch, it creates a /stable/
directory, which follows the latest tag or if there is no tag, the
latest branch.

Related issue where I ask for clarification as to their default
behavior in this regards.
https://github.com/readthedocs/readthedocs.org/issues/6415

ISSUE: RELENG-2549
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I33a8df14e68340ba268c953a471daa02ee6fe766

diff --git a/docs/jjb/lf-rtdv3-jobs.rst b/docs/jjb/lf-rtdv3-jobs.rst
index 04e1953..8fe73f9 100644 (file)
--- a/docs/jjb/lf-rtdv3-jobs.rst
+++ b/docs/jjb/lf-rtdv3-jobs.rst
@@ -30,36 +30,73 @@ recent one under the title "stable."  For more details please see
 control this process using Jenkins job configuration parameters as
 discussed below.
 
 control this process using Jenkins job configuration parameters as
 discussed below.
 

-User setup:
+User setup
+----------

 
 

-To transform your rst documentation into a read the docs page, this job must be configured and
-created as described in Admin setup below. Once this is complete the following files must be
-added to your repository:
+To transform your rst documentation into a read the docs page, this job must be
+configured and created as described in Admin setup below. Once this is complete
+the following files must be added to your repository:

 
 .. code-block:: bash
 
 
 .. code-block:: bash
 

-    .readthedocs.yaml
-    tox.ini
-    docs
-    docs/_static
-    docs/_static/logo.png
-    docs/conf.yaml
-    docs/favicon.ico
-    docs/index.rst
-    docs/requirements-docs.txt
-    docs/conf.py
-
-Rather than have you copy and paste these files from a set of docs here, the following repo
-contains a script that will do this for you. Please refer to the explanation presented in:
-https://github.com/lfit-sandbox/test
-This is all currently a beta feature, so feedback is encouraged.
-the script `docs_script.sh` is not needed, you can copy the files by hand if you prefer
-
-Once these files are correctly configured in your repository you can test locally:
+   .readthedocs.yaml
+   tox.ini
+   docs
+   docs/_static
+   docs/_static/logo.png
+   docs/conf.yaml
+   docs/favicon.ico
+   docs/index.rst
+   docs/requirements-docs.txt
+   docs/conf.py
+
+Rather than have you copy and paste these files from a set of docs here, the
+following repo contains a script that will do this for you. Please refer to the
+explanation presented in: https://github.com/lfit-sandbox/test. This is all
+currently a beta feature, so feedback is encouraged. The script
+`docs_script.sh` is not needed, you can copy the files by hand if you prefer.
+
+Once these files are correctly configured in your repository you can test
+locally:

 
 .. code-block:: bash
 
 
 .. code-block:: bash
 

-    tox -e docs,docs-linkcheck
+   tox -e docs,docs-linkcheck
+
+
+Stable Branch Instructions
+--------------------------
+
+If your project does not create branches, you can skip this step.
+Once you branch your project modify your conf.yaml and add the following line:
+
+.. code-block:: bash
+
+   version: 'ReleaseBranchName'
+
+This will update the docs and change "master" on the top bar to your branch
+name. This change should be done against your release branch, this change will
+trigger a Read The Docs build which will create a new landing point for your
+documentation.
+
+This landing point is called /stable/ and is selectable as a version in the
+bottom right corner of all Read The Docs pages.  Once all projects have
+branched and modified their conf.py they will have available their /stable/
+documentation. The process to release the documentation (that is to change the
+default landing point of your docs from /latest/ to /stable/) is to change the
+default-version in the jenkins job config as follows:
+
+From:
+
+.. code-block:: bash
+
+   default-version: latest
+
+To:
+
+.. code-block:: bash
+
+   default-version: ReleaseBranchName

 
 
 Admin setup:
 
 
 Admin setup:


@@ -78,10 +115,13 @@ Normally this project is called doc or docs or documentation and all other docs
 be set as a subproject of this job.
 
 examples:
 be set as a subproject of this job.
 
 examples:

-global-vars-sandbox.sh:
-MASTER_RTD_PROJECT=doc-test
-global-vars-production.sh:
-MASTER_RTD_PROJECT=doc
+
+.. code-block:: bash
+
+   global-vars-sandbox.sh:
+   MASTER_RTD_PROJECT=doc-test
+   global-vars-production.sh:
+   MASTER_RTD_PROJECT=doc

 
 In this way sandbox jobs will create docs with a test suffix and will not stomp on production
 documentation.
 
 In this way sandbox jobs will create docs with a test suffix and will not stomp on production
 documentation.


@@ -92,43 +132,46 @@ example file: ci-management/jjb/rtd/rtd.yaml
 
 .. code-block:: bash
 
 
 .. code-block:: bash
 

-    ---
-    - project:
-        name: rtdv3-global-verify
-        build-node: centos7-builder-1c-1g
-        jobs:
-          - rtdv3-global-verify
-        stream:
-          - master:
-              branch: master
-          - foo:
-              branch: stable/{stream}
-
-    - project:
-        name: rtdv3-global-merge
-        build-node: centos7-builder-1c-1g
-        jobs:
-          - rtdv3-global-merge
-        stream:
-          - master:
-              branch: master
-          - foo:
-              branch: stable/{stream}
+   ---
+   - project:
+       name: rtdv3-global-verify
+       build-node: centos7-builder-1c-1g
+       default-version: latest
+       jobs:
+         - rtdv3-global-verify
+       stream:
+         - master:
+             branch: master
+         - foo:
+             branch: stable/{stream}
+
+   - project:
+       name: rtdv3-global-merge
+       default-version: latest
+       build-node: centos7-builder-1c-1g
+       jobs:
+         - rtdv3-global-merge
+       stream:
+         - master:
+             branch: master
+         - foo:
+             branch: stable/{stream}

 
 Or add both jobs via a job group:
 
 
 .. code-block:: bash
 
 
 Or add both jobs via a job group:
 
 
 .. code-block:: bash
 

-    ---
-    - project:
-        name: rtdv3-global
-        build-node: centos7-builder-1c-1g
-        jobs:
-          - rtdv3-global
-        stream:
-          - master:
-              branch: master
+   ---
+   - project:
+       name: rtdv3-global
+       default-version: latest
+       build-node: centos7-builder-1c-1g
+       jobs:
+         - rtdv3-global
+       stream:
+         - master:
+             branch: master

 
 
 Github jobs must be per project, and will be covered by a diffrent set of jobs once these are proven.
 
 
 Github jobs must be per project, and will be covered by a diffrent set of jobs once these are proven.


@@ -137,13 +180,14 @@ Job requires an lftools config section, this is to provide api access to read th
 
 .. code-block:: bash
 
 
 .. code-block:: bash
 

-    [rtd]
-    endpoint = https://readthedocs.org/api/v3/
-    token = [hidden]
+   [rtd]
+   endpoint = https://readthedocs.org/api/v3/
+   token = [hidden]

 
 Merge Job will create a project on read the docs if none exist.
 Merge Job will assign a project as a subproject of the master project.
 Merge job will trigger a build to update docs.
 
 Merge Job will create a project on read the docs if none exist.
 Merge Job will assign a project as a subproject of the master project.
 Merge job will trigger a build to update docs.

+Merge job will change the default version if needed.

 
 Macros
 ======
 
 Macros
 ======


@@ -179,9 +223,10 @@ Merge job which triggers a build of the docs to readthedocs.
     :branch: Git branch to fetch for the build. (default: master)
     :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
     :build-timeout: Timeout in minutes before aborting build. (default: 15)
     :branch: Git branch to fetch for the build. (default: master)
     :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
     :build-timeout: Timeout in minutes before aborting build. (default: 15)

-    :project-pattern: Project to trigger build against. (default: \*\*)
-    :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+    :default-version: default page to redirect to for documentation (default /latest/)

     :disable-job: Whether to disable the job (default: false)
     :disable-job: Whether to disable the job (default: false)

+    :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+    :project-pattern: Project to trigger build against. (default: \*\*)

     :stream: Keyword representing a release code-name.
         Often the same as the branch. (default: master)
     :submodule-recursive: Whether to checkout submodules recursively.
     :stream: Keyword representing a release code-name.
         Often the same as the branch. (default: master)
     :submodule-recursive: Whether to checkout submodules recursively.

diff --git a/jjb/lf-macros.yaml b/jjb/lf-macros.yaml
index 6984811..9411345 100644 (file)
--- a/jjb/lf-macros.yaml
+++ b/jjb/lf-macros.yaml
@@ -276,21 +276,11 @@
 
 - builder:
     name: lf-rtdv3-build
 
 - builder:
     name: lf-rtdv3-build

-

     builders:
     builders:

-      - conditional-step:
-          condition-kind: file-exists
-          condition-filename: .readthedocs.yaml
-          on-evaluation-failure: dont-run
-          steps:
-            - inject:
-                properties-content: |
-                  TOX_ENVS=docs,docs-linkcheck
-            - lf-infra-pre-build
-            - lf-infra-tox-install:
-                python-version: "python3"
-            - shell: !include-raw: ../shell/tox-run.sh
-            - shell: !include-raw: ../shell/rtdv3.sh
+      - inject:
+          properties-content: |
+            DEFAULT_VERSION={default-version}
+      - shell: !include-raw: ../shell/rtdv3.sh

 
 - builder:
     name: check-info-votes
 
 - builder:
     name: check-info-votes

diff --git a/jjb/lf-rtdv3-jobs.yaml b/jjb/lf-rtdv3-jobs.yaml
index a5054aa..ba03713 100644 (file)
--- a/jjb/lf-rtdv3-jobs.yaml
+++ b/jjb/lf-rtdv3-jobs.yaml
@@ -8,6 +8,7 @@
 
     branch: master
     stream: master
 
     branch: master
     stream: master

+    default-version: latest

     disabled: "{disable-job}"
     build-days-to-keep: 7
     build-timeout: 15
     disabled: "{disable-job}"
     build-days-to-keep: 7
     build-timeout: 15


@@ -49,11 +50,25 @@
       - lf-infra-publish
 
     builders:
       - lf-infra-publish
 
     builders:

-      - config-file-provider:
-          files:
-            - file-id: lftoolsini
-              target: "$HOME/.config/lftools/lftools.ini"
-      - lf-rtdv3-build
+      - conditional-step:
+          condition-kind: file-exists
+          condition-filename: .readthedocs.yaml
+          on-evaluation-failure: dont-run
+          steps:
+            - config-file-provider:
+                files:
+                  - file-id: lftoolsini
+                    target: "$HOME/.config/lftools/lftools.ini"
+            - lf-infra-pre-build
+            - lf-infra-tox-install:
+                python-version: "python3"
+            - inject:
+                properties-content: |
+                  TOX_ENVS=docs,docs-linkcheck
+            - lf-infra-tox-run:
+                parallel: "true"
+            - lf-rtdv3-build:
+                default-version: "{default-version}"

 
 - job-template:
     name: "rtdv3-global-verify-{stream}"
 
 - job-template:
     name: "rtdv3-global-verify-{stream}"


@@ -93,6 +108,8 @@
               branches:
                 - branch-compare-type: "ANT"
                   branch-pattern: "**/{branch}"
               branches:
                 - branch-compare-type: "ANT"
                   branch-pattern: "**/{branch}"

+                - branch-compare-type: "ANT"
+                  branch-pattern: "refs/tags/**"

               file-paths: "{obj:gerrit_trigger_file_paths}"
               forbidden-file-paths:
                 - compare-type: REG_EXP
               file-paths: "{obj:gerrit_trigger_file_paths}"
               forbidden-file-paths:
                 - compare-type: REG_EXP


@@ -148,4 +165,9 @@
               branches:
                 - branch-compare-type: "ANT"
                   branch-pattern: "**/{branch}"
               branches:
                 - branch-compare-type: "ANT"
                   branch-pattern: "**/{branch}"

+                - branch-compare-type: "ANT"
+                  branch-pattern: "refs/tags/**"

               file-paths: "{obj:gerrit_trigger_file_paths}"
               file-paths: "{obj:gerrit_trigger_file_paths}"

+              forbidden-file-paths:
+                - compare-type: REG_EXP
+                  pattern: ".*global-jjb.*"

diff --git a/releasenotes/notes/lf-rtdv3-jobs-b89edbf358caf261.yaml b/releasenotes/notes/lf-rtdv3-jobs-b89edbf358caf261.yaml
new file mode 100644 (file)
index 0000000..48b0c3b
--- /dev/null
+++ b/releasenotes/notes/lf-rtdv3-jobs-b89edbf358caf261.yaml
@@ -0,0 +1,12 @@
+---
+features:
+  - |
+    Project can now set their default docs landing page
+    used for self serve release of docs.
+    By default landing page is /latest/ on release landing page should
+    generally be set to /stable/
+upgrade:
+  - |
+    default-version is now an optional parameter
+    'latest' is the default behavior
+    change to 'stable' when docs are released

diff --git a/shell/rtdv3.sh b/shell/rtdv3.sh
index 3246767..29808af 100644 (file)
--- a/shell/rtdv3.sh
+++ b/shell/rtdv3.sh
@@ -12,7 +12,7 @@ echo "---> rtdv3.sh"
 set -euo pipefail
 
 project_dashed="${PROJECT////-}"
 set -euo pipefail
 
 project_dashed="${PROJECT////-}"

-umbrella="$(echo "$GERRIT_URL" | awk -F"." '{print $2}')"
+umbrella=$(echo "$GERRIT_URL" | awk -F'.' '{print $2}')

 if [[ "$SILO" == "sandbox" ]]; then
   rtdproject="$umbrella-$project_dashed-test"
 else
 if [[ "$SILO" == "sandbox" ]]; then
   rtdproject="$umbrella-$project_dashed-test"
 else


@@ -24,8 +24,8 @@ masterproject="$umbrella-$MASTER_RTD_PROJECT"
 
 echo "INFO:"
 echo "INFO: Project: $PROJECT"
 
 echo "INFO:"
 echo "INFO: Project: $PROJECT"

-echo "INFO: Read the Docs Project: https://$rtdproject.readthedocs.io"
-echo "INFO: Read the Docs master Project: https://$masterproject.readthedocs.io"
+echo "INFO: Read the Docs Sub Project: https://$rtdproject.readthedocs.io"
+echo "INFO: Read the Docs Master Project: https://$masterproject.readthedocs.io"

 
 
 if [[ "$JOB_NAME" =~ "verify" ]]; then
 
 
 if [[ "$JOB_NAME" =~ "verify" ]]; then


@@ -38,7 +38,7 @@ echo "INFO: Verify job completed"
 fi
 
 if [[ "$JOB_NAME" =~ "merge" ]]; then
 fi
 
 if [[ "$JOB_NAME" =~ "merge" ]]; then

-echo "INFO: Running merge job"
+echo "INFO: Performing merge action"

 
   # This retuns null if project exists.
   project_exists=false
 
   # This retuns null if project exists.
   project_exists=false


@@ -69,23 +69,45 @@ echo "INFO: Running merge job"
   if [[ "$rtdproject" != "$masterproject" ]]; then
     subproject_exists=false
     while read -r subproject; do
   if [[ "$rtdproject" != "$masterproject" ]]; then
     subproject_exists=false
     while read -r subproject; do

-        if [[ "$subproject" == "$rtdproject" ]]; then
-          subproject_exists=true
-          break
-        fi
+      if [[ "$subproject" == "$rtdproject" ]]; then
+        subproject_exists=true
+        break
+      fi

     done < <(lftools rtd subproject-list "$masterproject")
 
     if $subproject_exists; then
     done < <(lftools rtd subproject-list "$masterproject")
 
     if $subproject_exists; then

-      echo "INFO: subproject relationship already created"
+      echo "INFO: subproject $rtdproject relationship already created"

     else
     else

-      echo "INFO: Need to create subproject relationship"
+      echo "INFO: Creating subproject relationship"

       lftools rtd subproject-create "$masterproject" "$rtdproject"
       echo "INFO sleeping for 10 seconds"
       sleep 10
     fi
   fi
 
       lftools rtd subproject-create "$masterproject" "$rtdproject"
       echo "INFO sleeping for 10 seconds"
       sleep 10
     fi
   fi
 

-  # api v3 method does not update latest whith stream.
-  lftools rtd project-build-trigger "$rtdproject" "$STREAM"
-  lftools rtd project-build-trigger "$rtdproject" latest
+  # api v3 method does not update /latest/ when master is triggered.
+  # Also, when we build anything other than master we want to trigger /stable/ as well.
+  # allow projects to change their landing page from latest to branch_name
+
+  current_version="$(lftools rtd project-details "$rtdproject" | yq -r .default_version)"
+  if [[ -z ${DEFAULT_VERSION:-} ]]; then
+    echo "DEFAULT_VERSION (default-version) value cannot be empty"
+    exit 1
+  fi
+  default_version="${DEFAULT_VERSION}"
+
+  echo "INFO: current default version $current_version"
+  if [[ $current_version != "$default_version" ]]; then
+    echo "INFO: Setting rtd landing page to $default_version"
+    lftools rtd project-update "$rtdproject" default_version="$default_version"
+  fi
+
+  lftools rtd project-build-trigger "$rtdproject" "$GERRIT_BRANCH"
+  if [[ $GERRIT_BRANCH == "master" ]]; then
+    echo "INFO: triggering latest"
+    lftools rtd project-build-trigger "$rtdproject" latest
+  else
+    echo "INFO: triggering stable"
+    lftools rtd project-build-trigger "$rtdproject" stable
+  fi

 fi
 fi