From: Aric Gardner Date: Thu, 21 Nov 2019 21:32:18 +0000 (-0500) Subject: RTD Allow projects to change their default-version X-Git-Tag: v0.49.0~4^2 X-Git-Url: https://gerrit.linuxfoundation.org/infra/gitweb?a=commitdiff_plain;h=026509df164f628deb0f4660a172cca364ff58ae;p=releng%2Fglobal-jjb.git RTD Allow projects to change their default-version Allow docs release via jjb config Changes the default landing page from /latest/ to /stable/ by default we se to /latest/ (a no-op) And on release change to stable via jjb config. read the docs, by design creates /latest/ and /stable/ landing points for the user. If read the docs sees a tag or a new branch, it creates a /stable/ directory, which follows the latest tag or if there is no tag, the latest branch. Related issue where I ask for clarification as to their default behavior in this regards. https://github.com/readthedocs/readthedocs.org/issues/6415 ISSUE: RELENG-2549 Signed-off-by: Aric Gardner Change-Id: I33a8df14e68340ba268c953a471daa02ee6fe766 --- diff --git a/docs/jjb/lf-rtdv3-jobs.rst b/docs/jjb/lf-rtdv3-jobs.rst index 04e19535..8fe73f9b 100644 --- a/docs/jjb/lf-rtdv3-jobs.rst +++ b/docs/jjb/lf-rtdv3-jobs.rst @@ -30,36 +30,73 @@ recent one under the title "stable." For more details please see control this process using Jenkins job configuration parameters as discussed below. -User setup: +User setup +---------- -To transform your rst documentation into a read the docs page, this job must be configured and -created as described in Admin setup below. Once this is complete the following files must be -added to your repository: +To transform your rst documentation into a read the docs page, this job must be +configured and created as described in Admin setup below. Once this is complete +the following files must be added to your repository: .. code-block:: bash - .readthedocs.yaml - tox.ini - docs - docs/_static - docs/_static/logo.png - docs/conf.yaml - docs/favicon.ico - docs/index.rst - docs/requirements-docs.txt - docs/conf.py - -Rather than have you copy and paste these files from a set of docs here, the following repo -contains a script that will do this for you. Please refer to the explanation presented in: -https://github.com/lfit-sandbox/test -This is all currently a beta feature, so feedback is encouraged. -the script `docs_script.sh` is not needed, you can copy the files by hand if you prefer - -Once these files are correctly configured in your repository you can test locally: + .readthedocs.yaml + tox.ini + docs + docs/_static + docs/_static/logo.png + docs/conf.yaml + docs/favicon.ico + docs/index.rst + docs/requirements-docs.txt + docs/conf.py + +Rather than have you copy and paste these files from a set of docs here, the +following repo contains a script that will do this for you. Please refer to the +explanation presented in: https://github.com/lfit-sandbox/test. This is all +currently a beta feature, so feedback is encouraged. The script +`docs_script.sh` is not needed, you can copy the files by hand if you prefer. + +Once these files are correctly configured in your repository you can test +locally: .. code-block:: bash - tox -e docs,docs-linkcheck + tox -e docs,docs-linkcheck + + +Stable Branch Instructions +-------------------------- + +If your project does not create branches, you can skip this step. +Once you branch your project modify your conf.yaml and add the following line: + +.. code-block:: bash + + version: 'ReleaseBranchName' + +This will update the docs and change "master" on the top bar to your branch +name. This change should be done against your release branch, this change will +trigger a Read The Docs build which will create a new landing point for your +documentation. + +This landing point is called /stable/ and is selectable as a version in the +bottom right corner of all Read The Docs pages. Once all projects have +branched and modified their conf.py they will have available their /stable/ +documentation. The process to release the documentation (that is to change the +default landing point of your docs from /latest/ to /stable/) is to change the +default-version in the jenkins job config as follows: + +From: + +.. code-block:: bash + + default-version: latest + +To: + +.. code-block:: bash + + default-version: ReleaseBranchName Admin setup: @@ -78,10 +115,13 @@ Normally this project is called doc or docs or documentation and all other docs be set as a subproject of this job. examples: -global-vars-sandbox.sh: -MASTER_RTD_PROJECT=doc-test -global-vars-production.sh: -MASTER_RTD_PROJECT=doc + +.. code-block:: bash + + global-vars-sandbox.sh: + MASTER_RTD_PROJECT=doc-test + global-vars-production.sh: + MASTER_RTD_PROJECT=doc In this way sandbox jobs will create docs with a test suffix and will not stomp on production documentation. @@ -92,43 +132,46 @@ example file: ci-management/jjb/rtd/rtd.yaml .. code-block:: bash - --- - - project: - name: rtdv3-global-verify - build-node: centos7-builder-1c-1g - jobs: - - rtdv3-global-verify - stream: - - master: - branch: master - - foo: - branch: stable/{stream} - - - project: - name: rtdv3-global-merge - build-node: centos7-builder-1c-1g - jobs: - - rtdv3-global-merge - stream: - - master: - branch: master - - foo: - branch: stable/{stream} + --- + - project: + name: rtdv3-global-verify + build-node: centos7-builder-1c-1g + default-version: latest + jobs: + - rtdv3-global-verify + stream: + - master: + branch: master + - foo: + branch: stable/{stream} + + - project: + name: rtdv3-global-merge + default-version: latest + build-node: centos7-builder-1c-1g + jobs: + - rtdv3-global-merge + stream: + - master: + branch: master + - foo: + branch: stable/{stream} Or add both jobs via a job group: .. code-block:: bash - --- - - project: - name: rtdv3-global - build-node: centos7-builder-1c-1g - jobs: - - rtdv3-global - stream: - - master: - branch: master + --- + - project: + name: rtdv3-global + default-version: latest + build-node: centos7-builder-1c-1g + jobs: + - rtdv3-global + stream: + - master: + branch: master Github jobs must be per project, and will be covered by a diffrent set of jobs once these are proven. @@ -137,13 +180,14 @@ Job requires an lftools config section, this is to provide api access to read th .. code-block:: bash - [rtd] - endpoint = https://readthedocs.org/api/v3/ - token = [hidden] + [rtd] + endpoint = https://readthedocs.org/api/v3/ + token = [hidden] Merge Job will create a project on read the docs if none exist. Merge Job will assign a project as a subproject of the master project. Merge job will trigger a build to update docs. +Merge job will change the default version if needed. Macros ====== @@ -179,9 +223,10 @@ Merge job which triggers a build of the docs to readthedocs. :branch: Git branch to fetch for the build. (default: master) :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7) :build-timeout: Timeout in minutes before aborting build. (default: 15) - :project-pattern: Project to trigger build against. (default: \*\*) - :git-url: URL clone project from. (default: $GIT_URL/$PROJECT) + :default-version: default page to redirect to for documentation (default /latest/) :disable-job: Whether to disable the job (default: false) + :git-url: URL clone project from. (default: $GIT_URL/$PROJECT) + :project-pattern: Project to trigger build against. (default: \*\*) :stream: Keyword representing a release code-name. Often the same as the branch. (default: master) :submodule-recursive: Whether to checkout submodules recursively. diff --git a/jjb/lf-macros.yaml b/jjb/lf-macros.yaml index 69848114..94113458 100644 --- a/jjb/lf-macros.yaml +++ b/jjb/lf-macros.yaml @@ -276,21 +276,11 @@ - builder: name: lf-rtdv3-build - builders: - - conditional-step: - condition-kind: file-exists - condition-filename: .readthedocs.yaml - on-evaluation-failure: dont-run - steps: - - inject: - properties-content: | - TOX_ENVS=docs,docs-linkcheck - - lf-infra-pre-build - - lf-infra-tox-install: - python-version: "python3" - - shell: !include-raw: ../shell/tox-run.sh - - shell: !include-raw: ../shell/rtdv3.sh + - inject: + properties-content: | + DEFAULT_VERSION={default-version} + - shell: !include-raw: ../shell/rtdv3.sh - builder: name: check-info-votes diff --git a/jjb/lf-rtdv3-jobs.yaml b/jjb/lf-rtdv3-jobs.yaml index a5054aae..ba037137 100644 --- a/jjb/lf-rtdv3-jobs.yaml +++ b/jjb/lf-rtdv3-jobs.yaml @@ -8,6 +8,7 @@ branch: master stream: master + default-version: latest disabled: "{disable-job}" build-days-to-keep: 7 build-timeout: 15 @@ -49,11 +50,25 @@ - lf-infra-publish builders: - - config-file-provider: - files: - - file-id: lftoolsini - target: "$HOME/.config/lftools/lftools.ini" - - lf-rtdv3-build + - conditional-step: + condition-kind: file-exists + condition-filename: .readthedocs.yaml + on-evaluation-failure: dont-run + steps: + - config-file-provider: + files: + - file-id: lftoolsini + target: "$HOME/.config/lftools/lftools.ini" + - lf-infra-pre-build + - lf-infra-tox-install: + python-version: "python3" + - inject: + properties-content: | + TOX_ENVS=docs,docs-linkcheck + - lf-infra-tox-run: + parallel: "true" + - lf-rtdv3-build: + default-version: "{default-version}" - job-template: name: "rtdv3-global-verify-{stream}" @@ -93,6 +108,8 @@ branches: - branch-compare-type: "ANT" branch-pattern: "**/{branch}" + - branch-compare-type: "ANT" + branch-pattern: "refs/tags/**" file-paths: "{obj:gerrit_trigger_file_paths}" forbidden-file-paths: - compare-type: REG_EXP @@ -148,4 +165,9 @@ branches: - branch-compare-type: "ANT" branch-pattern: "**/{branch}" + - branch-compare-type: "ANT" + branch-pattern: "refs/tags/**" file-paths: "{obj:gerrit_trigger_file_paths}" + forbidden-file-paths: + - compare-type: REG_EXP + pattern: ".*global-jjb.*" diff --git a/releasenotes/notes/lf-rtdv3-jobs-b89edbf358caf261.yaml b/releasenotes/notes/lf-rtdv3-jobs-b89edbf358caf261.yaml new file mode 100644 index 00000000..48b0c3b2 --- /dev/null +++ b/releasenotes/notes/lf-rtdv3-jobs-b89edbf358caf261.yaml @@ -0,0 +1,12 @@ +--- +features: + - | + Project can now set their default docs landing page + used for self serve release of docs. + By default landing page is /latest/ on release landing page should + generally be set to /stable/ +upgrade: + - | + default-version is now an optional parameter + 'latest' is the default behavior + change to 'stable' when docs are released diff --git a/shell/rtdv3.sh b/shell/rtdv3.sh index 3246767f..29808afa 100644 --- a/shell/rtdv3.sh +++ b/shell/rtdv3.sh @@ -12,7 +12,7 @@ echo "---> rtdv3.sh" set -euo pipefail project_dashed="${PROJECT////-}" -umbrella="$(echo "$GERRIT_URL" | awk -F"." '{print $2}')" +umbrella=$(echo "$GERRIT_URL" | awk -F'.' '{print $2}') if [[ "$SILO" == "sandbox" ]]; then rtdproject="$umbrella-$project_dashed-test" else @@ -24,8 +24,8 @@ masterproject="$umbrella-$MASTER_RTD_PROJECT" echo "INFO:" echo "INFO: Project: $PROJECT" -echo "INFO: Read the Docs Project: https://$rtdproject.readthedocs.io" -echo "INFO: Read the Docs master Project: https://$masterproject.readthedocs.io" +echo "INFO: Read the Docs Sub Project: https://$rtdproject.readthedocs.io" +echo "INFO: Read the Docs Master Project: https://$masterproject.readthedocs.io" if [[ "$JOB_NAME" =~ "verify" ]]; then @@ -38,7 +38,7 @@ echo "INFO: Verify job completed" fi if [[ "$JOB_NAME" =~ "merge" ]]; then -echo "INFO: Running merge job" +echo "INFO: Performing merge action" # This retuns null if project exists. project_exists=false @@ -69,23 +69,45 @@ echo "INFO: Running merge job" if [[ "$rtdproject" != "$masterproject" ]]; then subproject_exists=false while read -r subproject; do - if [[ "$subproject" == "$rtdproject" ]]; then - subproject_exists=true - break - fi + if [[ "$subproject" == "$rtdproject" ]]; then + subproject_exists=true + break + fi done < <(lftools rtd subproject-list "$masterproject") if $subproject_exists; then - echo "INFO: subproject relationship already created" + echo "INFO: subproject $rtdproject relationship already created" else - echo "INFO: Need to create subproject relationship" + echo "INFO: Creating subproject relationship" lftools rtd subproject-create "$masterproject" "$rtdproject" echo "INFO sleeping for 10 seconds" sleep 10 fi fi - # api v3 method does not update latest whith stream. - lftools rtd project-build-trigger "$rtdproject" "$STREAM" - lftools rtd project-build-trigger "$rtdproject" latest + # api v3 method does not update /latest/ when master is triggered. + # Also, when we build anything other than master we want to trigger /stable/ as well. + # allow projects to change their landing page from latest to branch_name + + current_version="$(lftools rtd project-details "$rtdproject" | yq -r .default_version)" + if [[ -z ${DEFAULT_VERSION:-} ]]; then + echo "DEFAULT_VERSION (default-version) value cannot be empty" + exit 1 + fi + default_version="${DEFAULT_VERSION}" + + echo "INFO: current default version $current_version" + if [[ $current_version != "$default_version" ]]; then + echo "INFO: Setting rtd landing page to $default_version" + lftools rtd project-update "$rtdproject" default_version="$default_version" + fi + + lftools rtd project-build-trigger "$rtdproject" "$GERRIT_BRANCH" + if [[ $GERRIT_BRANCH == "master" ]]; then + echo "INFO: triggering latest" + lftools rtd project-build-trigger "$rtdproject" latest + else + echo "INFO: triggering stable" + lftools rtd project-build-trigger "$rtdproject" stable + fi fi