Variable expansion order of precedence seems to be:
-1. project section definition
-2. job-template variable definition
-3. defaults.yaml variable definition
+1. job-group section definition
+2. project section definition
+3. job-template variable definition
+4. defaults.yaml variable definition
.. note:: Defaults set variables in job-templates and are NOT used in Macros.
+global-jjb should not provide job-group definitions and leave it up to users of
+global-jjb to create their own as a job-group as a variable defined in a job
+group the highest precendence. Global JJB should strive to be purely a
+job-template and macro library for downstream consumers.
+
Final thoughts
--------------
to projects whom use the same build technology. The intention is to save time
for projects from having to define their own job templates.
+Release Notes
+-------------
+
+Global JJB provides regular releases. The release notes for all releases are
+available in the relnotes directory in Git.
+
+https://github.com/lfit/releng-global-jjb/tree/master/relnotes
+
Guides
------
* 2.30
* 2.31
+* 2.32
Cloud configuration are managed via a directory structure in ci-management as
follows:
ReadTheDocs Merge
-----------------
-Merge job which triggers a POST of the docs project to readthedocs
+Merge job which triggers a POST of the docs project to readthedocs. There is some setup
+required on read the docs to get started with this. After you have created the
+individual read the docs project (lets call it "PROJECT" here), then browse to
+https://readthedocs.org/dashboard/PROJECT/integrations/ and click on
+"Generic API incoming webhook". Here you will see a custom url to trigger the job as
+well as a token. Persist the token in the project Jenkins global-settings
+file. You will need to put it in a username/password credentials type to make
+it available in the global-settings file. set ```rtd-build-url to the
+custom build url as defined in read the docs. Also set the rtd-server-id
+to the id you specified in the global-settings file, this will enable the script to parse
+the token out. Set ```rtd-project``` to your read the docs project name.
+
:Template Names:
- {project-name}-rtd-merge-{stream}
:jenkins-ssh-credential: Credential to use for SSH. (Generally set
in defaults.yaml)
:rtd-project: This is the name of the project on ReadTheDocs.org.
+ :rtd-build-url: This is the generic webhook url from readthedocs.org
+ :global-settings-file: This is the location of the Jenkins global settings file.
+ This file contains the entry with the location for the readthedocs build token.
+ :rtd-server-id: This is the id of the entry in the global-settings-file.
+
:Optional parameters:
name: lf-rtd-trigger-build
builders:
- inject:
- properties-content: RTD_PROJECT={rtd-project}
+ properties-content: |
+ RTD_PROJECT={rtd-project}
+ RTD_BUILD_URL={rtd-build-url}
+ SETTINGS_FILE={global-settings-file}
+ SERVER_ID={rtd-server-id}
- shell: !include-raw-escape: ../shell/rtd-trigger-build.sh
- builder:
publishers:
- postbuildscript:
builders:
- - lf-infra-sysstat
- - lf-infra-ship-logs
- script-only-if-succeeded: false
- script-only-if-failed: false
- mark-unstable-if-failed: false
+ - role: BOTH
+ build-on:
+ - ABORTED
+ - FAILURE
+ - NOT_BUILT
+ - SUCCESS
+ - UNSTABLE
+ build-steps:
+ - lf-infra-sysstat
+ - lf-infra-ship-logs
+ mark-unstable-if-failed: true
- workspace-cleanup:
exclude:
# Do not clean up *.jenkins-trigger files for jobs that use a
######################
# Default parameters #
######################
+ rtd-project: ''
+ rtd-build-url: ''
+ global-settings-file: '$WORKSPACE/settings.xml'
+ rtd-server-id: ''
+
gerrit_trigger_file_paths:
- compare-type: ANT
builders:
- lf-rtd-trigger-build:
rtd-project: '{rtd-project}'
+ rtd-build-url: '{rtd-build-url}'
+ global-settings-file: '{global-settings-file}'
+ rtd-server-id: '{rtd-server-id}'
- job-template:
name: '{project-name}-rtd-merge-{stream}'
virtualenv --quiet "/tmp/v/git-review"
# shellcheck source=/tmp/v/git-review/bin/activate disable=SC1091
source "/tmp/v/git-review/bin/activate"
-pip install --quiet --upgrade pip
+pip install --quiet --upgrade pip setuptools
pip install --quiet --upgrade git-review
set -u
# End git-review workaround
flavors["v2-highcpu-32"]="21dfb8a3-c472-4a2c-a8e1-4da8de415ff8"
image_name=$(get_cfg "$cfg_file" IMAGE_NAME "")
+ volume_size=$(get_cfg "$cfg_file" VOLUME_SIZE "")
hardware_id=$(get_cfg "$cfg_file" HARDWARE_ID "")
network_id=$(get_cfg "$cfg_file" NETWORK_ID "")
user_data_id=$(get_cfg "$cfg_file" USER_DATA_ID "jenkins-init-script")
fs_root=$(get_cfg "$cfg_file" FS_ROOT "/w")
retention_time=$(get_cfg "$cfg_file" RETENTION_TIME "0")
- echo " new BootSource.Image(\"$image_name\"),"
+ if [ ! -z "$volume_size" ]; then
+ echo " new BootSource.VolumeFromImage(\"$image_name\", $volume_size),"
+ else
+ echo " new BootSource.Image(\"$image_name\"),"
+ fi
echo " \"${flavors[${hardware_id}]}\","
echo " \"$network_id\","
echo " \"$user_data_id\","
virtualenv "/tmp/v/jjb"
# shellcheck source=/tmp/v/jjb/bin/activate disable=SC1091
source "/tmp/v/jjb/bin/activate"
-pip install --quiet --upgrade pip
+pip install --quiet --upgrade pip setuptools
pip install --quiet --upgrade pipdeptree
pip install --quiet --upgrade "jenkins-job-builder==$JJB_VERSION"
virtualenv --quiet "/tmp/v/lftools"
# shellcheck source=/tmp/v/lftools/bin/activate disable=SC1091
source "/tmp/v/lftools/bin/activate"
-pip install --quiet --upgrade pip
+pip install --quiet --upgrade pip setuptools
case $LFTOOLS_MODE in
gerrit)
virtualenv --quiet "/tmp/v/python"
# shellcheck source=/tmp/v/node/bin/activate disable=SC1091
source "/tmp/v/python/bin/activate"
-pip install --quiet --upgrade pip
+pip install --quiet --upgrade pip setuptools
pip install --quiet --upgrade pipdeptree
pip install --quiet --upgrade nodeenv
virtualenv "/tmp/v/${PIP_PACKAGES%% *}"
# shellcheck source=/tmp/v/venv/bin/activate disable=SC1091
source "/tmp/v/${PIP_PACKAGES%% *}/bin/activate"
-pip install --quiet --upgrade pip
+pip install --quiet --upgrade pip setuptools
pip install --quiet --upgrade pipdeptree
# PIP_PACKAGES needs to be passed through as a space separated list of packages
RTD_BUILD_VERSION="${GERRIT_BRANCH/\//-}"
fi
-curl -X POST --data "version_slug=$RTD_BUILD_VERSION" "https://readthedocs.org/build/$RTD_PROJECT"
+CREDENTIAL=$(xmlstarlet sel -N "x=http://maven.apache.org/SETTINGS/1.0.0" \
+ -t -m "/x:settings/x:servers/x:server[x:id='${SERVER_ID}']" \
+ -v x:username -o ":" -v x:password \
+ "$SETTINGS_FILE")
+
+RTD_BUILD_TOKEN=$(echo "$CREDENTIAL" | cut -f2 -d:)
+
+curl -X POST -d "branches=$RTD_BUILD_VERSION" -d "token=$RTD_BUILD_TOKEN" "$RTD_BUILD_URL"
+
echo "---> Fetching project"
if [ "$GERRIT_PROJECT" != "$PROJECT" ]; then
+ # Only test projects that are a submodule of docs
+ if ! git submodule | grep "$GERRIT_PROJECT"; then
+ echo "WARN: Project is not a submodule of docs. This likely means " \
+ "the project is not participating in the monolithic docs build " \
+ "and should have their own verify job. Quitting job run..."
+ exit 0
+ fi
+
cd "docs/submodules/$GERRIT_PROJECT"
fi
virtualenv --quiet -p "$PYTHON_VERSION" "/tmp/v/tox"
# shellcheck source=/tmp/v/tox/bin/activate disable=SC1091
source "/tmp/v/tox/bin/activate"
-pip install --quiet --upgrade pip
+pip install --quiet --upgrade pip setuptools
pip install --quiet --upgrade pipdeptree
pip install --quiet --upgrade argparse detox tox tox-pyenv
[testenv:jjb]
deps =
- jenkins-job-builder==2.0.2
+ jenkins-job-builder>=2.0.3
commands =
jenkins-jobs -l DEBUG test --recursive -o {toxinidir}/archives/job-configs {toxinidir}/jjb:{toxinidir}/.jjb-test
Code Review / releng / global-jjb.git / commitdiff