In the case of template definitions of a parameter below is not passed
the one defined in default clouds will be inherited.
- :IMAGE_NAME: The image name to use for this template.
- (required)
+ :IMAGE_NAME: The image name to use for this template. (required)
:HARDWARE_ID: OpenStack flavor to use. (required)
:LABELS: Labels to assign to the vm. (default: FILE_NAME)
+ :VOLUME_SIZE: Volume size to assign to vm. (default: "")
+ :HARDWARE_ID: Hardware Id to assign to vm. (default: "")
:NETWORK_ID: OpenStack network to use. (default: "")
:USER_DATA_ID: User Data to pass into the instance.
(default: jenkins-init-script)
:START_TIMEOUT: Number of milliseconds to wait for the agent to be
provisioned and connected. (default: 600000)
:KEY_PAIR_NAME: SSH Public Key Pair to use for authentication.
- (default: jenkins)
+ (default: jenkins-ssh)
:NUM_EXECUTORS: Number of executors to enable for the instance.
(default: 1)
- :JVM_OPTIONS: JVM Options to pass to Java. (default: "")
+ :JVM_OPTIONS: JVM Options to pass to Java. (default: null)
:FS_ROOT: File system root for the workspace. (default: "/w")
+ :NODE_PROPERTIES: Node properties. (default: null)
:RETENTION_TIME: Number of minutes to wait for an idle slave to be used
again before it's removed. If set to -1, the slave will be kept
forever. (default: 0)
:CONNECTION_TYPE: The connection type for Jenkins to connect to the build
minion. Valid options: JNLP, SSH. (default: "SSH")
+ :CONFIG_TYPE: Configuration drive. (default: null)
For a live example see the OpenDaylight project jenkins-config directory.
https://github.com/opendaylight/releng-builder/tree/master/jenkins-config
This template supports Maven and Container release jobs.
+This template uses a git commit choosing strategy that builds the merged
+commit with the release yaml file, not the tip of the target branch, so
+projects can repeat the release action in case of merge job failure.
+
:Template Name: {project-name}-release-merge
:Comment Trigger: remerge
signs the tag and pushes the tag to the git server. The release merge
template accepts neither a branch nor a stream parameter.
+These templates use a git commit choosing strategy that builds the merged
+commit with the release yaml file, not the tip of the target branch, so
+projects can repeat the release action in case of merge job failure.
+
:Template Names:
- {project-name}-pypi-release-merge
PackageCloud Release Verify
~~~~~~~~~~~~~~~~~~~~~~~~~~~
-This template supports PackageCloud release jobs.
+This template supports PackageCloud release jobs. Checks that the specified
+packages are present in the staging repository and absent from the release
+repository.
:Template Name: {project-name}-packagecloud-release-verify
PackageCloud Release Merge
~~~~~~~~~~~~~~~~~~~~~~~~~~
-This template supports PackageCloud release jobs.
+This template supports PackageCloud release jobs. Promotes the specified
+packages from the staging repository to the release repository.
+
+This template uses a git commit choosing strategy that builds the merged
+commit with the release yaml file, not the tip of the target branch, so
+projects can repeat the release action in case of merge job failure.
:template name: {project-name}-packagecloud-release-merge
submodule-recursive: "{submodule-recursive}"
submodule-timeout: "{submodule-timeout}"
submodule-disable: "{submodule-disable}"
- choosing-strategy: gerrit
+ # merge jobs always build from tip
+ choosing-strategy: default
triggers:
- gerrit:
submodule-recursive: "{submodule-recursive}"
submodule-timeout: "{submodule-timeout}"
submodule-disable: "{submodule-disable}"
- choosing-strategy: gerrit
+ # merge jobs always build from tip
+ choosing-strategy: default
triggers:
- timed: "{obj:cron}"
submodule-disable: true
submodule-recursive: false
submodule-timeout: 10
- choosing-strategy: default
+ # release merge jobs build from commit not tip
+ choosing-strategy: gerrit
triggers:
- gerrit:
submodule-recursive: "{submodule-recursive}"
submodule-timeout: "{submodule-timeout}"
submodule-disable: "{submodule-disable}"
+ # release merge jobs build from commit not tip
choosing-strategy: gerrit
triggers:
submodule-disable: true
submodule-recursive: false
submodule-timeout: 10
- choosing-strategy: default
+ # release merge jobs build from commit not tip
+ choosing-strategy: gerrit
triggers:
- gerrit:
--- /dev/null
+---
+fixes:
+ - |
+ A recent change has made the "null" string a bad value for FLOATING_IP_POOL.
+ By making it an empty string, we recreate the old functionality of having
+ the default floating IP pool set to "No value".
--- /dev/null
+---
+fixes:
+ - |
+ Branch discovery and build polling implemented.
+ If a Branch has not been seen by rtd
+ we trigger a build with rtd and poll till that build
+ is complete.
+ we can then enable the branch and trigger a build
+ against it, again polling all builds untill they are
+ complete.
--- /dev/null
+---
+fixes:
+ - |
+ Use git choosing strategy default in tox and pypi merge jobs for
+ gerrit. This makes those jobs consistent with maven and other
+ merge jobs for gerrit that always build from tip of the target
+ branch to create artifacts from the latest & greatest code.
+ Building from tip (not from Gerrit commit/merge point) avoids
+ confusion about content when changes are merged out of order.
+ For example, a fix is submitted and merged, but the merge job
+ fails. In the mean time, a different change that happened
+ earlier in commit history gets merged (or the merge job is
+ retriggered), causing a new artifact to be pushed. But that
+ artifact does not have the expected fix.
+
+ Add comments to release merge jobs why their choosing strategy
+ is not default.
+
+ Document the git commit choosing strategy for the release merge jobs.
--- /dev/null
+---
+fixes:
+ - |
+ Extend release-job.sh to detect if distribution_type is missing
+ from the release yaml file and show a meaningful error. The
+ shell option pipefile causes the script to halt silently if
+ niet fails to find that key, which utterly baffles users.
--- /dev/null
+---
+fixes:
+ - |
+ Use choosing strategy Gerrit Trigger in container/jar and
+ package cloud release merge jobs. This retains the current
+ behavior in the simple merge case, and ensures that a job
+ triggered by a "remerge" comment uses the release file at
+ that commit. The previous choosing strategy, default, uses
+ the tip of the target branch. That does not allow recovery
+ from merge job failure if the target branch has advanced past
+ the commit with the release file.
instance_cap=$(get_cfg "$cfg_file" INSTANCE_CAP "null")
fi
- floating_ip_pool=$(get_cfg "$cfg_file" FLOATING_IP_POOL "null")
+ floating_ip_pool=$(get_cfg "$cfg_file" FLOATING_IP_POOL "")
security_groups=$(get_cfg "$cfg_file" SECURITY_GROUPS "default")
availability_zone=$(get_cfg "$cfg_file" AVAILABILITY_ZONE "")
start_timeout=$(get_cfg "$cfg_file" START_TIMEOUT "600000")
kpn_default="$(get_cfg "$(dirname "$cfg_file")/cloud.cfg" KEY_PAIR_NAME "jenkins-ssh")"
key_pair_name=$(get_cfg "$cfg_file" KEY_PAIR_NAME "$kpn_default")
num_executors=$(get_cfg "$cfg_file" NUM_EXECUTORS "1")
- jvm_options=$(get_cfg "$cfg_file" JVM_OPTIONS "null")
+ jvm_options=$(get_cfg "$cfg_file" JVM_OPTIONS "")
fs_root=$(get_cfg "$cfg_file" FS_ROOT "/w")
connection_type=$(get_cfg "$cfg_file" CONNECTION_TYPE "SSH")
launcher_factory=$(get_launcher_factory "$connection_type")
# which accompanies this distribution, and is available at
# http://www.eclipse.org/legal/epl-v10.html
##############################################################################
-echo "---> build-cost.sh"
+echo "---> job-cost.sh"
set -euf -o pipefail
# shellcheck disable=SC1090
source ~/lf-env.sh
+# AWS job cost not supported, exit
+if grep -qi amazon /sys/devices/virtual/dmi/id/bios_vendor ; then
+ echo "INFO: Not able to calculate job cost on AWS"
+ exit 0
+fi
+
lf-activate-venv python-openstackclient
if [[ -z ${JOB_NAME:-} ]]; then
# Convert to integer by truncating fractional part' and round up by one
((uptime=${uptime%\.*}+1))
+# EC2 and OpenStack have simiar instace metadata APIs at this IP
+# AWS docs: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html
+# Nova docs: https://docs.openstack.org/nova/latest/user/metadata.html
instance_type=$(curl -s http://169.254.169.254/latest/meta-data/instance-type)
echo "INFO: Retrieving Pricing Info for: $instance_type"
url="https://pricing.vexxhost.net/v1/pricing/$instance_type/cost?seconds=$uptime"
json_block=$(curl -s "$url")
-cost=$(jq .cost <<< "$json_block")
-resource=$(jq .resource <<< "$json_block" | tr -d '"')
+# check if JSON returned and can be parsed
+if jq <<< "$json_block"; then
+ cost=$(jq .cost <<< "$json_block")
+ resource=$(jq .resource <<< "$json_block" | tr -d '"')
+else
+ echo "ERROR: Pricing API returned invalid json"
+ cost=0
+ resource=0
+fi
# Archive the cost date
mkdir -p "$WORKSPACE/archives/cost"
cat << EOF > "$WORKSPACE/archives/cost.csv"
$JOB_NAME,$BUILD_NUMBER,$date,$resource,$uptime,$cost,$stack_cost
EOF
-
pipup="python -m pip install -q --upgrade pip lftools jsonschema niet twine yq"
echo "INFO: $pipup"
$pipup
+# show installed versions
+python -m pip --version
+python -m pip freeze
#Functions.
fi
# Jenkins parameter drop-down defaults DISTRIBUTION_TYPE to None
+ # in the contain/maven release job; get value from release yaml.
+ # Packagecloud and PyPI jobs set the appropriate value.
DISTRIBUTION_TYPE="${DISTRIBUTION_TYPE:-None}"
if [[ $DISTRIBUTION_TYPE == "None" ]]; then
- DISTRIBUTION_TYPE=$(niet ".distribution_type" "$release_file")
+ if ! DISTRIBUTION_TYPE=$(niet ".distribution_type" "$release_file"); then
+ echo "ERROR: Failed to get distribution_type from $release_file"
+ exit 1
+ fi
fi
PATCH_DIR=$(mktemp -d)
echo "---> rtdv3.sh"
set -euo pipefail
+watchbuild(){
+ echo "INFO: Running build against branch $1"
+ local buildid
+ local result
+ buildid=$(lftools rtd project-build-trigger "$rtdproject" "$1" | jq '.build.id')
+
+ result=null
+ while [ $result == null ]; do
+ sleep 10
+ result=$(lftools rtd project-build-details "$rtdproject" "$buildid" | jq '.success')
+ echo "INFO Current result of running build $result"
+ if [[ $result == failed ]]; then
+ echo "INFO: read the docs build completed with status: $result"
+ exit 1
+ fi
+ done
+ echo "INFO: read the docs build completed with status: $result"
+}
+
project_dashed="${PROJECT////-}"
umbrella=$(echo "$GERRIT_URL" | awk -F'.' '{print $2}')
if [[ "$SILO" == "sandbox" ]]; then
lftools rtd project-update "$rtdproject" default_version="$default_version"
fi
- lftools rtd project-build-trigger "$rtdproject" "$GERRIT_BRANCH"
if [[ $GERRIT_BRANCH == "master" ]]; then
- echo "INFO: triggering latest"
- lftools rtd project-build-trigger "$rtdproject" latest
+ echo "INFO: triggering $rtdproject latest"
+ watchbuild latest
else
- echo "INFO: triggering stable"
- lftools rtd project-build-trigger "$rtdproject" stable
+
+ #read the docs only understands lower case branch names
+ branch=$(echo "$GERRIT_BRANCH" | tr '[:upper:]' '[:lower:]')
+ echo "INFO: Checking if read the docs has seen branch $branch"
+
+ #if this is 404. then run discover branch
+ if ! lftools rtd project-version-details "$rtdproject" "$branch" | jq '.active'; then
+ echo "INFO: read the docs has not seen branch $branch for project $rtdproject"
+ echo "INFO: triggering $rtdproject latest to instantiate new branch discovery"
+ watchbuild latest
+ fi
+
+ echo "INFO: triggering $rtdproject $branch"
+ watchbuild "$branch"
+
+ #Make newly discovered branches visible in the u/i
+ isactive=$(lftools rtd project-version-details "$rtdproject" "$branch" | jq '.active')
+ if [[ "$isactive" == false ]]; then
+ echo "INFO: Marking $branch as active for project $rtdproject"
+ lftools rtd project-version-update "$rtdproject" "$branch" true
+ fi
+
fi
fi
Code Review / releng / global-jjb.git / commitdiff