Aric Gardner [Thu, 26 Sep 2019 17:12:39 +0000 (13:12 -0400)]
RTD builds, python version should be python3
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ib44e4291a88be05983d8df3bbf389403ad1a329e
Eric Ball [Wed, 25 Sep 2019 22:00:27 +0000 (15:00 -0700)]
Change Packer version back to 1.4.2
Packer 1.4.3 frequently has issues setting metadata after a build
completes.
There is a WIP change[0] to upgrade to 1.4.4, but per notes there,
it will not be available until October. Until then, the version
should be downgraded instead, as all Packer builds with 1.4.3 are
failing.
0: https://gerrit.linuxfoundation.org/infra/c/releng/global-jjb/+/61742
Change-Id: Ideb5aa607a9fccbd319e4638413414c5cb4c8a23
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Aric Gardner [Wed, 25 Sep 2019 18:35:57 +0000 (18:35 +0000)]
Merge "Revert "Fix rtd verify job choosing strategy""
Tim Johnson [Mon, 9 Sep 2019 17:51:54 +0000 (10:51 -0700)]
Capture sudo log on builders
Created new script sudo-logs.sh to archive the sudo logs
Issue: RELENG-1994
Change-Id: I86a44e6abf18e35c7dc0e1840797e0dde5c764fb
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
Chris Lott [Wed, 25 Sep 2019 16:12:51 +0000 (16:12 +0000)]
Revert "Fix rtd verify job choosing strategy"
This reverts commit
c02c7a2ac05e3a9b987edc15d555a5124a2ad976.
Reason for revert: Acumos RTD jobs are triggered by commits to ANY Acumos repo on a *.rst file. This change causes the doc job to checkout the commit ID from that other repo, which is never present in the doc repo.
Issue-ID: RELENG-911
Change-Id: Icab10bb8b0e3e3cc28458751a9de1edf88fdc3d7
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
Jessica Wagantall [Wed, 25 Sep 2019 10:10:53 +0000 (10:10 +0000)]
Merge "Lift PyPI templates to Global-JJB from Acumos"
Aric Gardner [Mon, 23 Sep 2019 17:27:05 +0000 (13:27 -0400)]
Move python version to python3
Remove pin workarounds for python2
install tox via python3 pip
so that the version of tox is run from
/home/jenkins/.local/lib/python3.6/site-packages/tox/__init__.py
fix release-job to use venv
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I4b2d778f3fd81565c5dd009d50c969696faba0d2
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
Lott, Christopher (cl778h) [Wed, 21 Aug 2019 14:54:25 +0000 (10:54 -0400)]
Lift PyPI templates to Global-JJB from Acumos
New templates provide features to build and push Python
source and binary distributions to a PyPI repository.
Check version string and push a tag on release.
New job groups:
{project-name}-gerrit-pypi-jobs
{project-name}-github-pypi-jobs
New templates:
{project-name}-pypi-verify-{stream}
gerrit-pypi-verify
github-pypi-verify
{project-name}-pypi-merge-{stream}
gerrit-pypi-merge
github-pypi-merge
{project-name}-pypi-release-verify-{stream}
gerrit-pypi-release-verify
github-pypi-release-verify
{project-name}-pypi-release-merge-{stream}
gerrit-pypi-release-merge
github-pypi-release-merge
Change-Id: If74925623f2a5804de8684b9a9933573dfea29a1
Issue-Id: RELENG-2317
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
Tim Johnson [Fri, 20 Sep 2019 22:21:31 +0000 (15:21 -0700)]
Global JJB Fatal Shellcheck Errors
Fix all tox warnings/errors
Issue: RELENG-1214
Change-Id: Ic4c1743828dbe4a230eaf2950c03d8c7fbfaf458
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
Bengt Thuree [Wed, 18 Sep 2019 08:56:15 +0000 (10:56 +0200)]
Change sonar trigger to run-sonar
It was stage-release, and with this change it
will be similar to other jobs.
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Change-Id: I4c458979ec696c7d368bf410dabaa43ee7abb2f7
Thanh Ha [Thu, 19 Sep 2019 16:17:37 +0000 (12:17 -0400)]
Set DISTRIBUTION_TYPE to choice
As for as I can tell from the release_job.sh file there are only 3
valid options "None" as the default and choices are container or maven.
Set parameter to choices to help the user of the job select the right
option.
Change-Id: I587d63103011b9f42252a58ce3a79c44b46ceefd
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Jessica Wagantall [Tue, 17 Sep 2019 23:56:24 +0000 (23:56 +0000)]
Merge "Add sonar-prescan jobs"
Jessica Wagantall [Tue, 17 Sep 2019 17:02:58 +0000 (17:02 +0000)]
Merge "Allow java opts in WS scans"
Jessica Wagantall [Mon, 16 Sep 2019 23:47:54 +0000 (16:47 -0700)]
Allow java opts in WS scans
Allow java options for WhiteSource
Unified Agent calls
Issue: RELENG-2387
Change-Id: I4b0d717a38448e1a8ced1ba63416c1b0a92d26e2
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Aric Gardner [Mon, 16 Sep 2019 23:30:54 +0000 (23:30 +0000)]
Merge "Fix rtd verify job choosing strategy"
Houa Yang [Fri, 13 Sep 2019 16:03:35 +0000 (11:03 -0500)]
Fix rtd verify job choosing strategy
Change default setting of choosing strategy from default to gerrit
for rtd verify job. If the stratgey is not gerrit, you will need
to manually set the choosing strategy to gerrit in the jenkins job
before you can build using a specific parameter refspec.
Issue-Id:RELENG-911
Signed-off-by: Houa Yang <hyang@contractor.linuxfoundation.org>
Change-Id: If5f10f83048b907123bf423394ef0adf85607806
Anil Belur [Mon, 16 Sep 2019 17:18:02 +0000 (22:48 +0530)]
Add template to update OpenStack cloud images
This job finds and updates OpenStack cloud images on the
ci-management source repository.
The job is triggered in two ways:
1. When a packer merge job completes, the new image name created is
passed down to the job.
2. Manually trigger the job to update all images.
When the job is triggered through an upstream packer merge job, this
only generates a change request for the new image built.
When the job is triggered manually, this job finds the latest
images on OpenStack cloud and compares them with the images currently
used in the source ci-management source repository. If the compared
images have newer time stamps are **all** updated through a change
request.
This job requires a Jenkins configuration merge and verify job setup and
working on Jenkins.
Issue: RELENG-2352
Change-Id: I2c75407bbe9d6866ded492310b8d34b47c3a1341
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Aric Gardner [Mon, 16 Sep 2019 15:32:36 +0000 (11:32 -0400)]
pyenv pick latest installed version
Logic to pick latest version of pyev.
Builders may have diffrent versions installed.
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I2eb5cc75d815778bb35f166d1512711705b6ccd7
Jessica Wagantall [Fri, 13 Sep 2019 19:47:11 +0000 (12:47 -0700)]
Update WhiteSource scan timer
Run WhiteSource jobs weekly on Sunday.
Issue: RELENG-2386
Change-Id: I26d5b4a89e262ff2b046bdae8298e61597b71bc1
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Eric Ball [Fri, 13 Sep 2019 01:26:06 +0000 (18:26 -0700)]
Add sonar-prescan jobs
These jobs allow the caller to provide a builder that runs prior to
the Sonar scan.
Issue: RELENG-2314
Change-Id: I97ec30b27c83a7ce1ac7dc8a1736672dc8d15eee
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Andrew Grimberg [Fri, 13 Sep 2019 17:25:12 +0000 (17:25 +0000)]
Merge "Update release docs."
Aric Gardner [Mon, 9 Sep 2019 20:32:05 +0000 (16:32 -0400)]
Update release docs.
Add note about adding env variables to global-vars-releng.sh
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I3596434d7601829fdd92f27ce0646006452a4c89
Aric Gardner [Fri, 13 Sep 2019 15:31:00 +0000 (11:31 -0400)]
Pin more-itertools for python2
Pin more-itertools for python2 tox-install
tox-pyenv requires more-itertools.
Force to a compatible version if pip installed
via python2
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ia5ade84925cdc6eac2f3407cb732eaf4946769e4
Anil Belur [Fri, 13 Sep 2019 07:28:33 +0000 (12:58 +0530)]
Pin python-cinderclient to 4.3.0.
A new version of python-cinderclient 5.0.0 is released which breaks
the openstack jobs.
ImportError: No module named v1.contrib
Issue: IT-17567
Change-Id: I2736c464ff82a3b08433031a5e924e1345c2c317
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Tim Johnson [Thu, 12 Sep 2019 22:58:33 +0000 (15:58 -0700)]
Cleanup tox warnings
Issue: RELENG-2365
Change-Id: I5a739a4446bfeabf73d020e6f72a8440b9d1c913
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
Jessica Wagantall [Wed, 11 Sep 2019 23:28:17 +0000 (16:28 -0700)]
Update release container info
- Update schema to require pull and push container
registries
- This will fix issues where for example ONAP pulls from
snapshots exclusively where other projects like O-RAN
use both snapshots or staging.
- Allows for future push to DockerHub
- Benefits projects like O-RAN and Acumos where they already
provide this information for other docker jobs.
- Pull and push registries need to be defined as Jenkins
global variables.
- Can be overwritten in releases files.
- Update some echos to allow users to find information faster.
Change-Id: Ibd0088c6419bacca16ded4ae86d70304c5bad45c
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Jessica Wagantall [Thu, 12 Sep 2019 18:22:42 +0000 (11:22 -0700)]
Fix WS CLI Agent calls
Remove quotes from optional variable
WSS_UNIFIED_AGENT_OPTIONS which, if empty, will
cause WS Unified Agent CLI issues.
Upgrade WS Unified Agent CLI to latest version 19.8.1
Change-Id: I3d21bad21bceee0a0c626b145c1ca74e0cd21ddc
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Jessica Wagantall [Wed, 11 Sep 2019 20:17:37 +0000 (13:17 -0700)]
Fix import key release jobs
Import GPG signing key before verifying Gerrit
tag details.
Change-Id: I132fb8dbba51de995b0e42765344bf218340415c
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Aric Gardner [Wed, 11 Sep 2019 15:22:00 +0000 (15:22 +0000)]
Merge "Add python templates to run tox on merge"
Lott, Christopher (cl778h) [Tue, 20 Aug 2019 15:46:31 +0000 (11:46 -0400)]
Add python templates to run tox on merge
Makes the Python job groups and templates parallel to
the maven job groups and templates. Although no build
is required by most Python projects, re-testing after
merge is a chance to reveal failures in the new tree.
Change-Id: I1fb793e08deb435a838d76d634dbbad730a3d7fa
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
Aric Gardner [Wed, 11 Sep 2019 11:52:55 +0000 (11:52 +0000)]
Merge "Use configured python version to install tox-pyenv"
Lott, Christopher (cl778h) [Mon, 9 Sep 2019 15:08:37 +0000 (11:08 -0400)]
Use configured python version to install tox-pyenv
Revise the tox-install.sh script to use PYTHON env var to invoke pip,
instead of hardcoded 'python'.
Add python-version parameter to Read-The-Docs doc, macro and templates.
Change-Id: Ib0fe98dca62ce67f0ad002036b54f02faf27e888
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
Issue: RELENG-2361
Jessica Wagantall [Mon, 9 Sep 2019 20:54:53 +0000 (20:54 +0000)]
Merge "Fix self release script indent"
Lott, Christopher (cl778h) [Sun, 8 Sep 2019 11:04:37 +0000 (07:04 -0400)]
Remove passive voice in doc to silence warnings
The write-good feature as invoked by tox coala WriteGoodLintBear
complains about usage of passive voice e.g., 'is applied'.
These changes silence all but two warnings about use of the word
'validate' which is a well-known Maven goal.
This makes no functional change to any templates.
Change-Id: I5a68a475997cbeeae5ce41a2e487d67cc3c2644f
Issue: RELENG-2365
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
Anil Belur [Sun, 8 Sep 2019 03:56:42 +0000 (09:26 +0530)]
Fix self release script indent
Use 4 spaces indent. This makes the code easily readable
and consistant with rest of the code in global-jjb.
Change-Id: I62edca97517fe456d0fe688e2826b0d0fce8432a
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Lott, Christopher (cl778h) [Fri, 6 Sep 2019 00:38:46 +0000 (20:38 -0400)]
Repair tox-sonar behavior by restoring mvn goal
Use existing builder lf-infra-maven-sonar, drop the locally
defined builder lf-infra-tox-sonar, to get desired behavior
of generating code analysis then publishing to Sonar. This
broke when hardcoded goals were removed from maven-sonar.sh.
Change-Id: I862c4857d60361d77b390425d50f3c7c77e3919d
Issue: RELENG-2359
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
Jessica Wagantall [Wed, 4 Sep 2019 00:25:52 +0000 (17:25 -0700)]
Update release vars functions
Release schema verification needs to happen
first before we attempt to assign values to
the variables.
Organize variable setup into functions.
Maven release files expects different variables
than container release files.
Rename "version" variable in container release
files to "container_release_tag" which is a
better user friendly name given the fact that
container versions are rather called tags.
Internally, we still process it as "version"
to allow reuse of the tag function.
Issue: RELENG-2353
Change-Id: Ie0456a1fa87ea62855cb5d1140f6ae9f32b3e566
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Aric Gardner [Wed, 4 Sep 2019 16:21:48 +0000 (16:21 +0000)]
Merge "Clarify limit of one release yaml file"
Lott, Christopher (cl778h) [Tue, 3 Sep 2019 19:06:03 +0000 (15:06 -0400)]
Clarify limit of one release yaml file
Specify that Centos + Sigul + Docker are required.
Rewrap all paragraphs to eliminate long lines.
Change-Id: I57f46405cb962aed31c578354147b2b414da811f
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
Anil Belur [Wed, 4 Sep 2019 11:48:26 +0000 (17:18 +0530)]
Remove the unused MAVEN_CENTRAL_URL variable
Remove the unused MAVEN_CENTRAL_URL variable. The self-release job is
designed to work with any Nexus repository info published in
`staging-repo.txt.gz`, which makes the `MAVEN_CENTRAL_URL` redundant,
hence remove the unused variable.
Issue: RELENG-2129
Change-Id: Iabb772d908f8001a6626d728a9140ff14e3088f2
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Lott, Christopher (cl778h) [Tue, 3 Sep 2019 17:53:27 +0000 (13:53 -0400)]
Minor rewording and reformatting of release doc
Change-Id: I4d91d42bc2bd5cee4ae1e8dcebaba659bc168b6a
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
Anil Belur [Sat, 31 Aug 2019 03:04:35 +0000 (03:04 +0000)]
Merge "INFO validate job improvements"
Houa Yang [Fri, 30 Aug 2019 14:23:08 +0000 (09:23 -0500)]
Add Sonar plug-in configurations to docs
Add documentation for configuring the Jenkins Sonarqube Scanner plug-in.
Signed-off-by: Houa Yang <hyang@contractor.linuxfoundation.org>
Change-Id: I21b21efc6cefa3f4f643a0235d670c6e1ec4ac08
Aric Gardner [Thu, 29 Aug 2019 18:54:03 +0000 (14:54 -0400)]
INFO validate job improvements
removed +x trace as job is too verbose
verify now checks that repositories matches $PROJECT
Catches projects that replace / with - in their INFO file
Also ensures that repositories only has one entry.
We are not supporting multiple projects with a single INFO.yaml
file.
ISSUE: RELENG-2344
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Id847b7c10ef0515fd6f6a4b7d6809ebd37a9b51d
Anil Belur [Thu, 29 Aug 2019 14:49:46 +0000 (14:49 +0000)]
Merge "Remove detox in favor of tox --parallel"
Anil Belur [Wed, 28 Aug 2019 18:52:30 +0000 (00:22 +0530)]
Update lftools version to v0.26.2
Change-Id: I069dda979d203b1960821e4d112a5591843fd441
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Anil Belur [Wed, 28 Aug 2019 18:02:03 +0000 (23:32 +0530)]
Add DRY_RUN param to the release job
Add DRY_RUN build param to do a test run the job with
publishing artifacts.
Change-Id: I1b015db4be8ed3e8c39bec2cefe022819e31e59f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Anil Belur [Wed, 28 Aug 2019 16:31:17 +0000 (16:31 +0000)]
Merge "Verify both repos before attempting release"
Aric Gardner [Wed, 28 Aug 2019 15:39:00 +0000 (11:39 -0400)]
Verify both repos before attempting release
We have run into a case where the repo on ODL
was good, and the repo on nexus was failed.
Cover this case by running the loop twice.
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I995a34918481db9e41d5486c06995f57100c1199
Thanh Ha [Tue, 27 Aug 2019 22:13:38 +0000 (18:13 -0400)]
Remove detox in favor of tox --parallel
Tox officially supports parallel mode as of Tox 3.7 so drop the
usage of detox. Set parallel mode to 'auto' which will run as many
threads as available CPUs.
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: Id59ad10ed9b521a5c0ca3fef3e7e15053c2dc32c
Houa Yang [Tue, 27 Aug 2019 21:32:58 +0000 (16:32 -0500)]
Fix cmake docs
Remove unnecessary info for :install-prefix parameter.
Issue: RELENG-2210
Signed-off-by: Houa Yang <hyang@contractor.linuxfoundation.org>
Change-Id: I6cebee91b62f240c9cd8ade4b80573621b060ef0
Thanh Ha [Tue, 27 Aug 2019 19:08:54 +0000 (15:08 -0400)]
Configure basepython to python3
More and more libraries that we are pulling in require Python 3 so
to avoid having to play the pin dependencies whack-a-mole game let's
bump or base python where possible to python3.
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I65208f80b9f0479a10b1a8b6a4ba52956c2c3e3c
Anil Belur [Tue, 27 Aug 2019 03:25:00 +0000 (08:55 +0530)]
Upgrade lftools to v0.26.1
Change-Id: I66a72fcb7cff0148eb0f680df13e8744b17c7025
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Jessica Wagantall [Mon, 26 Aug 2019 20:59:33 +0000 (13:59 -0700)]
Fix use-release-file varaible
Rename "RELEASE_FILE" parameter to "USE_RELEASE_FILE" in
release-jobs. This will match the actual varaible default value
better and will not collide with the local "release_file" in
the script.
Fix "USE_RELEASE_FILE" if statement. We are now using a bool
instead of a string.
Changing the if statements to evaluate bools.
Change-Id: I9909a16487fabacc0261f6489ac81a00cbf0f965
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Anil Belur [Mon, 26 Aug 2019 19:12:17 +0000 (00:42 +0530)]
Pin more-itertools~=5.0.0
Pin more-itertools ~= 5.0.0, since version 6.0.0 requires Python 3.4
https://github.com/erikrose/more-itertools/releases
Error:
more-itertools requires Python '>=3.4' but the running Python is 2.7.5
Pin tox~<3.5.0
Change-Id: I1f2eefefc3f5b58a14441db78fb3f0bf9061e711
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Eric Ball [Sat, 24 Aug 2019 00:25:49 +0000 (00:25 +0000)]
Merge "Add pre-build scripts to Python templates"
Aric Gardner [Wed, 7 Aug 2019 18:05:51 +0000 (14:05 -0400)]
Add support for container releases
- Add support for distribution_type container
- Add conditional build step.
If {build-node} is ^.*-docker-.*
docker login will occur for verify and merge.
- Trigger updated to support hidden releases
directory. (releases\/.*\.yaml|\.releases\/.*\.yaml)
- Container support:
container_release_file example:
containers:
- name: example
version: 1.5.1-20190806T184921Z
- Verify pulls container
- Grab the image_id
- Merge
- Apply tag
- Push the container
ISSUE: RELENG-2292
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ibc6cf1a22bd6c0136a94d2b434c6cd332c12b43f
Lott, Christopher (cl778h) [Fri, 23 Aug 2019 12:36:56 +0000 (08:36 -0400)]
Add pre-build scripts to Python templates
New parameter pre-build-script accepts a shell script
that can install prerequisite libraries, rearrange the
source tree to cope with nested projects, etc.
Change-Id: I1fd7c2c094a664244e697ff5a61b82a2e7752faa
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
Eric Ball [Fri, 23 Aug 2019 22:25:38 +0000 (22:25 +0000)]
Merge "Add lf-sonar-common job-template"
Aric Gardner [Fri, 23 Aug 2019 18:13:56 +0000 (18:13 +0000)]
Merge "Allow Build with Params releases"
Aric Gardner [Fri, 23 Aug 2019 00:02:05 +0000 (17:02 -0700)]
Allow Build with Params releases
Add support for "Build with Parameters" for projects
that do not want to use a release file for maven builds.
Restructure shell/release-job.sh into functions.
Change-Id: Iada9c9a50710c24fbfe87b2ea0def10f3dd04bd7
Authored-By: Aric Gardner <agardner@linuxfoundation.org>
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Houa Yang [Thu, 15 Aug 2019 21:40:44 +0000 (16:40 -0500)]
Add lf-sonar-common job-template
Add lf-sonar-common job-template for sonar-project.properties
Issue: RELENG-1304
Signed-off-by: Houa Yang <hyang@contractor.linuxfoundation.org>
Change-Id: I6788bf9afb3b25fb751b806f08fddcf2fed42d26
Jessica Wagantall [Thu, 22 Aug 2019 23:23:16 +0000 (16:23 -0700)]
Fix release-schema.yaml
Fix missing extension in ID for release-schema.yaml.
Make "distribution_type" mandatory in future
release files.
Change-Id: I813f6b863fabfafcc9560df753c380675e8cc09e
Co-Authored-By: Aric Gardner <agardner@linuxfoundation.org>
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Andrew Grimberg [Mon, 19 Aug 2019 21:52:32 +0000 (21:52 +0000)]
Merge "Upgrade lftools to v0.26.0"
Andrew Grimberg [Mon, 19 Aug 2019 21:42:57 +0000 (21:42 +0000)]
Merge "Update packer-version to 1.4.3"
Aric Gardner [Mon, 19 Aug 2019 19:07:54 +0000 (15:07 -0400)]
Hard code release job compare pattern.
This change:
https://gerrit.acumos.org/r/c/ci-management/+/4979
The project section has:
gerrit_trigger_file_paths:
- compare-type: REG_EXP
pattern: '^((?!migrate-cms-to-cds|\/COMMIT_MSG).)*$'
This is overwriting the default parameters of this job.
gerrit_trigger_file_paths:
- compare-type: REG_EXP
pattern: "releases/*.yaml"
I belive I should hard code this value so that this
does not happen.
Pattern going forward will actually support
pattern: '(releases\/.*\.yaml|\.releases\/.*\.yaml)'
I will update 16470 and hard code as well, if this is
merged.
ISSUE: RELENG-2316
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ib520672a548f34762c723972152abea898cd98a8
Suresh Channamallu [Mon, 19 Aug 2019 18:52:59 +0000 (14:52 -0400)]
Update packer-version to 1.4.3
This new packer version fixed an issues in
docker image, where its unable to install the
packages into docker containers due to checking
of wrong container OS.
Fix in 1.4.3:
builder/docker: Check container os, not host os,
when creating container dir default [GH-7939]
Issue: RELENG-2180
Signed-off-by: Suresh Channamallu <schannamallu@linuxfoundation.org>
Change-Id: I3fe36e5669942d1f7fadfd2b711f9d7a3ca1e5a9
Anil Belur [Fri, 16 Aug 2019 02:48:13 +0000 (08:18 +0530)]
Upgrade lftools to v0.26.0
Change-Id: Iaa97109a097a85d0847372b8d7c9515960a4cd2e
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Aric Gardner [Thu, 8 Aug 2019 17:02:19 +0000 (13:02 -0400)]
Update release-jobs docs.
Feedback has allowed me to
Clarify the procedure
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I421cb6eebfccb48bde02d2a9d75ff020f895fde3
Andrew Grimberg [Thu, 8 Aug 2019 16:44:56 +0000 (16:44 +0000)]
Merge "Resolve ShellCheck errors in shell scripts"
Thanh Ha [Thu, 8 Aug 2019 02:20:22 +0000 (22:20 -0400)]
Use --config-xml option for JJB archives
The --config-xml option will produce actual config xml files in the
format 'job_name/config.xml' similar to the jobs directory in Jenkins
master instead of creating generically named 'job_name' files.
This allows the jobs to be dropped into a Jenkins jobs directory
path and Jenkins will be able to pick it up.
This is potentially useful for the ability to quickly drop in the
config into a Jenkins test environment.
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I59517d905a2448e46c50d98a471e10d7ca0fb808
Thanh Ha [Thu, 8 Aug 2019 02:32:05 +0000 (22:32 -0400)]
Resolve ShellCheck errors in shell scripts
* SC2086: Double quote to prevent globbing and word splitting.
* SC2027: The surrounding quotes actually unquote this.
Remove or escape them.
* SC2016: Expressions don't expand in single quotes,
use double quotes for that.
* SC2236: Use -n instead of ! -z.
* SC2233: Remove superfluous (..) around condition.
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I73207aa87646472bdb1c7848ba9608ac702bda53
Jessica Wagantall [Tue, 6 Aug 2019 23:18:55 +0000 (23:18 +0000)]
Merge "Use release credentials only for merge job wrappers"
Aric Gardner [Tue, 6 Aug 2019 15:00:49 +0000 (15:00 +0000)]
Merge "Spell check fixes in release-jobs docs"
Anil Belur [Tue, 6 Aug 2019 03:38:38 +0000 (13:38 +1000)]
Use release credentials only for merge job wrappers
Release creds are only required for promoting the repo, which uses diff ACL
as compared to normal user.
Therefore dont use the release creds for the verify jobs and the
scm sections in both the job templates.
Issue: RELENG-2129
Change-Id: Ibb36d11ba57c5e59a63c414b0b338846eefd8983
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Aric Gardner [Tue, 6 Aug 2019 02:08:41 +0000 (22:08 -0400)]
Had an if where I needed a fi
Also the git fetch needs the dashed version
git fetch "$PATCH_DIR/${PROJECT//\//-}.bundle"
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ib4245be2b46645fca93225a5c32efb904acd68c8
Anil Belur [Tue, 6 Aug 2019 00:15:37 +0000 (10:15 +1000)]
Upgrade lftools version to v0.25.4
Change-Id: I10460e230bccbf69cf8e8e12244f6c7a88cdccca
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Anil Belur [Sun, 4 Aug 2019 01:10:28 +0000 (11:10 +1000)]
Spell check fixes in release-jobs docs
Change-Id: I76aacf289bd46a1578c0741d58f2f3be4f48f28f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Aric Gardner [Mon, 5 Aug 2019 17:32:04 +0000 (13:32 -0400)]
Exit 0 if release job has already tagged a repo.
The release merge job is a one way operation.
Given this, Release jobs should only exist in the format
{project-name}-release-verify
and
{project-name}-release-merge
They should trigger from a change to any branch/**
As a developer working on {example-branch} will not want to
check out master to push a new release file.
Edge Cases:
Inevitably a release file will be pulled in from master to {example-branch}
or a remerge will be requested
This will again trigger the release jobs.
since in this case the repo is already tagged, the job should not report
a failure.
This is solved by having the verfiy and merge exit 0 when the repo is already tagged
if git tag -v "$VERSION"; then
echo "Repo already tagged $VERSION"
echo "This job has already run exit 0"
exit 0
if
Example Workflow - Milestone 0 is released from master:
releases/1.4.0.yaml (on branch master)
tag: 1.4.0
log_dir: 'optf-osdf-maven-stage-master/97/'
Repo is tagged and artifact is pushed!
Milestone 3 is released from dublin:
releases/1.4.3.yaml (on branch dublin)
tag: 1.4.3
log_dir: 'optf-osdf-maven-stage-dublin/20/'
Repo is tagged and artifact is pushed!
Note:
the ref is determined by the log file.
So We could check if this ref can be found on the {branch}
to reduce human errors.
example:
odlparent-maven-stage-4.0.x/152/patches/taglist.log.gz
has:
odlparent
c8964051cf81e2bd5d831fe6e5d568c5eedeeb8e
Psudo code:
STREAM=${STREAM:-'nostream'}
echo "--> Checking if $ref is on stable/$STREAM"
if ! (git branch -a --contains $ref | grep "stable/$STREAM"); then
echo "--> ERROR: $ref for $repo is not on stable/$STREAM!"
fi
Also in this patch:
To reduce possible errors:
Don't get project name from the release file:
PROJECT="$(niet ".project" "$release_file")"
we can just use ${PROJECT//\//-} when we need
the /'s changed to -'s
It will allways be correct when passed to jenkins
from gerrit
Also remove quiet from wget,
if there is a failure I want to see it on the console.
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I1e95e864bc2b49a62b3f7016cb2f1292a6687d9a
Aric Gardner [Sun, 4 Aug 2019 23:26:45 +0000 (23:26 +0000)]
Merge "Add release notes for lftools v0.25.3 release"
Anil Belur [Sun, 4 Aug 2019 22:55:50 +0000 (08:55 +1000)]
Improve the release job script
Change-Id: Ie62e2f3771632012423e0b791aca1551b4b31112
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Anil Belur [Sun, 4 Aug 2019 08:35:30 +0000 (18:35 +1000)]
Update the release job tigger
Update release job template to tigger on any branch name, and not just
'**/master'. ODL projects branches are version '4.0.x' which requires
passing the branch name to the template.
Issue: RELENG-2129
Change-Id: I9c29d1c986fee73f6bad4fe39af8f9ffbda201c6
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Anil Belur [Sun, 4 Aug 2019 01:08:22 +0000 (11:08 +1000)]
Add release notes for lftools v0.25.3 release
This was missed in CR Id64b36e8f59d500800d6b9078f56b314348ed1f2
Change-Id: Ie5412ea5724db51fe1eb73fd5f9370b2b6e3b6d2
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Aric Gardner [Sat, 3 Aug 2019 15:59:44 +0000 (11:59 -0400)]
Verify job needs target/variable switch
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I46501ed93f873c87a7d297398c1eb0dc9d23700f
Jessica Wagantall [Fri, 2 Aug 2019 23:04:08 +0000 (16:04 -0700)]
Fix release-job.sh
Use {GERRIT_PROJECT} when calling Gerrit in
release-merge job.
Change-Id: I0d62d5c713858e7e43b2fc6a6462813b01bea2f8
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Andrew Grimberg [Fri, 2 Aug 2019 22:42:47 +0000 (15:42 -0700)]
Fix signing pubkey declaration
Change-Id: Id9fb1cfa8c992949266622e3a2ed2100e8bc78f6
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Aric Gardner [Fri, 2 Aug 2019 21:48:04 +0000 (21:48 +0000)]
Merge "Update lftools version to v0.25.3"
Anil Belur [Fri, 2 Aug 2019 09:34:31 +0000 (19:34 +1000)]
Handle multiple repos listed in staging-repo.
Handle multiple repos listed in staging-repo for ODL.
Also multiple release files in the same commit will not work.
exiting if this case is detected
Also
git diff HEAD^1 --name-only
is bad. if there is another merge it fails.
Change to
git diff-tree --no-commit-id -r $GERRIT_PATCHSET_REVISION \
--name-only -- "releases/"
StrictHostKeyChecking is enabled on the builder.
disabled
Also needed to change
{jenkins-ssh-credential}
to
{jenkins-ssh-release-credential}
Also
Give group ``self-serve-release`` Forge Committer rights
on refs/tags/
and allow on Create Signed Tag Create Annotated Tag
Also
- file-id: signing-pubkey
target: SIGNING_PUBKEY
Also fix nexus url needs https://
Issue: RELENG-2129
Change-Id: I55dc0a4022f93a6c3c0f389c04c88c38c53eff64
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Anil Belur [Fri, 2 Aug 2019 21:34:35 +0000 (07:34 +1000)]
Update lftools version to v0.25.3
Change-Id: Id64b36e8f59d500800d6b9078f56b314348ed1f2
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Jessica Wagantall [Fri, 2 Aug 2019 02:11:16 +0000 (19:11 -0700)]
Add upgrade information
Add upgrade information for commit
729b2e3535f009a146719df7d7b52e036b2505cb
Change-Id: Id27a14aaff31153dad534794ef70534d0bfe8bf7
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Aric Gardner [Fri, 2 Aug 2019 01:27:18 +0000 (21:27 -0400)]
Import the pubkey and check the tag
with git tag -v $VERSION
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I954dfa1c8ca09f64ff8a35c138290565de8de4b9
Aric Gardner [Thu, 1 Aug 2019 18:09:34 +0000 (14:09 -0400)]
Project pattern was incorrectly set to **
must be {project}
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ic2db72185a3b2e2eb22011b2bc5e8f6012c235c3
Anil Belur [Wed, 31 Jul 2019 23:39:20 +0000 (23:39 +0000)]
Merge "Self serve release docs"
Aric Gardner [Wed, 19 Jun 2019 18:10:42 +0000 (14:10 -0400)]
Self serve release docs
And code fixes for new:
jenkins-ssh-release-credential
Fix in release-job.sh
These were overlooked for merge:
git config user.name
git config user.email
gerrit_ssh=$(echo "$GERRIT_URL" | awk -F"/" '{print $3}')
git remote set-url origin ssh://"$RELEASE_USERNAME"@"$gerrit_ssh":29418/$PROJECT
Also included a small capitalization fix in lf-info-vote.rst
ISSUE: RELENG-2127
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Change-Id: I11ad122153d71a3d25d6b8839e09435f8b27b19d
Andrew Grimberg [Wed, 31 Jul 2019 22:17:17 +0000 (22:17 +0000)]
Merge "Log $JOB_NAME with the jjb-deploy"
Jessica Wagantall [Wed, 31 Jul 2019 22:15:21 +0000 (22:15 +0000)]
Merge "Update lf-info-yaml-verify"
Anil Belur [Mon, 29 Jul 2019 01:33:00 +0000 (11:33 +1000)]
Log $JOB_NAME with the jjb-deploy
Change-Id: Ifb2715402dc1dcb21708e4dc8851e0c621d59145
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Andrew Grimberg [Wed, 31 Jul 2019 22:03:18 +0000 (22:03 +0000)]
Merge "Consolidate lf-infra-jjbini with JJB 2.0"
Anil Belur [Mon, 29 Jul 2019 03:35:20 +0000 (13:35 +1000)]
Consolidate lf-infra-jjbini with JJB 2.0
There is no need to have two separate versions of the
macro anymore since we are already on JJB 2.0
Change-Id: I2295489f8d0bebf9e1628db91c38e74d3839b8a0
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Jessica Wagantall [Tue, 30 Jul 2019 01:05:00 +0000 (18:05 -0700)]
Update lf-info-yaml-verify
Move info-schema to schema/info-schema.yaml
to keep schemas consistency.
Download only needed files for lf-info-yaml-verify
rather than cloning the entire repo.
Issue: RELENG-2242
Change-Id: Ie135c761d1401957be33c98634467967cc085ce2
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>