releng/global-jjb.git
6 years agoAdd docs for Python XC CLM 60/10160/7
Jessica Wagantall [Fri, 20 Apr 2018 00:44:06 +0000 (17:44 -0700)]
Add docs for Python XC CLM

Add documentation for Python XC CLM:
- Description of the job
- Job templates
- Description of parameters

Changes made in response to the comments in
I5e5c420539ab3a3e5b292620da4418c16d557221

Change-Id: I19713d37aa38159f26e437c961b6d1b4fa6f77fc
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoMerge "Rename sonar-cli.sh to nexus-iq-cli.sh"
Thanh Ha [Mon, 23 Apr 2018 17:52:39 +0000 (17:52 +0000)]
Merge "Rename sonar-cli.sh to nexus-iq-cli.sh"

6 years agoFail Sphinx build even on warnings 85/10185/1
Thanh Ha [Fri, 20 Apr 2018 18:47:16 +0000 (14:47 -0400)]
Fail Sphinx build even on warnings

Resolve warnings caused by the -W flag.

Change-Id: I0c771da88af487e3a6ddb55b01d524d5a274226d
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump version to 0.19.0-dev 84/10184/1
Thanh Ha [Fri, 20 Apr 2018 18:46:52 +0000 (14:46 -0400)]
Bump version to 0.19.0-dev

Change-Id: I000964a1bd8a4d7a9ed09fb29b130258e63ba922
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease global-jjb v0.18.0 01/10101/3 v0.18.0
Thanh Ha [Wed, 18 Apr 2018 21:23:32 +0000 (17:23 -0400)]
Release global-jjb v0.18.0

Change-Id: I0538f3c15e6df43369ef5e7756a76236d52d0a9e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Update JJB version 2.0.5"
Thanh Ha [Fri, 20 Apr 2018 16:24:16 +0000 (16:24 +0000)]
Merge "Update JJB version 2.0.5"

6 years agoUpdate packer version to 1.2.2 71/10171/1
Anil Belur [Fri, 20 Apr 2018 14:19:56 +0000 (00:19 +1000)]
Update packer version to 1.2.2

Change-Id: I188bb7b14e5122afc25b64765d846c4d1298e8f6
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoUpdate comment triggers XC CLM 03/9903/7
Jessica Wagantall [Fri, 20 Apr 2018 00:55:42 +0000 (17:55 -0700)]
Update comment triggers XC CLM

Update comment triggers for Python XC CLM
jobs. From "run-xc-clm" to "run-clm" to make it more
consistent with other clm jobs.

Changes made in response to the comments in
I5e5c420539ab3a3e5b292620da4418c16d557221

Change-Id: I637d03805e19460c6c9abb20af61ee4d9d34763e
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoRename sonar-cli.sh to nexus-iq-cli.sh 61/10161/3
Jessica Wagantall [Fri, 20 Apr 2018 00:50:49 +0000 (17:50 -0700)]
Rename sonar-cli.sh to nexus-iq-cli.sh

Rename sonar-cli.sh script to nexus-iq-cli.sh since
it is a more appropriated name for this script as Sonar
tasks are not involved on this job. Just Nexus-IQ scans.

Change-Id: I39a48c1cb8e24ad669a9a7f0ab4a7fcdb57ff1d5
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoAdd new odl- flavors to cloud list 42/10142/1
Thanh Ha [Thu, 19 Apr 2018 17:46:29 +0000 (13:46 -0400)]
Add new odl- flavors to cloud list

Change-Id: Icf5551056956f8d10367b3bf1efa106b1ce0b395
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoUpdate JJB version 2.0.5 85/10085/1
Anil Belur [Wed, 18 Apr 2018 11:03:06 +0000 (21:03 +1000)]
Update JJB version 2.0.5

JJB version 2.0.5 contains most recent changes required
to update and manage views on Jenkins.

RELENG-76
Change-Id: Ib230f452b060a21ed02229ff292198c6384ce44d
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoReplace pipdeptree with pip freeze 99/9999/1
Thanh Ha [Sat, 14 Apr 2018 17:15:07 +0000 (13:15 -0400)]
Replace pipdeptree with pip freeze

While pipdeptree provides more detailed information it's
dependency with pip seems to break things if we don't
install in the correct order. Let's revert to pip freeze
for now until we have a better way to maintain these debugging
packages.

Issue: RELENG-884
Change-Id: Ib6d3008e2d0e195bc9a01a1c4dd476d92a9deace
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoPin pip to 9.0.3 98/9998/1
Thanh Ha [Sat, 14 Apr 2018 17:10:25 +0000 (13:10 -0400)]
Pin pip to 9.0.3

Pin pip versions to 9.0.3 in all scripts that try to install pip.

Issue: RELENG-884
Change-Id: I25df8beb412e06d14db88d90debafa46013bb7ac
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add lf_tox_sonar job"
Jessica Wagantall [Thu, 12 Apr 2018 21:53:23 +0000 (21:53 +0000)]
Merge "Add lf_tox_sonar job"

6 years agoAdd lf_tox_sonar job 94/9894/16
Jessica Wagantall [Tue, 10 Apr 2018 22:24:16 +0000 (15:24 -0700)]
Add lf_tox_sonar job

Python jobs uses tox.ini to configure
their project coverage report. Adding a tox call
before the sonar calls makes the proper updates
to the reports based on that coverage.

Change-Id: I4e6787274ae15c1a9b3b866a55e02f3587b906fb
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoMerge "Update RTD to use Generic API v2 webhook"
Thanh Ha [Wed, 11 Apr 2018 18:12:53 +0000 (18:12 +0000)]
Merge "Update RTD to use Generic API v2 webhook"

6 years agoUpdate default jjb version 13/9913/1
Jeremy Phelps [Wed, 11 Apr 2018 18:00:19 +0000 (13:00 -0500)]
Update default jjb version

JJB contains bits in postbuildscript
publisher that require a minimum of
version 2.0.3.

Change-Id: Ibc62684a9b7dec125064fb5ce823e3f6452581db
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
6 years agoUpdate RTD to use Generic API v2 webhook 56/9656/9
Jeremy Phelps [Tue, 27 Mar 2018 16:45:57 +0000 (11:45 -0500)]
Update RTD to use Generic API v2 webhook

The old tokenless generic trigger url has been deprecated. Update
to using the new v2 API that requires a token.

This is a backwards incompatible change and modifies 2 things:

  * rtd-project is now the rtd-build-url parameter
  * rtd-token is a new required parameter

Issue: RELENG-879
Change-Id: I786240ab14cf95a2896d92d1c87aef9d7815d964
Co-Authored-By: Thanh Ha <thanh.ha@linuxfoundation.org>
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoForce rm for cleanup macro 91/9891/3
Thanh Ha [Tue, 10 Apr 2018 20:27:23 +0000 (16:27 -0400)]
Force rm for cleanup macro

Using -f (force) rm will return a 0 status and not print a
message if a file is not found.

Issue: RELENG-880
Change-Id: Ia06166362526cf00754b4a21e6b977d57fe91db6
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoFix pipconf name to the default one 90/9890/1
Thanh Ha [Tue, 10 Apr 2018 20:23:50 +0000 (16:23 -0400)]
Fix pipconf name to the default one

In Managed Files we do not give the filename a dot so remove
it to match what we really look for.

Change-Id: Iaa5df87921d7d7333ecb87bc11c707884e4fb277
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Validate packer version before install"
Anil Belur [Sat, 7 Apr 2018 00:30:22 +0000 (00:30 +0000)]
Merge "Validate packer version before install"

6 years agoFix pip version 51/9751/2
Ry Jones [Thu, 5 Apr 2018 15:27:09 +0000 (08:27 -0700)]
Fix pip version

pip has released a beta version which breaks pipdeptree.

RELENG-868
Change-Id: I755a2521f28fc6441a69dd0ed93c0f85502b787b
Signed-off-by: Ry Jones <ry@walledcity.org>
6 years agoValidate packer version before install 29/9629/6
Anil Belur [Mon, 26 Mar 2018 22:03:25 +0000 (15:03 -0700)]
Validate packer version before install

Packer 1.1.3 is the minimum version required to be installed
to build images on vexxhost. The install script does not check
the version and only checks the availability of the binary in the
$PATH before updating packer.

JIRA: RELENG-838
Change-Id: I8c343176d8f39e3e70b04dddbdec94ffed4354b1
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoPin tox to <3.0.0 as it breaks detox 20/9720/2
Jeremy Phelps [Tue, 3 Apr 2018 13:40:56 +0000 (08:40 -0500)]
Pin tox to <3.0.0 as it breaks detox

Issue-ID: RELENG-856
Change-Id: Id13aee7244cce6328fd10cbe495b5fb3e7069924
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
6 years agoMerge "Revert "Update RTD to use custom trigger url""
Thanh Ha [Wed, 28 Mar 2018 16:47:02 +0000 (16:47 +0000)]
Merge "Revert "Update RTD to use custom trigger url""

6 years agoRevert "Update RTD to use custom trigger url" 54/9654/1
Thanh Ha [Wed, 28 Mar 2018 16:40:52 +0000 (16:40 +0000)]
Revert "Update RTD to use custom trigger url"

This reverts commit 12d6179c0e7c208a360a3c61c0b715b49310b39e.

Change-Id: I87a8f3b25ab977cdcfd3c23da6cebd610edaf325
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Export packer binary dir in $PATH"
Andrew Grimberg [Wed, 28 Mar 2018 16:23:13 +0000 (16:23 +0000)]
Merge "Export packer binary dir in $PATH"

6 years agoUpdate RTD to use custom trigger url 34/9634/4
Jeremy Phelps [Tue, 27 Mar 2018 16:45:57 +0000 (11:45 -0500)]
Update RTD to use custom trigger url

The generic trigger url is now deprecated.  Going forward
users will have to use the Generic API incoming webhook
that is defined per project in the read the docs service.

Change-Id: Iac0774266e787705e7fb903712cbfb4b4a41fb55
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
6 years agoAdd link to release notes 34/9534/1
Thanh Ha [Wed, 21 Mar 2018 17:11:02 +0000 (13:11 -0400)]
Add link to release notes

Issue: RELENG-552
Change-Id: Ifddf4463ac6afa2395be25cddf989fe05dd9fec1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Quit build if submodule not found"
Thanh Ha [Mon, 19 Mar 2018 19:55:02 +0000 (19:55 +0000)]
Merge "Quit build if submodule not found"

6 years agoQuit build if submodule not found 96/9496/1
Thanh Ha [Mon, 19 Mar 2018 19:30:08 +0000 (15:30 -0400)]
Quit build if submodule not found

In the monolithic docs build we only care to build if the
project is really a submodule of the monolithic docs build.
If not then skip the build as the project should be providing
their own docs verify job.

Issue: RELENG-845
Change-Id: I9bc146170e150097b3b9a30ec0456173bba9b1ef
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Support PostBuildScript 2.x"
Andrew Grimberg [Mon, 19 Mar 2018 17:37:46 +0000 (17:37 +0000)]
Merge "Support PostBuildScript 2.x"

6 years agoEnsure setuptools is updated 78/9478/1
Thanh Ha [Sun, 18 Mar 2018 03:55:41 +0000 (23:55 -0400)]
Ensure setuptools is updated

Similar to pip, we should ensure that setuptools is updated as well.
pip 9.0.2 appears to break without setuptools also being updated.

Change-Id: I3b66140a1a5b292a6e44e955b31ddd62cdc6ed03
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd support for VolumeFromImage boot source 70/9470/2
Thanh Ha [Fri, 16 Mar 2018 18:47:18 +0000 (14:47 -0400)]
Add support for VolumeFromImage boot source

Allows us the ability to configure a volume size for images. This
feature requires OpenStack Cloud Plugin 2.32 to use.

Change-Id: Ic557373c16afc3df3af3571e5f3883f5b18e8e8c
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Document job-group defaults order of priority"
Anil Belur [Fri, 16 Mar 2018 06:44:07 +0000 (06:44 +0000)]
Merge "Document job-group defaults order of priority"

6 years agoExport packer binary dir in $PATH 61/9461/1
Anil Belur [Thu, 15 Mar 2018 12:55:28 +0000 (22:55 +1000)]
Export packer binary dir in $PATH

Add packer binary path to $PATH to avoid packer being
installed multiple times in builder merge jobs.

JIRA: RELENG-838
Change-Id: Ie414ff39d0829135cc797b187f693930cf45c6d5
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoSupport PostBuildScript 2.x 87/9387/3
Thanh Ha [Mon, 12 Mar 2018 01:48:09 +0000 (21:48 -0400)]
Support PostBuildScript 2.x

This is a breaking change that requires consumers of global-jjb
to switch to JJB 2.0.3 and install PostBuildScript 2.x in Jenkins.

It updates lf-infra-publish macro to switch to the PostBuildScript 2.x
YAML definitions.

Issue: RELENG-832
Change-Id: I04020587acfb8b6550c0221ebfb97a501f46b15a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump version to 0.18.0-dev 93/9393/1
Thanh Ha [Mon, 12 Mar 2018 15:15:28 +0000 (11:15 -0400)]
Bump version to 0.18.0-dev

Change-Id: Ib2d45332a9626b0aaf82443e1f37846a00569e46
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease global-jjb v0.17.0 86/9386/1 v0.17.0
Thanh Ha [Mon, 12 Mar 2018 01:24:16 +0000 (21:24 -0400)]
Release global-jjb v0.17.0

Change-Id: I13cf313e1875eabb50290c2409ef632eaa7183f7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoDocument job-group defaults order of priority 81/7281/3
Thanh Ha [Fri, 3 Nov 2017 19:20:03 +0000 (15:20 -0400)]
Document job-group defaults order of priority

We discovered that job-group variable definitions appear to
have higher precedence than project definitions. This caused a problem
for us in global-jjb as we would like to allow projects to override
default variables.

We should make it best practice that global-jjb stops providing
job-group definitions and deprecating any existing job groups.

Via patch I2b824e3cc89667698db8ebfdd3ce43f551e961d7 we were able to
resolve this for JJB version.

Change-Id: I93c8256193933a04e8732963216f69424cef3a5f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoDocument pyenv usage in the tox job 84/8384/3
Thanh Ha [Fri, 12 Jan 2018 19:10:06 +0000 (14:10 -0500)]
Document pyenv usage in the tox job

Change-Id: Id07a1e4065b03489260dfe1d0b86cacf8d60238d
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoPin JJB to 2.0.2 for tests 85/9385/1
Thanh Ha [Mon, 12 Mar 2018 01:00:15 +0000 (21:00 -0400)]
Pin JJB to 2.0.2 for tests

JJB >2.0.2 requires use to update our shiplogs postbuildscript as
support for the 2.x version of postbuildscript takes over the
default code path.

Change-Id: Iff3cd415f12a54aa3fcbf48d7f10b6df01cd1279
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd CLM support for Python 02/9302/14
Jessica Wagantall [Wed, 7 Mar 2018 01:10:41 +0000 (17:10 -0800)]
Add CLM support for Python

Add template to support CLM scans for python
based projects. This template uses the nexus-iq-cli
package directly and executes an extended report to
scan python based projects.

Change-Id: I5e5c420539ab3a3e5b292620da4418c16d557221
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
6 years agoAdd explicit links for Packer jobs 04/9304/1
Thanh Ha [Wed, 7 Mar 2018 01:41:00 +0000 (20:41 -0500)]
Add explicit links for Packer jobs

Change-Id: I90e4d89008f0f6550f640a8c03b268d1bde884e9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd script to manage Jenkins cloud cfg 29/8429/18
Thanh Ha [Thu, 18 Jan 2018 16:09:36 +0000 (11:09 -0500)]
Add script to manage Jenkins cloud cfg

A groovy script to manage the Jenkins global cloud cfg
programmatically. This script reads configuration from
the directory jenkins-config/clouds to configure cloud
configuration in Jenkins. Refer to the documentation for
details.

No upgrade consideration is necessary as this script
automatically detects the absence of the jenkins-config/clouds
directory and disables the Cloud configuration.

Issue: RELENG-804
Change-Id: Idad3078942618b7d8fd08a40bf9e4e5ed5a0dc48
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMake rtd jobs verbose 09/9109/2
Lott, Christopher (cl778h) [Thu, 1 Mar 2018 14:40:28 +0000 (09:40 -0500)]
Make rtd jobs verbose

Add set -x flag so these scripts echo steps in Jenkins output.

Change-Id: Ia09f3194407f6c94d8cb1adfbb4d32e9a4592ff6
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
6 years agoAdd a description of global-jjb 82/9082/2
Thanh Ha [Wed, 28 Feb 2018 01:53:47 +0000 (20:53 -0500)]
Add a description of global-jjb

Change-Id: I10926e554e4c99f6336837f7393d065f8e653599
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add triggers to jenkins-cfg job"
Thanh Ha [Wed, 28 Feb 2018 01:53:04 +0000 (01:53 +0000)]
Merge "Add triggers to jenkins-cfg job"

6 years agoGIT_URL variable corruption in script 57/9057/2
Thanh Ha [Tue, 27 Feb 2018 05:29:49 +0000 (00:29 -0500)]
GIT_URL variable corruption in script

Not sure why but GIT_URL seems to get corrupt inside of this
script. Use GERRIT_URL instead to clone patches.

Additionally also ignore a patch if a patch that matches the
Gerrit that triggered it is passed in.

Issue: RELENG-784
Change-Id: I85de941ad64d64597f69c1c924295b26e1dc1bec
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd triggers to jenkins-cfg job 83/8883/1
Thanh Ha [Fri, 16 Feb 2018 19:55:55 +0000 (14:55 -0500)]
Add triggers to jenkins-cfg job

Allows the job to run on a timed trigger as well as on patch
merged events. Default timed trigger is run @daily.

Change-Id: I54b04e7814718816fe221a33d80aa1c1574095d7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd local-{templates}.yaml path to packer pattern 78/8878/1
Thanh Ha [Thu, 15 Feb 2018 22:09:05 +0000 (17:09 -0500)]
Add local-{templates}.yaml path to packer pattern

Common packer supports local improvements via local-*.yaml files
so we need to add this to the file paths.

Change-Id: Id6d723a174de5f272cc267496279d7e79cef8b8f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoGit submodule update after fetching patch 59/8859/1
Thanh Ha [Wed, 14 Feb 2018 21:13:11 +0000 (16:13 -0500)]
Git submodule update after fetching patch

For projects that have submodules we need to call
`git submodule update` after fetching the patch so that
submodule updates caused by the patch are pulled in.

Running that command on a repo that does not contain submodules
returns cleanly as a noop.

Change-Id: Iec76e4c42d9e0132571af477f3ae770e5e360781
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd a note about requiring jjbini configuration 39/8839/1
Thanh Ha [Tue, 13 Feb 2018 18:21:45 +0000 (13:21 -0500)]
Add a note about requiring jjbini configuration

Document that jjbini needs to include JJB 2.0 style configuration
sections for the "production" and "sandbox" systems.

Change-Id: I102c57774a75bd1545f21a67bb2d98ccb0f270a1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump version to 0.17.0-dev 30/8830/1
Thanh Ha [Mon, 12 Feb 2018 23:55:07 +0000 (18:55 -0500)]
Bump version to 0.17.0-dev

Change-Id: I9c352b633fcf83a0b60e2351401cd76c42df238d
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease global-jjb v0.16.0 86/8786/3 v0.16.0
Thanh Ha [Fri, 9 Feb 2018 19:12:42 +0000 (14:12 -0500)]
Release global-jjb v0.16.0

Change-Id: I6acf020cda29198ad930ceeffe08e09ad0c7d6bd
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoCreate job to manage Jenkins Global Variables 34/8434/8
Thanh Ha [Fri, 19 Jan 2018 00:24:56 +0000 (19:24 -0500)]
Create job to manage Jenkins Global Variables

Parses a configuration file containing variable listings in the form
"KEY=value" one per line and creates the appropriate groovy
script to push these environment variables to Jenkins Global Config.

Configuration file is stored in
ci-management/jenkins-config/global-vars-SILO.sh

(Where silo is production | sandbox)

While JJB provided us a way to store job configuration in a git repo
we have never had a good way to manage global Jenkins configuration.
This patch enables us to manage one aspect, Global Environment
variables in the ci-management repository.

Issue: RELENG-753
Change-Id: I7d134950b25d469686cace2f79fd8c297727166c
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add pip.conf to wrappers"
Thanh Ha [Fri, 9 Feb 2018 18:13:01 +0000 (18:13 +0000)]
Merge "Add pip.conf to wrappers"

6 years agoUpdate submodule documentation 67/8767/2
Andrew Grimberg [Thu, 8 Feb 2018 21:26:14 +0000 (13:26 -0800)]
Update submodule documentation

Production deployments of global-jjb should be using the GitHub mirror
as their primary source as GitHub rarely has outages. This decouples the
master Gerrit instance from downstream consumption such that we are more
free to do maintenance whenever we desire.

Change-Id: I37139965902fa922cbb14f7ad01c22248bc1847b
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
6 years agoset +x around sensitive code docker login code 47/8747/1
Thanh Ha [Thu, 8 Feb 2018 04:19:43 +0000 (23:19 -0500)]
set +x around sensitive code docker login code

Call `set +x` as a guard just in case a script elsewhere is calling
`set -x` which could cause the credentials to print to console.

Change-Id: I6802887c7a3c9809d9d13d6b3cc90e3570f120fc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd : to jjb-deploy: 81/8681/3
Jamo Luhrsen [Tue, 6 Feb 2018 03:28:23 +0000 (19:28 -0800)]
Add : to jjb-deploy:

This is more intuitive to me, as the keyword takes an
argument. I realized this when I used the ODL keyword
copy-logs:

Change-Id: Ic321d6be56ef518a2056099cb5690ea6880c16c2
Signed-off-by: Jamo Luhrsen <jluhrsen@redhat.com>
6 years agoAdd test to check for double curly braces 88/8688/1
Thanh Ha [Tue, 6 Feb 2018 16:36:31 +0000 (11:36 -0500)]
Add test to check for double curly braces

This will cause the build to fail if double curly braces are in
JJB output XML. This should help catch the JJB bug that adds
curly braces when using !include-raw-escape.

Issue: RELENG-777
Change-Id: Iaf9a9ef695229fa24dfc639ca60a4b9a9928a234
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRepair shell-include directive 85/8685/2
Christopher Lott (cl778h) [Tue, 6 Feb 2018 16:11:18 +0000 (11:11 -0500)]
Repair shell-include directive

Change stage macro to use include-raw instead of include-raw-escape to
work around a JJB bug that inserts lftools_activate with double braces.

Change-Id: Iec5e0b12a287f1945eedc895a25dd16e74b47082
Signed-off-by: Christopher Lott (cl778h) <clott@research.att.com>
6 years agoAdd maven metadata validator tool 75/8675/3
Thanh Ha [Mon, 5 Feb 2018 20:03:01 +0000 (15:03 -0500)]
Add maven metadata validator tool

Add a tool to validate maven-metadata.xml to catch timestamp issues
in builds and fail the build if a mismatch is detected.

Issue: RELENG-772
Change-Id: I35a7b9c3b12321a3a7af24ab36397e38ebb01de9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd pip.conf to wrappers 58/8658/3
Thanh Ha [Fri, 2 Feb 2018 20:46:58 +0000 (15:46 -0500)]
Add pip.conf to wrappers

This is a global-jjb breaking change as it now causes us to require
the pipconf file configured in Config File Provider. This allows us
to configure for projects that have Nexus 3 a PyPi proxy for faster
local downloads.

Contents of pip.conf should look something like:

[global]
timeout = 60
index-url = https://nexus3.opendaylight.org/repository/PyPi/simple

Change-Id: I4ac816f1b78f09c101e6d614bdd3c459fe0e133a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAllow goal override for maven-build macro 38/8438/2
Thanh Ha [Fri, 19 Jan 2018 01:24:26 +0000 (20:24 -0500)]
Allow goal override for maven-build macro

Splits out the maven-build into a macro that can be used
to allow projects to override the default goals if they
wish to tweak them for their projects.

Change-Id: I8cabbac997d165bfe8b68d76b78cdb4bb7cc4bbc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSplit maven-stage script into a reusable macro 37/8437/2
Thanh Ha [Fri, 19 Jan 2018 01:16:13 +0000 (20:16 -0500)]
Split maven-stage script into a reusable macro

Change-Id: If8d2208e2469d295637a9e82545b2289e5a7588b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add sphinx link to rtd jobs page"
Thanh Ha [Fri, 2 Feb 2018 00:56:50 +0000 (00:56 +0000)]
Merge "Add sphinx link to rtd jobs page"

6 years agoMerge "Add Links to Plugins mentioned in README"
Thanh Ha [Thu, 1 Feb 2018 20:12:03 +0000 (20:12 +0000)]
Merge "Add Links to Plugins mentioned in README"

6 years agoFix local platform file path 37/8637/2
Thanh Ha [Thu, 1 Feb 2018 18:50:47 +0000 (13:50 -0500)]
Fix local platform file path

The local vars path is incorrect in the if-statement but
correct in the declaration below it. Update to match.

Change-Id: I1e8ec19adf842668fb46e54aaf8beebfcef9c34d
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoDisable unbound variable checking for activate 36/8636/1
Thanh Ha [Thu, 1 Feb 2018 18:15:04 +0000 (13:15 -0500)]
Disable unbound variable checking for activate

virtualenv activate contains unbound variables so must be disabled
when calling `source activate`.

Change-Id: I8af6b55962580778fbb349c8164f661c4c462928
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd sphinx link to rtd jobs page 35/8635/1
Thanh Ha [Thu, 1 Feb 2018 02:26:16 +0000 (21:26 -0500)]
Add sphinx link to rtd jobs page

Change-Id: I67bf0e81fd3e6156cd31dbb4700c8909cf2bf87d
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agodeploy cleanup not removing all unmodified files 23/8623/1
Thanh Ha [Wed, 31 Jan 2018 22:00:23 +0000 (17:00 -0500)]
deploy cleanup not removing all unmodified files

The linting patch If87fceb79e384e26f021a9c85a2e9d7413ce421e
broke the cleanup script so unmodified maven-metadata.xml files
were not properly being cleaned up before upload to Nexus. This
caused the maven-merge job to update maven-metadata.xml files
outside of what was actually modified. So when 2 maven-merge jobs
running on different streams are run at the same time they would
overwrite each other with old data.

Issue: RELENG-766
Change-Id: I18da721bfb5db6eef9c15ffabd4482e0aa24e3a6
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoupdate docs to use minutes over seconds 13/8613/1
Sam Hague [Wed, 31 Jan 2018 13:31:56 +0000 (08:31 -0500)]
update docs to use minutes over seconds

Change-Id: Ia1e7b101ee6a3605d6c0ec4ac163a6f8c362921b
Signed-off-by: Sam Hague <shague@redhat.com>
6 years agoAdd Links to Plugins mentioned in README 92/8592/2
Trevor Bramwell [Tue, 30 Jan 2018 18:41:12 +0000 (10:41 -0800)]
Add Links to Plugins mentioned in README

Includes a link to the download location of the PostBuildScript plugin
since it is no longer being released.

Change-Id: Ia2d34921fa9d870fd5792d938f9b2844853a38e9
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
6 years agoAdd script to update Java alternatives 93/8493/15
Jessica Wagantall [Wed, 24 Jan 2018 00:31:47 +0000 (16:31 -0800)]
Add script to update Java alternatives

Use ${java-version} to set the correct java, javac
and java_sdk_openjdk alternative for lf-maven-jobs.

Issue-ID: RELENG-752
Change-Id: Iff0c2e7d92c4cc802925ae67cdda1f4247db52be
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoRemove jdk variable from lf-maven-jobs 71/8471/3
Jessica Wagantall [Tue, 23 Jan 2018 00:14:53 +0000 (16:14 -0800)]
Remove jdk variable from lf-maven-jobs

Setting this variable to {java-version} creates some issues for
projects using the Ubuntu nodes as they need to use explicitly
'System' as their JDK version.

Change-Id: I0a12bc83f7c8c1c8fcdf8f2a1d5f47b8360b94c9
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoBump default timeout to 15 minutes for tox 80/8580/1
Thanh Ha [Mon, 29 Jan 2018 22:20:55 +0000 (17:20 -0500)]
Bump default timeout to 15 minutes for tox

Increase tox jobs default timeout to 15 minutes. Considering many
things appear to be taking longer in Vexxhost this is likely
necessary.

Change-Id: I592146b1f0d47a9aedfdc91c1878a6f94b3d363c
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRefactor macro/template cron param 66/8566/3
Christopher Lott (cl778h) [Mon, 29 Jan 2018 17:51:21 +0000 (12:51 -0500)]
Refactor macro/template cron param

Be consistent with cron parameter: define in macro, use in template.
Never hard-code a trigger in a job template.
Document the parameters in the RST file.

Change-Id: I70a1deefd23bc9498f6c6013fb8751298dec7f53
Signed-off-by: Christopher Lott (cl778h) <clott@research.att.com>
6 years agoMerge "Split maven-deploy into a macro"
Andrew Grimberg [Mon, 22 Jan 2018 23:02:07 +0000 (23:02 +0000)]
Merge "Split maven-deploy into a macro"

6 years agoBump global-jjb to 0.16.0-dev 66/8466/1
Thanh Ha [Mon, 22 Jan 2018 21:47:11 +0000 (16:47 -0500)]
Bump global-jjb to 0.16.0-dev

Change-Id: I0f5e3d41bcdc3f6938208d3b710b20df9dfb5c70
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease global-jjb v0.15.0 56/8456/1 v0.15.0
Thanh Ha [Mon, 22 Jan 2018 02:02:41 +0000 (21:02 -0500)]
Release global-jjb v0.15.0

Change-Id: I1186feb6ff4fefb5311bcfb84ad34b4001b9cef7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoIncrease maven-jobs default timeout to 60 mins 43/8443/1
Thanh Ha [Fri, 19 Jan 2018 16:50:13 +0000 (11:50 -0500)]
Increase maven-jobs default timeout to 60 mins

The new Vexxhost cloud seems to be slower in general and
continuous to cause OpenDaylight grief when it comes to running
jobs. Increase the timeout to 60 mins to account for the slower
infra.

Change-Id: I04325ce9d700ac4879a17f1b0547bc0b861c7cac
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSplit maven-deploy into a macro 36/8436/2
Thanh Ha [Fri, 19 Jan 2018 01:10:24 +0000 (20:10 -0500)]
Split maven-deploy into a macro

Split out maven-deploy into a reusable macro.

Change-Id: I6ddad68576cdbf9d9ef8b4805da94dc6a80cbefd
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Improve verbosity in maven shell scripts"
Thanh Ha [Fri, 19 Jan 2018 01:02:36 +0000 (01:02 +0000)]
Merge "Improve verbosity in maven shell scripts"

6 years agoUse clean install for Sonar 31/8431/1
Thanh Ha [Thu, 18 Jan 2018 19:22:07 +0000 (14:22 -0500)]
Use clean install for Sonar

The Sonar job does not need to deploy so switch to using
`mvn clean install` to build the project before running
sonar scan.

Change-Id: I577a57417d0c56f403e66b9fb7201bf58fef818e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoImprove verbosity in maven shell scripts 30/8430/1
Thanh Ha [Thu, 18 Jan 2018 19:20:46 +0000 (14:20 -0500)]
Improve verbosity in maven shell scripts

There is no harm in enabling debug level verbosity for these scripts
and may help in improving troubleshooting.

Change-Id: I9edfee2f7d2415117d854ad7cdba280cc9751004
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRevert "Switch global-jjb to lfdocs-conf" 23/8423/1
Thanh Ha [Wed, 17 Jan 2018 20:23:26 +0000 (15:23 -0500)]
Revert "Switch global-jjb to lfdocs-conf"

Need to revert because conf.yaml causes deployment issues in
jjb recursive runs as it's not expecting a non-jjb yaml. Let's
not use lfdocs-conf in global-jjb until we have a way around
that.

This reverts commit ac79981ec42d37f943c2a4159e2980913cb1716d.

Change-Id: I0eb5d269b31c76e5da7affd80d93715cee8e4037
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoDefault to python2 for tox 19/8419/1
Thanh Ha [Wed, 17 Jan 2018 19:26:54 +0000 (14:26 -0500)]
Default to python2 for tox

This caused many ODL projects to break due to it now assuming
py3 by default. The only project that needs python3 tox run is
Acumos so let us default this back to python2 which is the OS level
default and explicitly set python3 in Acumos.

Change-Id: I97df6eee112a2e9e1e8d18eb9e16990a1c703538
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSwitch global-jjb to lfdocs-conf 12/8412/1
Thanh Ha [Wed, 17 Jan 2018 03:08:58 +0000 (22:08 -0500)]
Switch global-jjb to lfdocs-conf

Consolidate conf.py into lfdocs-conf package.

Change-Id: I18b975acaa2ef28dfdaeb9ebf29357a9ed58202d
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSet default archive patterns for maven-verify 99/8399/3
Thanh Ha [Tue, 16 Jan 2018 14:22:59 +0000 (09:22 -0500)]
Set default archive patterns for maven-verify

Adds some default archive patterns for the maven-verify job
template.

Change-Id: I18e85f859fbba088ab338672f4fb28e7a02787af
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add maven-merge job to maven job group"
Thanh Ha [Mon, 15 Jan 2018 19:15:23 +0000 (19:15 +0000)]
Merge "Add maven-merge job to maven job group"

6 years agoAdd maven-merge job to maven job group 98/8398/1
Thanh Ha [Mon, 15 Jan 2018 19:10:22 +0000 (14:10 -0500)]
Add maven-merge job to maven job group

Tested this for a couple weeks now. Let's start deploying it
more broadly.

Change-Id: Iddefb7a53668c311a41cc8b8bf216cb97e3c1bd2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd -e flag to mvn calls 97/8397/2
Thanh Ha [Mon, 15 Jan 2018 18:47:27 +0000 (13:47 -0500)]
Add -e flag to mvn calls

Enables full stack traces and display execution errors during
maven output.

Change-Id: Idb16eb1187bcf857530b6288ed8728de8cabdf69
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Do not force expected-xml"
Thanh Ha [Mon, 15 Jan 2018 19:05:50 +0000 (19:05 +0000)]
Merge "Do not force expected-xml"

6 years agoPrint file size before deploy maven file to nexus 94/8394/1
Anil Belur [Sun, 14 Jan 2018 23:11:45 +0000 (09:11 +1000)]
Print file size before deploy maven file to nexus

The size of the file is useful to determine if nexus deploy failures
are due to the file size exceeding the limits.

Change-Id: I1c0eda93ab964a30e37ea910a7a5859a65de8ee3
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoDo not force expected-xml 92/8392/1
Thanh Ha [Sat, 13 Jan 2018 17:05:34 +0000 (12:05 -0500)]
Do not force expected-xml

After using this for a few months now I'm convinced this is not the
right way to achieve what we want here. XML differences should be
informational and not build breaking. Pretty much every change
requires XML changes so I think this has been more annoying than
helpful.

This new patch removes expected-xml and instead outputs XML diffs
instead which can be stored in logs and reviewed.

Change-Id: Ia4039489502499df1b15a4d9fe1e553af3c5bfcd
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoFix default platform location to common-packer 91/8391/1
Thanh Ha [Sat, 13 Jan 2018 17:03:40 +0000 (12:03 -0500)]
Fix default platform location to common-packer

The default search path should be common-packer and only loading
the local project's version if it exists.

Change-Id: Iae1af0e804806ef99761c2e02a39214443a74b23
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump packer build timeout to 90 minutes 90/8390/2
Thanh Ha [Sat, 13 Jan 2018 04:09:06 +0000 (23:09 -0500)]
Bump packer build timeout to 90 minutes

With all of the consolidation of packaging into the new baseline
scripts as well as known slowness in Vexxhost builds for baseline
are now taking around 60 minutes to run. Bump to 90 minutes so that
there is a buffer for us to continue developing baseline.

Change-Id: I195a74d902111c937bebf7c8e599a3d641e74a37
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd a check for missing vars/*.json 88/8388/3
Thanh Ha [Fri, 12 Jan 2018 21:18:03 +0000 (16:18 -0500)]
Add a check for missing vars/*.json

If a project does not have any then we should detect it and skip
trying to parse those files.

Change-Id: I9fd1fee88a68e745709779de9fec3f2013b7483f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoUpdate expected xml to JJB 2.0 format 89/8389/1
Thanh Ha [Fri, 12 Jan 2018 21:56:34 +0000 (16:56 -0500)]
Update expected xml to JJB 2.0 format

Change-Id: I73ac8158a85559ec8d09682b8192aecb7f78ba8b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>