releng/global-jjb.git
5 years agoMerge "Ignore false-neg ShellCheck SC2154 unused vars" v0.27.0
Anil Belur [Thu, 15 Nov 2018 03:46:07 +0000 (03:46 +0000)]
Merge "Ignore false-neg ShellCheck SC2154 unused vars"

5 years agoMerge "Add error handling to stack script"
Thanh Ha (zxiiro) [Thu, 15 Nov 2018 02:38:58 +0000 (02:38 +0000)]
Merge "Add error handling to stack script"

5 years agoMerge "Allow JJB Cache directory to be configured"
Thanh Ha (zxiiro) [Thu, 15 Nov 2018 02:38:46 +0000 (02:38 +0000)]
Merge "Allow JJB Cache directory to be configured"

5 years agoAdd error handling to stack script 44/13544/1
Thanh Ha [Thu, 15 Nov 2018 01:26:54 +0000 (09:26 +0800)]
Add error handling to stack script

Handle case where JENKINS_URL does not return 200 ok.

Change-Id: Ie2067c3a5f3f489e25c361a4f417a6325e4abbd1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMigrate cleanup orphaned nodes cron script 84/13484/3
Thanh Ha [Tue, 13 Nov 2018 06:03:09 +0000 (14:03 +0800)]
Migrate cleanup orphaned nodes cron script

Generalize script for reuse in global-jjb.

Issue: RELENG-1434
Change-Id: I7901e58dbc0136c339a2d488c9eef8d46cd5446a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAllow JJB Cache directory to be configured 43/13343/2
Trevor Bramwell [Tue, 6 Nov 2018 00:30:38 +0000 (16:30 -0800)]
Allow JJB Cache directory to be configured

When JJB is ran concurrently on the same system with caching enabled, a
timeout can occur when one job fails after waiting 1 minute to aquire
the lock on the JJB cache directory.

Even if caching is disabled/ignored this timeout can still occur as JJB
always tries to lock the cache directory.

Having the ability to configure the cache directory will allow projects
to override it dynamically (with Jenkins env-vars) and run concurrent
jobs on the same node.

Change-Id: I1187847f658e754621369005e627a0f33a0253bc
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
5 years agoIgnore false-neg ShellCheck SC2154 unused vars 19/13519/1
Daniel Farrell [Wed, 14 Nov 2018 15:01:41 +0000 (15:01 +0000)]
Ignore false-neg ShellCheck SC2154 unused vars

ShellCheck 0.4.4 incorrectly fails on the ignored lines, thinking they
are vars that are declared but not used. It only fails one the first
line that uses a new prefix before the hyphen.

I tried using single/double/no quotes, all of which should be valid
bash, but ShellCheck fails on all.

Change-Id: I9ccca21a9d2504ecf0ec3f09707eb82353bd8bbc
Signed-off-by: Daniel Farrell <dfarrell@redhat.com>
5 years agoMigrate cleanup-stale-stacks cron script 80/13480/6
Thanh Ha [Tue, 13 Nov 2018 03:39:45 +0000 (11:39 +0800)]
Migrate cleanup-stale-stacks cron script

Generalize script for use in global-jjb.

Issue: RELENG-1432
Change-Id: I9c450c36b0f44a5583bee1932016e7cb3d5ef834
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd orphaned-volumes feature to release notes 79/13479/1
Thanh Ha [Tue, 13 Nov 2018 03:39:08 +0000 (11:39 +0800)]
Add orphaned-volumes feature to release notes

This was missed when If84c0f4341386ed758ce9d8556d4c4e574601797
was reviewed.

Change-Id: I529cc36ef19102207594adfe28531a7a1472092a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMigrate cleanup-stale-volumes cron script 56/13456/1
Thanh Ha [Mon, 12 Nov 2018 07:48:45 +0000 (15:48 +0800)]
Migrate cleanup-stale-volumes cron script

Generalize the cleanup script for general use in global-jjb.

Issue: RELENG-1435
Change-Id: If84c0f4341386ed758ce9d8556d4c4e574601797
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Allow image protection script to be disabled"
Thanh Ha (zxiiro) [Fri, 9 Nov 2018 23:49:46 +0000 (23:49 +0000)]
Merge "Allow image protection script to be disabled"

5 years agoMerge "Migrate cleanup old images script from ODL"
Thanh Ha (zxiiro) [Fri, 9 Nov 2018 17:26:35 +0000 (17:26 +0000)]
Merge "Migrate cleanup old images script from ODL"

5 years agoMerge "Refactor the common-view and csit-view template"
Anil Belur [Thu, 8 Nov 2018 03:50:36 +0000 (03:50 +0000)]
Merge "Refactor the common-view and csit-view template"

5 years agoAllow image protection script to be disabled 09/13409/3
Thanh Ha [Wed, 7 Nov 2018 20:22:55 +0000 (04:22 +0800)]
Allow image protection script to be disabled

Change-Id: Ie15fc59fba1a4efc475136bf30ed4c94045203d2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMigrate cleanup old images script from ODL 03/13403/6
Thanh Ha [Wed, 7 Nov 2018 20:22:55 +0000 (04:22 +0800)]
Migrate cleanup old images script from ODL

Generalize the cleanup script for reuse in global-jjb.

Issue: RELENG-1436
Change-Id: I88e1bc3136b2f9b2542e441a74f26d742679c353
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoInstall 'openstack' module for lftools 97/13397/2
Thanh Ha [Wed, 7 Nov 2018 19:28:39 +0000 (03:28 +0800)]
Install 'openstack' module for lftools

The lftools optional 'openstack' module will now be installed as part
of pre-build script.

Change-Id: I05bdcf07258ffcff9242c9ced10256d1cdaeef6d
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRm openstack-install for lf-infra-pre-build 96/13396/1
Thanh Ha [Wed, 7 Nov 2018 19:15:53 +0000 (03:15 +0800)]
Rm openstack-install for lf-infra-pre-build

lf-infra-pre-build is able to manage openstack client installs
so remove the openstack-install script. No need to release note since
no project should be using openstack-cron yet as in it's current
state is incomplete.

Change-Id: Ic6ee5ae0e5815d2730b3fa6154fd8e6d5b096844
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRefactor the common-view and csit-view template 68/13268/5
Thanh Ha [Wed, 31 Oct 2018 23:21:18 +0000 (19:21 -0400)]
Refactor the common-view and csit-view template

Make the 2 views easier to use by projects. common-view is now
project-view, the intent of the project-view is that it
automatically matches jobs starting with {project-name} to
automatically create a view tab for all of a single project's jobs.

The intent of the CSIT view template is to have a common template
that loads the most useful columns for CSIT views. Since we do
not know the search pattern and view name of these views ahead of
time it has 2 required paramaters ``view-name`` and ``view-regex``.

There is no need to force configuration in defaults.yaml for these 2
templates. For project-view it's intent is to load all project jobs
into a single view using a well known regex which does not need to
be modified. For csit-view we don't know how the regex is going to
be ahead of time so this view-template should just load the common
column configuration and require the view-template user to explicitly
declare their regex and view name.

Additionally add a new 'common-view' which accepts a view-name and
view-regex for users who want to pull in a default view-template
for their view.

Move documentation from release-notes to documentation. Release notes
should be about informing users about new features where detailed
documentation should instead to to the documentation of the views.

Change-Id: Ife3cfe2e37f019dfb87ac8e90b21afb6bdf06889
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoVerify jenkins-config files have matching images 83/13283/8
Eric Ball [Thu, 25 Oct 2018 21:54:32 +0000 (14:54 -0700)]
Verify jenkins-config files have matching images

This verify job will check that for each config file in
jenkins-config/clouds/**, there is an image on the cloud provider
that matches the IMAGE_NAME field.

Issue-ID: RELENG-1133
Change-Id: I9bf79657af27e8c7ff7b63ced2bd8a0dd544e6d6
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoPull in tox implicitly through detox 18/13318/1
Thanh Ha [Mon, 5 Nov 2018 07:23:44 +0000 (15:23 +0800)]
Pull in tox implicitly through detox

detox's own requirements.txt limits the version of tox that it pulls
in. This can often conflict with the explicit version we set.
Let's use the version of tox as required by detox.

Issue: https://jira.opendaylight.org/browse/RELENG-136
Change-Id: I0b98afc6c993279c17cdd02d67471fadde23e552
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMove required params to defaults section in views 96/13196/5
Anil Belur [Fri, 26 Oct 2018 05:08:43 +0000 (15:08 +1000)]
Move required params to defaults section in views

JIRA: RELENG-1393
Change-Id: I03b7244650f30daebba246f5af7af350309079ce
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd option to stage to Maven Central 18/12318/9
Thanh Ha [Sat, 18 Aug 2018 16:26:15 +0000 (12:26 -0400)]
Add option to stage to Maven Central

Provide an option to also stage to OSSRH in preparation to release
to Maven Central.

In addition to printing the staging repo-id also print the URL to
the staging repo for convenience in stage-repo.txt.

Issue: RELENG-1398
Issue: https://jira.opendaylight.org/browse/RELENG-28
Change-Id: I0628cb5c7185e2143e448ceb4fa4752384581a5e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoFix github-maven-merge scm config 27/13227/1
Jeremy Phelps [Mon, 29 Oct 2018 19:39:35 +0000 (14:39 -0500)]
Fix github-maven-merge scm config

Specifying a refspec on this scm config causes no merge job to be
triggered on a push action.

Change-Id: I46066dc6e371f49ed58dcc4f439cb0fae63808db
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
5 years agoMerge "Add view templates to manage Jenkins views"
Andrew Grimberg [Thu, 25 Oct 2018 15:20:32 +0000 (15:20 +0000)]
Merge "Add view templates to manage Jenkins views"

5 years agoDocument Gerrit Branch Lock job 90/13190/1
Thanh Ha [Thu, 25 Oct 2018 14:57:25 +0000 (10:57 -0400)]
Document Gerrit Branch Lock job

Change-Id: I2fddf938cc45bd852bf4e4f01a6e3836a4b79338
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd view templates to manage Jenkins views 89/12989/11
Anil Belur [Tue, 16 Oct 2018 01:26:40 +0000 (11:26 +1000)]
Add view templates to manage Jenkins views

Add view templates for common views and csit jobs.
Update docs and JJB tests.

JIRA: RELENG-1259
Change-Id: I0fda1c089ef516bfd3bb3fb04ea4660d410fb094
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Add submodule-timeout for submodule operations"
Thanh Ha (zxiiro) [Fri, 19 Oct 2018 19:10:19 +0000 (19:10 +0000)]
Merge "Add submodule-timeout for submodule operations"

5 years agoAdd submodule-timeout for submodule operations 79/12879/9
Jessica Wagantall [Thu, 4 Oct 2018 00:48:05 +0000 (17:48 -0700)]
Add submodule-timeout for submodule operations

Some ONAP projects using CLM (SDC) require more submodule
timeout to let the Nexus report run successfully. Allow an
option to provide a submodule-timeout to gerrit-maven-clm
keeping the default to 10 minutes.

Add submodule-timeout parameter for both lf-infra-gerrit-scm
and lf-infra-github-scm.

Main issue and tests described in the asociated Jira.

Issue: RELENG-1297
Change-Id: Ia7d292df9bac53d760767d9ee2caeec2f84c48ac
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoMerge "Bump lftools version to v0.18.0"
Andrew Grimberg [Fri, 19 Oct 2018 17:05:23 +0000 (17:05 +0000)]
Merge "Bump lftools version to v0.18.0"

5 years agoBump lftools version to v0.18.0 88/13088/1
Thanh Ha [Fri, 19 Oct 2018 16:28:16 +0000 (12:28 -0400)]
Bump lftools version to v0.18.0

Change-Id: I999f2a79044dadfb543fe36d8a506d115d3cdf99
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoFix jenkins-init scripts dir 29/12729/2
Thanh Ha [Fri, 21 Sep 2018 23:41:20 +0000 (19:41 -0400)]
Fix jenkins-init scripts dir

Resolve issue caused by I5060dd0c1b67fa020a84844936eebb970ae2cbb6
which moved the global-jjb directory around and caused scripts
that depended on the old path to start failing.

Change-Id: I705a8c79bd69c9fdaff8dc677e0f22cb6b42d74b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMake a working publisher for Windows builds 78/13078/2 v0.26.0
Thanh Ha [Thu, 18 Oct 2018 20:22:40 +0000 (16:22 -0400)]
Make a working publisher for Windows builds

This publisher is missing sysstat / shiplogs but we will have to
develop them for Windows machines. In the meantime at least deploy
a builder that works.

Issue: RELENG-1228
Change-Id: I427779c8e580abf6126c04032ca527a5ffd627bf
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd conditional build step to use clouds.yaml 73/13073/3
Jeremy Phelps [Thu, 18 Oct 2018 19:15:30 +0000 (14:15 -0500)]
Add conditional build step to use clouds.yaml

This was missed in patch Idf23a30b111dd922120fb6195d0411e2fc993dab
when we were implementing this feature.

Issue: RELENG-1205
Change-Id: Id2017172293463c616158b79f3af826ea754fa08
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
5 years agoUpdate reno to 2.11.2 66/13066/2
Thanh Ha [Thu, 18 Oct 2018 18:18:23 +0000 (14:18 -0400)]
Update reno to 2.11.2

Issue: RELENG-1355
Change-Id: I1bbd3be83f52d63bcde956f392c3396fa236b329
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoSet PATH in Ubuntu 18.04 VM's 12/13012/4
Jeremy Phelps [Tue, 16 Oct 2018 21:26:46 +0000 (16:26 -0500)]
Set PATH in Ubuntu 18.04 VM's

This is a workaround for a bash 3.4 bug that does not
set ~/.local/bin in PATH.

Issue-ID: RELENG-1320
Change-Id: I94bcc14f1dcfd1e813e09d1bf448d465549ea04c
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
5 years agoMerge "Allow maven-clm job to specify AppID namespace"
Thanh Ha (zxiiro) [Tue, 16 Oct 2018 16:19:41 +0000 (16:19 +0000)]
Merge "Allow maven-clm job to specify AppID namespace"

5 years agoClarify git-url and git-clone-url 88/12988/3
Jeremy Phelps [Mon, 15 Oct 2018 23:52:01 +0000 (18:52 -0500)]
Clarify git-url and git-clone-url

Issue-ID: RELENG-446
Change-Id: Iac125015c0ed9b7e22feef604a14b21dd5d607d3
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
5 years agoAllow maven-clm job to specify AppID namespace 86/12986/1
Thanh Ha [Mon, 15 Oct 2018 22:55:48 +0000 (18:55 -0400)]
Allow maven-clm job to specify AppID namespace

This allows maven-clm jobs to insert a project namespace into the
AppID field in Nexus IQ in cases where a Nexus IQ server might be
shared by multiple projects and name collision might be an issue.

This patch inserts an optional nexus-iq-namespace parameter which
by default is blank. This should not break backwards compatibility
while allowing the new namespace feature to work. We recommend when
using the namespace to add a trailing - to the value. Eg. 'odl-',
this is to make the namespace look nice for example "odl-aaa" will be
the result of namespace odl-, and project is aaa.

Issue: RELENG-1307
Change-Id: If4313ba00da67541d337b70f7946ad6f21fee037
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoFix deprecation warning to not execute code 58/12958/1
Daniel Farrell [Thu, 11 Oct 2018 21:02:03 +0000 (17:02 -0400)]
Fix deprecation warning to not execute code

The warning about lftools-install script being deprecated was quoted
incorrectly, so the `code` it suggested was actually executed. This
caused this script to fail, resulting in builds being marked unstable
and logs failing to deploy.

Change-Id: I394baf37a1a82d54fdbee62010af5be0be5527ff
Signed-off-by: Daniel Farrell <dfarrell@redhat.com>
5 years agoAdd lf flavors 06/12906/5
Jeremy Phelps [Tue, 9 Oct 2018 16:24:51 +0000 (11:24 -0500)]
Add lf flavors

The lf flavors are the same as the v2 flavors with the addition of
a 30gb root disk attached. Also reordered the list.

Change-Id: I9989759328b0d6d22ebf7c870e9f4dcbcdcfa253
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
5 years agoAdd additional link steps for global-jjb install 95/12895/2
Jeremy Phelps [Fri, 5 Oct 2018 17:12:07 +0000 (12:12 -0500)]
Add additional link steps for global-jjb install

Change-Id: I8e81f0924dc8561d6cf2f33e97f6f696c8233382
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
5 years agoSkip vote for releng-rtd from gerrit trigger 61/12861/3
Aric Gardner [Tue, 2 Oct 2018 20:49:38 +0000 (16:49 -0400)]
Skip vote for releng-rtd from gerrit trigger

There should be an easy way to set jobs coming from global jjb to be
non-voting. I would like to add the rtd job as non
voting first and then after some time switch it to voting.

Change-Id: Id96886973c180e47f79c0550f8c35749715617a3
Jira: RELENG-1294
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
5 years agoFix typos in docs 79/12779/1
Anil Belur [Wed, 26 Sep 2018 12:54:20 +0000 (18:24 +0530)]
Fix typos in docs

Change-Id: I1a12cebe248c368bab73abd50cc5045214435b8f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd fix note for jjb-cleanup change 18/12718/1 v0.25.1
Andrew Grimberg [Fri, 21 Sep 2018 17:45:39 +0000 (10:45 -0700)]
Add fix note for jjb-cleanup change

Change-Id: I676b953d2906bf4e9cf32bbd2625359540173a25
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoMake sure that jjb-cleanup.sh allows unbound vars 12/12712/1
Andrew Grimberg [Fri, 21 Sep 2018 15:30:28 +0000 (08:30 -0700)]
Make sure that jjb-cleanup.sh allows unbound vars

Due to our merging of scripts in some of the build steps it's possible
for this script to end up merged in with other scripts that are setting
-u, this will then cause the jjb-cleanup to fail. As such, we need to
explicitly allow unbound variables to be safe

Change-Id: I312e9d737bffefe30a5238809dd03bde1eb0ca6b
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoMerge "Trigger jjb-verify on all changes in jjb/shell dir" v0.25.0
Thanh Ha (zxiiro) [Fri, 21 Sep 2018 13:53:55 +0000 (13:53 +0000)]
Merge "Trigger jjb-verify on all changes in jjb/shell dir"

5 years agoTrigger jjb-verify on all changes in jjb/shell dir 84/12684/2
Anil Belur [Thu, 20 Sep 2018 15:54:09 +0000 (21:24 +0530)]
Trigger jjb-verify on all changes in jjb/shell dir

Replace single file regex with change to any file in
shell dir.

Change-Id: I0ca9a9d1bddd2c21034e70c1413f7d2a64d8ee58
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd a Windows wrapper macro 51/12651/4
Thanh Ha [Tue, 18 Sep 2018 22:38:14 +0000 (18:38 -0400)]
Add a Windows wrapper macro

We currently don't support ssh-agents on Windows. Split the wrapper
so that we have a Windows specific one.

Issue: RELENG-939
Change-Id: I0474206ddc706d8bb5c5c6762237bf227dc6cf82
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd macro to delete an openstack heat stack 72/12672/3
Thanh Ha [Thu, 20 Sep 2018 01:07:25 +0000 (21:07 -0400)]
Add macro to delete an openstack heat stack

This is migrated from OpenDaylight's releng/builder project.

Issue: RELENG-236
Change-Id: I3da707ea8315c861183b536114a7e32595e78c1f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoPin Sphinx to ~1.7.9 as 1.8.0 breaks us 73/12673/1
Thanh Ha [Thu, 20 Sep 2018 01:30:30 +0000 (21:30 -0400)]
Pin Sphinx to ~1.7.9 as 1.8.0 breaks us

Change-Id: I66725c168ba7e8d06c6d9d8186af461c98dd7815
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoTrigger jjb-verify on all changes in jjb dir 64/12664/2
Thanh Ha [Wed, 19 Sep 2018 19:11:07 +0000 (15:11 -0400)]
Trigger jjb-verify on all changes in jjb dir

Everything in the global-jjb directory should trigger jjb-verify
jobs. Update the regex to be less specific.

Change-Id: Ie0b6c5b60e3547e1f3616c8dde7493be6d6184ff
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoSet login shell for common-variables.sh 24/12624/1
Thanh Ha [Mon, 17 Sep 2018 13:23:01 +0000 (09:23 -0400)]
Set login shell for common-variables.sh

common-variables.sh is pulled in via list action before other
shell scripts so needs a login shell set so that the scripts
that come after correctly get their PATHs set.

Issue: RT 60957
Change-Id: If33d1689d1be590cfe0e557335225be18b5103f7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoSplit pip and setuptools install to 2 calls 07/12607/1
Thanh Ha [Fri, 14 Sep 2018 15:29:55 +0000 (11:29 -0400)]
Split pip and setuptools install to 2 calls

We seem to be getting more success installing pip and setuptools this
way and hopefully fixes the post-build scripts to not require all
jobs that use python to need to update their virtualenv calls.

Change-Id: I0fba7ba0f574cb29561fcfdb1404272c67165582
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoDeprecate lftools-install.sh script 99/12599/1
Thanh Ha [Fri, 14 Sep 2018 02:18:23 +0000 (22:18 -0400)]
Deprecate lftools-install.sh script

This script if used without calling `deactivate` gets in the way of
our post-build scripts and causes UNSTABLE build. Deprecate it as
we are no longer using it in global-jjb.

Change-Id: I84654b20490d5aa7e84e5e7330a59936dc7a6f0d
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoUpdate jobs to use lf-infra-pre-build 98/12598/1
Thanh Ha [Thu, 13 Sep 2018 23:38:34 +0000 (19:38 -0400)]
Update jobs to use lf-infra-pre-build

Use lf-infra-pre-build to install lftools via --user config. This
resolves the post-build issues caused by `pip install` due to
virtualenvs messing with the environment.

Change-Id: Iccd72e6f1d580037f55bff3199d05d467de17c58
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Update rtd jobs to use lf-infra-pre-build"
Jessica Wagantall [Fri, 14 Sep 2018 02:13:28 +0000 (02:13 +0000)]
Merge "Update rtd jobs to use lf-infra-pre-build"

5 years agoMerge "Use --user for tox install instead of virtualenv"
Jessica Wagantall [Fri, 14 Sep 2018 02:08:55 +0000 (02:08 +0000)]
Merge "Use --user for tox install instead of virtualenv"

5 years agoUpdate rtd jobs to use lf-infra-pre-build 92/12592/3
Thanh Ha [Fri, 14 Sep 2018 00:28:44 +0000 (20:28 -0400)]
Update rtd jobs to use lf-infra-pre-build

Use lf-infra-pre-build to install lftools via --user config. This
resolves the post-build install failing with missing PBR version.

Change-Id: I3aa48d98811a7469fba15fe4c3eb7474bc7b5196
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoUse --user for tox install instead of virtualenv 86/12586/3
Thanh Ha [Thu, 13 Sep 2018 19:31:30 +0000 (15:31 -0400)]
Use --user for tox install instead of virtualenv

Allows us to remove the workaround of installing lftools in
a venv before as the virtualenvs should no longer be conflicting.

Change-Id: Ic466cd74a568e0de2fbb0f3fd75205cd04c1e3f9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Update lftools to ~ 0.17.1"
Jessica Wagantall [Fri, 14 Sep 2018 02:01:02 +0000 (02:01 +0000)]
Merge "Update lftools to ~ 0.17.1"

5 years agoUse `python -m pip` to ensure updated pip is used 91/12591/1
Thanh Ha [Fri, 14 Sep 2018 00:04:51 +0000 (20:04 -0400)]
Use `python -m pip` to ensure updated pip is used

/usr/bin/pip is usually an OS level wrapper for pip. Calling
`python -m pip` ensures that we are getting the pip we are
installing rather than the OS local version.

Change-Id: I054d1cc0455c700e00450295cf76d925cff99f73
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoUpdate lftools to ~ 0.17.1 90/12590/1
Thanh Ha [Fri, 14 Sep 2018 00:02:07 +0000 (20:02 -0400)]
Update lftools to ~ 0.17.1

Change-Id: I69a2e669493b0f309e161f07152d93b0fa0c9ac6
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Use clouds.yaml for openstack packer builders"
Anil Belur [Thu, 13 Sep 2018 15:56:24 +0000 (15:56 +0000)]
Merge "Use clouds.yaml for openstack packer builders"

5 years agoMerge "Add sigul signing to maven-stage job"
Andrew Grimberg [Wed, 12 Sep 2018 18:52:34 +0000 (18:52 +0000)]
Merge "Add sigul signing to maven-stage job"

5 years agoFix package list script to not fail on diff 58/12558/1
Thanh Ha [Wed, 12 Sep 2018 17:48:19 +0000 (13:48 -0400)]
Fix package list script to not fail on diff

The diff command exits 1 if there is a difference between 2 files.
This is perfectly valid so ignore the exit code for this command.

Change-Id: I0da9d6bb99a5a0a9666697b27416fa5e5518e316
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoUse clouds.yaml for openstack packer builders 50/12550/4
Thanh Ha [Wed, 12 Sep 2018 00:03:52 +0000 (20:03 -0400)]
Use clouds.yaml for openstack packer builders

Since packer version 1.2.5, packer now supports pulling openstack
config from clouds.yaml. Let's consolidate to that file so that we
don't have duplicate configuration with the jobs that need the
openstack cli.

Issue: RELENG-1205
Change-Id: Idf23a30b111dd922120fb6195d0411e2fc993dab
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd sigul signing to maven-stage job 12/12312/10
Thanh Ha [Sat, 18 Aug 2018 02:15:07 +0000 (22:15 -0400)]
Add sigul signing to maven-stage job

Refactor the lf-sigul* macros into a new one called lf-sigul-sign-dir
to be used as a one shot macro to sign a directory. Macro initializes
and cleans up after itself so no need to call extra builder steps.

Issue: RELENG-1071
Change-Id: I0ff965c2ef24fa7a44a114445d8a97c3176e575a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoChange gjjb install to use symlinks in jjb-dir 28/12528/3
Thanh Ha [Mon, 10 Sep 2018 22:00:19 +0000 (18:00 -0400)]
Change gjjb install to use symlinks in jjb-dir

Update global-jjb install instructions to symlink the jjb and shell
directories in global-jjb and clone the submodule to the root of
ci-management instead.

This allows us to use tools that have YAML file configurations that
JJB will otherwise pick up and parse as invalid JJB config.

Issue: RELENG-1204
Change-Id: I5060dd0c1b67fa020a84844936eebb970ae2cbb6
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoUse Reno for release notes 04/12504/1
Thanh Ha [Fri, 7 Sep 2018 23:07:23 +0000 (19:07 -0400)]
Use Reno for release notes

Change-Id: I4ac1ce147ada7176cdac399b83edc646973dd80e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Refactor job-groups documentation"
Andrew Grimberg [Thu, 6 Sep 2018 15:08:12 +0000 (15:08 +0000)]
Merge "Refactor job-groups documentation"

5 years agoMerge "Remove maven-merge job todo"
Andrew Grimberg [Thu, 6 Sep 2018 15:05:45 +0000 (15:05 +0000)]
Merge "Remove maven-merge job todo"

5 years agoStrip out trailing metadata from version 61/12461/1
Thanh Ha [Wed, 5 Sep 2018 16:42:01 +0000 (12:42 -0400)]
Strip out trailing metadata from version

When manually building maven plugins maven puts in some metadata
like "2.38-SNAPSHOT (private-9853d407-zxiiro)" which throws off the
version_ge function as it does not know how to handle a version with
a space in the name. This change strips out only the leading version
ignoring everything after a space.

Change-Id: I02f0397ae5627f65638a456ba2f1304dd9a16de9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRefactor job-groups documentation 95/12395/2
Thanh Ha [Sat, 25 Aug 2018 15:21:27 +0000 (11:21 -0400)]
Refactor job-groups documentation

Improve the way job-groups are documented so that it stays properly
in sync. This update recommends projects define their own job-groups
and using the ones defined in global-jjb only as a starting point.

Update ensure-documented to ignore job-groups since they should be
self documenting now in the new format.

Change-Id: I945735ecd50596605139c6522903a49350a1ea10
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRemove maven-merge job todo 94/12394/1
Thanh Ha [Sat, 25 Aug 2018 14:54:52 +0000 (10:54 -0400)]
Remove maven-merge job todo

This is completed.

Change-Id: Iddc98ccc4d4f8f3bfc3650eec3b5bab7f156db3b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRelease global-jjb v0.24.0 86/12386/3 v0.24.0
Thanh Ha [Fri, 24 Aug 2018 16:43:42 +0000 (12:43 -0400)]
Release global-jjb v0.24.0

Change-Id: I5cc5be89985e0fcb99cb256053282b5e740b05c1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRe-enable CLM jobs 88/12388/1
Thanh Ha [Fri, 24 Aug 2018 18:49:55 +0000 (14:49 -0400)]
Re-enable CLM jobs

Change-Id: I1816f34e964ff56cba3e5d7af693993c97b1cc7d
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Add pre-build step to prep vm before build"
Jeremy Phelps [Fri, 24 Aug 2018 14:42:07 +0000 (14:42 +0000)]
Merge "Add pre-build step to prep vm before build"

5 years agoAdd option to disable-job for all jobs 77/12377/3
Thanh Ha [Thu, 23 Aug 2018 22:47:00 +0000 (18:47 -0400)]
Add option to disable-job for all jobs

Add option to all jobs to allow disabling the job. CLM will has this
setting set to "true" while we figure out the CLM issues however
for all other jobs "false" is default.

Change-Id: I9db6e8603a7d968aacf0eae17dea3554d6c97429
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Disable CLM jobs by default"
Thanh Ha (zxiiro) [Thu, 23 Aug 2018 19:50:08 +0000 (19:50 +0000)]
Merge "Disable CLM jobs by default"

5 years agoDisable CLM jobs by default 75/12375/1
Jeremy Phelps [Thu, 23 Aug 2018 19:12:19 +0000 (14:12 -0500)]
Disable CLM jobs by default

Change-Id: I8c3b4a68aaed349d026d08134de4147ea74a9696
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
5 years agoAdd pre-build step to prep vm before build 50/12150/4
Thanh Ha [Wed, 8 Aug 2018 00:27:59 +0000 (20:27 -0400)]
Add pre-build step to prep vm before build

This new build step will run before any jobs run to prep the
builder.

This change should not force using of $HOME/.local/bin just yet
but preps the macros in place for us to activate it in jobs later.
The current virtualenv method should override what's done here.

Change-Id: I55947020e2f0cdbb61f9d4e7e88927d9bc304d17
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoSet permit-all true for all jobs 44/12344/1
Thanh Ha [Tue, 21 Aug 2018 16:45:44 +0000 (12:45 -0400)]
Set permit-all true for all jobs

We need this set for all jobs otherwise GHPR will spam GitHub.
This was set for verify jobs but the merge jobs did not have them.

Change-Id: I76db9d6146228180c9af3f06780705dac33b7899
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Move openstack-cron job to global-jjb"
Andrew Grimberg [Mon, 20 Aug 2018 17:05:24 +0000 (17:05 +0000)]
Merge "Move openstack-cron job to global-jjb"

5 years agoMerge "Require IMAGE_NAME and HARDWARE_ID"
Andrew Grimberg [Mon, 20 Aug 2018 17:00:49 +0000 (17:00 +0000)]
Merge "Require IMAGE_NAME and HARDWARE_ID"

5 years agoMerge "Add job to verify upstream global-jjb patches"
Andrew Grimberg [Mon, 20 Aug 2018 16:19:08 +0000 (16:19 +0000)]
Merge "Add job to verify upstream global-jjb patches"

5 years agoMerge "Remove mvn-settings requirement from jjb job docs"
Andrew Grimberg [Mon, 20 Aug 2018 15:36:53 +0000 (15:36 +0000)]
Merge "Remove mvn-settings requirement from jjb job docs"

5 years agoMerge "Refactor staging-profile-id out of mvn-parameters"
Andrew Grimberg [Mon, 20 Aug 2018 15:29:05 +0000 (15:29 +0000)]
Merge "Refactor staging-profile-id out of mvn-parameters"

5 years agoMerge "Make staging jobs run daily"
Andrew Grimberg [Mon, 20 Aug 2018 15:27:31 +0000 (15:27 +0000)]
Merge "Make staging jobs run daily"

5 years agoMerge "Make maven-merge jobs run daily"
Andrew Grimberg [Mon, 20 Aug 2018 15:26:40 +0000 (15:26 +0000)]
Merge "Make maven-merge jobs run daily"

5 years agoRequire IMAGE_NAME and HARDWARE_ID 26/12326/1
Thanh Ha [Mon, 20 Aug 2018 15:18:41 +0000 (11:18 -0400)]
Require IMAGE_NAME and HARDWARE_ID

IMAGE_NAME had HARDWARE_ID are required settings so clearly mark
them as so instead of mentioning the default setting.

Change-Id: I30a00f1c92f7714f7ac8a308d12a6181fe4547cd
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Document the jjb-version variable"
Andrew Grimberg [Mon, 20 Aug 2018 15:18:30 +0000 (15:18 +0000)]
Merge "Document the jjb-version variable"

5 years agoMake maven-merge jobs run daily 21/12321/1
Thanh Ha [Sat, 18 Aug 2018 20:54:02 +0000 (16:54 -0400)]
Make maven-merge jobs run daily

Back when OpenDaylight was on Rackspace we were trying to cut back
on how often jobs run due to the infra being unstable and at the
time thought the maven-integration job and maven-merge jobs were
part of the problem. Since we moved to Vexxhost infra has been
significantly more stable and not as big a concern. Let's run
the maven-merge jobs on a daily basis so that project artifacts
have more chances to pass before the 3 week SNAPSHOT clearing
cron that's run in Nexus.

Change-Id: I7ac17b3b4b8729d0f143fb15c71143c3de1ac461
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMake staging jobs run daily 20/12320/1
Thanh Ha [Sat, 18 Aug 2018 20:50:58 +0000 (16:50 -0400)]
Make staging jobs run daily

As we move towards more automated releases having regularly running
staging jobs will allow projects to discover broken staging jobs
earlier. Let's run these jobs @daily as a default.

Change-Id: I50a3f27396fc921714d1faff16830b2e6b38730d
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRefactor staging-profile-id out of mvn-parameters 19/12319/1
Thanh Ha [Sat, 18 Aug 2018 20:44:52 +0000 (16:44 -0400)]
Refactor staging-profile-id out of mvn-parameters

This was originally a bad design decision. We should not be forcing
staging-profile-id variable against all users of
lf-infra-maven-parameters when only the staging job needs it.

This patch moves custom parameter definitions to the maven-stage job
and removes staging-profile-id requirement from all other jobs.

Change-Id: I58fe4c43c851cd9e61ddec847529b0dcb91e0af1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAllow unbound variables in sigul scripts 14/12314/1
Thanh Ha [Sat, 18 Aug 2018 02:30:02 +0000 (22:30 -0400)]
Allow unbound variables in sigul scripts

These scripts as they are designed expect unbound variables so
enable them at least until they are fixed.

Change-Id: I48182a1d5d5c3cf4d42249bfc8a55b7c0088ec36
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoUse include-raw instead of escape for sigul macros 13/12313/1
Thanh Ha [Sat, 18 Aug 2018 02:23:40 +0000 (22:23 -0400)]
Use include-raw instead of escape for sigul macros

These macros are not using any substitutions so are hitting
the bug that inserts curly braces.

Change-Id: I7af376c3c4ce1e2cd41ca4842e5042728cdafb69
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoFix typo in lf-sigual-install builders syntax 11/12311/1
Thanh Ha [Sat, 18 Aug 2018 02:13:53 +0000 (22:13 -0400)]
Fix typo in lf-sigual-install builders syntax

Change-Id: I8a75a63a577500783d7a5f22f1833a2f6451f3d5
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMove openstack-cron job to global-jjb 89/12089/10
Thanh Ha [Thu, 2 Aug 2018 22:23:02 +0000 (18:23 -0400)]
Move openstack-cron job to global-jjb

Move OpenDaylight's builder-openstack-cron job to global-jjb. This is
an initial patch that only migrates the "image-protection" script.

image-protection is used to parse jenkins-config/ and jjb/ directories
for any mention of images starting with "ZZCI" and flagging them as
protected. The assumption here is that ZZCI images are CI managed
and if they are referred to anywher by anything then they are
effectively considered in-use and should not be deletable so flag
them as "protected: true" in OpenStack.

Change-Id: Ib706663f0c08bd7a9160b0a6ce69dcd3b802f8f5
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd job to verify upstream global-jjb patches 93/12293/3
Thanh Ha [Fri, 17 Aug 2018 00:24:27 +0000 (20:24 -0400)]
Add job to verify upstream global-jjb patches

Add job that can be used by downstream projects to report upstream
to global-jjb patches when a proposed patch breaks their project.

This job sets -1 CR only and does not block the upstream patch as
there are cases where we may want to break backwards compatibility
on purpose. This job's purpose is to give us a clear view when it
does happen.

Change-Id: If0dda88ff24da7fd171484c37855f33b05f36553
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoDocument the jjb-version variable 10/12310/1
Thanh Ha [Sat, 18 Aug 2018 01:39:42 +0000 (21:39 -0400)]
Document the jjb-version variable

Change-Id: Ia07cb6b437e6e2107b8301d52f72aa3cace112da
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>