releng/global-jjb.git
5 years agoUpdate sigul macro to support sign mode 12/14812/8
Thanh Ha [Wed, 6 Mar 2019 17:36:29 +0000 (12:36 -0500)]
Update sigul macro to support sign mode

Sign mode is a parameter to allow the user to choose whether or not
to sign artifacts in parallel using gnu-parallel or in serial mode
(bash for loop) methods.

Cleanup obsolete lftools_activate method.

Change-Id: Ib202f93a37e2830e5048c5a766c17d75233c6423
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoUpdate lftools to version 0.21.0 21/14821/1
Thanh Ha [Thu, 7 Mar 2019 18:28:55 +0000 (13:28 -0500)]
Update lftools to version 0.21.0

Change-Id: Ibbcd6b25247dcad926fc67342f9eac47347902e0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Add cmake-sonar job"
Thanh Ha (zxiiro) [Wed, 6 Mar 2019 15:44:09 +0000 (15:44 +0000)]
Merge "Add cmake-sonar job"

5 years agoMinor cleanup of openstack-protect script 67/14767/3
Tim Johnson [Tue, 5 Mar 2019 17:13:47 +0000 (09:13 -0800)]
Minor cleanup of openstack-protect script

Change-Id: I45070e0b364136b28e4130aed3972603e8c2e3c1
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoRework image code to handle no results from YAML 63/14763/4
Thanh Ha [Tue, 5 Mar 2019 01:47:38 +0000 (20:47 -0500)]
Rework image code to handle no results from YAML

When grep has 0 matches it returns a non-zero exit code. In this case
a 0 match grep is a possibility for YAML files so ignore grep's exit
code in this instance.

Change-Id: Ia7eb55364feeb42bf35681c4ae0798fe25bbfad1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoSearch for CFG & YAML files in specific dirs 62/14762/1
Thanh Ha [Tue, 5 Mar 2019 00:55:02 +0000 (19:55 -0500)]
Search for CFG & YAML files in specific dirs

These file locations are standard locations in our CI setup so be
specific of where to search for them. This prevents the scripts
from picking up data from test files in global-jjb.

Change-Id: Ia486ae8704b77f9303b5a2e54c4ffaf5de41aef8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoBump lftools to v0.20.0 58/14758/1 v0.32.0
Thanh Ha [Mon, 4 Mar 2019 20:34:29 +0000 (15:34 -0500)]
Bump lftools to v0.20.0

Change-Id: I6fa72595c1843a1955f4b492c035ebc7863163b5
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd niet needed for artifcat release job 95/14695/2
Aric Gardner [Tue, 26 Feb 2019 14:36:58 +0000 (09:36 -0500)]
Add niet needed for artifcat release job

Change-Id: I4223d00667b5701377d057e76e82e8657960d262
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
5 years agoAdd Error Handling 02/14602/1 v0.31.0
Tim Johnson [Fri, 15 Feb 2019 18:21:52 +0000 (10:21 -0800)]
Add Error Handling

Packer Cleanup Script was ignoring all errors

Change-Id: I881abf36420a1976c4f3af552aeeca4a4512c555
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoMerge "Update lf-maven-stage macro with missing parts"
Thanh Ha (zxiiro) [Wed, 13 Feb 2019 17:52:56 +0000 (17:52 +0000)]
Merge "Update lf-maven-stage macro with missing parts"

5 years agoMerge "Add job to automatically announce lf releases"
Thanh Ha (zxiiro) [Tue, 12 Feb 2019 15:42:39 +0000 (15:42 +0000)]
Merge "Add job to automatically announce lf releases"

5 years agoAdd job to automatically announce lf releases 69/13269/6
Thanh Ha [Thu, 1 Nov 2018 03:39:39 +0000 (23:39 -0400)]
Add job to automatically announce lf releases

This job triggers when a Gerrit tag is pushed and sends an email to
lf-releng@lists.linuxfoundation.org to notifiy the release.

Change-Id: I5255ac6f3afbcadfc11e9640483613bd2e84250a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoUpdate lf-maven-stage macro with missing parts 93/14393/9
Bengt Thuree [Thu, 31 Jan 2019 00:14:03 +0000 (11:14 +1100)]
Update lf-maven-stage macro with missing parts

Added the following missing parts to the lf-maven-stage macro.

* lf-provide-maven-settings
* lf-infra-create-netrc
* lf-provide-maven-settings-cleanup

This improves the macro to be a complete function that does not require
preparatory steps to complete it.

New Usage:

      - lf-maven-stage:
          mvn-global-settings: 'global-settings'
          mvn-settings: 'settings'
          mvn-staging-id: 'staging-profile-id'

This patch also updates the lf-maven-central macro as necessary to
work along side the lf-maven-stage changes.

Issue: RELENG-1554
Change-Id: Ie4fbcb52229c74c4216df4d07f077cd691ab6aa6
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
5 years agoAdd cmake-sonar job 09/14509/1
Thanh Ha [Sat, 9 Feb 2019 22:02:00 +0000 (17:02 -0500)]
Add cmake-sonar job

This job only supports SonarCloud configuration.

Change-Id: I4f2b5100e3316f21c866155199aeed0533d13b8b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd support for SonarCloud 78/14478/3
Thanh Ha [Thu, 7 Feb 2019 00:32:00 +0000 (19:32 -0500)]
Add support for SonarCloud

If SonarCloud is used the old method is automatically disabled so that
Sonar is not run twice.

Example Usage:

- project:
    name: yangtools-sonar
    jobs:
      - gerrit-maven-sonar

    project: 'yangtools'
    project-name: 'yangtools'
    branch: 'master'
    mvn-settings: 'yangtools-settings'
    mvn-opts: '-Xmx1024m'
    sonarcloud: true
    sonarcloud-project-key: KEY
    sonarcloud-project-organization: ORGANIZATION
    sonarcloud-api-token: TOKEN

Issue: RELENG-1721
Change-Id: Ica1cd29bfc9246f2452370dd6ceb8b7e5b8e5329
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoModify cron entry 77/14477/6 v0.30.0
Suresh Channamallu [Wed, 6 Feb 2019 21:27:10 +0000 (16:27 -0500)]
Modify cron entry

Modify cron entry to pick from custom config

Issue-Id: RELENG-1728
Change-Id: Icd4d97a37672fd0dece6960fa6de7d9fa2fe8a1e
Signed-off-by: Suresh Channamallu <schannamallu@linuxfoundation.org>
5 years agoVerify build node HARDWARE_ID 12/14412/4
Eric Ball [Thu, 31 Jan 2019 23:18:21 +0000 (15:18 -0800)]
Verify build node HARDWARE_ID

This script already checks each file's IMAGE_NAME for validity; this
adds essentially the same validation to check the HARDWARD_ID.
Also add the job to the ci-jobs groups, and set submodule-recursive
value for github-jenkins-cfg-verify.

Issue: RELENG-1418
Change-Id: I0266a0fcc276b2960c9c1749a89bc70e8115d152
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoAdd Release Notes for Packer Build Description 59/14359/1
Trevor Bramwell [Tue, 29 Jan 2019 18:54:09 +0000 (10:54 -0800)]
Add Release Notes for Packer Build Description

This was missed in the previous commit to add packer images to the
Jenkins build description.

Issue: RELENG-1073
Change-Id: Iced92ada13d0f576a7b560df5bc1242d83ee9689
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
5 years agoFix disable-job param typo in docs 32/14332/1
Anil Belur [Mon, 28 Jan 2019 08:56:20 +0000 (18:56 +1000)]
Fix disable-job param typo in docs

Change-Id: I28011767f45b0d29743c74cbec15f55bc48f603b
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoSet Packer Image name as Build Description 86/11786/2
Trevor Bramwell [Fri, 13 Jul 2018 23:08:36 +0000 (16:08 -0700)]
Set Packer Image name as Build Description

With the Build Description Setter plugin installed in Jenkins, the
packer image name can be extracted after a successful build.

This allows users to no longer look through the build logs when trying
to find the image name for builds.

Issue: RELENG-1073
Change-Id: Iffebd2b7e6b282d3e17a7bd6de1452e063f64603
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
5 years agoExtend ${JOB_NAME} to include {java-version} param 33/14233/3
Anil Belur [Thu, 17 Jan 2019 10:42:54 +0000 (20:42 +1000)]
Extend ${JOB_NAME} to include {java-version} param

To support javadocs jobs to build with multiple versions of
openjdk{8,11}, extend with '{java-version}' var to the ${JOB-NAME}.
Extending the job name with (java-version) var, JJB ensures
that its not passing the multiple java-versions as a list.

The job failure can be seen on the lispflowmapping jobs.
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/lispflowmapping-maven-javadoc-verify-neon/71/
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/lispflowmapping-maven-javadoc-publish-neon/19

Change-Id: If2e189f3e76f36d73863d98f1217c0b42ada52f1
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoFix coala warning 69/14169/1 v0.29.1
Anil Belur [Mon, 14 Jan 2019 12:08:25 +0000 (22:08 +1000)]
Fix coala warning

The warning was introduced with the change c90444b3
which fails the tox verify jobs.

Change-Id: I4b6241d53820f588b80f54b28bc92b635fda1b4f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd release notes for checking openjdk $VERSION 37/14137/2 v0.29.0
Anil Belur [Thu, 10 Jan 2019 23:36:50 +0000 (09:36 +1000)]
Add release notes for checking openjdk $VERSION

The release notes is for I19d77c509c90bca5fea1af53dacc35772c7ac939.

Change-Id: I81d0b2082376517ec26c978d6baa3fbd4e2e3d48
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoFix version number extraction for all cases 14/14114/1
Anil Belur [Thu, 10 Jan 2019 01:03:11 +0000 (11:03 +1000)]
Fix version number extraction for all cases

When the jobs upgrade to a later version of Java passing
"openjdk11" or "openjdk10" the string substitution would only
return the last char which would potentially fail to pick up
the correct version of Java.

This change fixes this issue to handle all cases.

Change-Id: If81100bbdd116ee071c3ecdeeadc0a3016fa2a15
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoCheck MAVEN_PARAMS for -f flag in metadata fetch 06/14106/3
Eric Ball [Wed, 9 Jan 2019 17:07:16 +0000 (09:07 -0800)]
Check MAVEN_PARAMS for -f flag in metadata fetch

maven-fetch-metadata.sh was not respecting the "-f" (for file path)
flag in MAVEN_PARAMS, causing lf-maven-merge jobs that utilize this
flag to fail.

Issue: RELENG-1630
Change-Id: I23803fd2e4a46022d85dc2befab1438853c7c3ca
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoMerge "Add documentation regarding clouds.yaml"
Thanh Ha (zxiiro) [Wed, 9 Jan 2019 16:59:29 +0000 (16:59 +0000)]
Merge "Add documentation regarding clouds.yaml"

5 years agoCheck openjdk $VERSION before setting $JAVA_HOME 05/14105/1
Anil Belur [Wed, 9 Jan 2019 02:14:44 +0000 (12:14 +1000)]
Check openjdk $VERSION before setting $JAVA_HOME

The default version of openjdk installed is openjdk-8 which
is set through the Jenkins job.

For CentOS we are installing jdk-11 and jdk-10 under /opt. So
we need to check the version of openjdk for the job to pick up
the right version of JDK.

Issue: RELENG-1454
Change-Id: I19d77c509c90bca5fea1af53dacc35772c7ac939
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Compress and upload Javadoc for publish jobs"
Thanh Ha (zxiiro) [Mon, 7 Jan 2019 22:25:00 +0000 (22:25 +0000)]
Merge "Compress and upload Javadoc for publish jobs"

5 years agoMerge "Add puppet-verify job to lf-ci-jobs"
Thanh Ha (zxiiro) [Mon, 7 Jan 2019 16:41:00 +0000 (16:41 +0000)]
Merge "Add puppet-verify job to lf-ci-jobs"

5 years agoAdd documentation regarding clouds.yaml 73/14073/2
Thanh Ha [Fri, 4 Jan 2019 20:05:08 +0000 (15:05 -0500)]
Add documentation regarding clouds.yaml

Change-Id: I7baf6a95a1552d178ba6fe48ec89448995d0f927
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Pass PYTHONPATH to tox env to fix install issues"
Thanh Ha (zxiiro) [Wed, 2 Jan 2019 13:30:16 +0000 (13:30 +0000)]
Merge "Pass PYTHONPATH to tox env to fix install issues"

5 years agoEnsure that no lingering .netrc is overriding cfg 20/14020/1
Thanh Ha [Thu, 20 Dec 2018 12:26:55 +0000 (07:26 -0500)]
Ensure that no lingering .netrc is overriding cfg

If a .netrc already exists with permissions to the same server
as logs, credentials in .netrc are read in order so the log
credentials will not be used. Ensure that no such file pre-exists
before we try to ship logs.

Change-Id: I0449702fd27837208e25a26e34a62f0b8a551c67
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoPass PYTHONPATH to tox env to fix install issues 09/14009/2
Eric Ball [Sat, 15 Dec 2018 01:20:49 +0000 (17:20 -0800)]
Pass PYTHONPATH to tox env to fix install issues

We have seen install issues that only seem to happen on build
machines. The reason these problems do not appear on local builds is
still not clear, but passing in the PYTHONPATH (set to the working
directory) is an innocuous change that fixes the problem.

Change-Id: I70217404b5a1f993d233714e2e1cac4ae9e7fd2d
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoRevert lftools back to v0.18.0 87/13987/1 v0.28.3
Thanh Ha [Fri, 14 Dec 2018 02:00:33 +0000 (10:00 +0800)]
Revert lftools back to v0.18.0

Binary corruption issues in lftools v0.19.x stream so revert back to
v0.18.0 for now.

Change-Id: I9f9ceb94a4d5efad4307517bff72b898a10ae728
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoPin dogpile.cache to ~=0.6.8 86/13986/1
Thanh Ha [Fri, 14 Dec 2018 01:41:47 +0000 (09:41 +0800)]
Pin dogpile.cache to ~=0.6.8

dogpile.cache released a new version 0.7.x yesterday which started
breaking openstackclient. No upstream fix for this yet so pin this
library until we can upgrade.

Change-Id: I308d170fe1498bc4e03b3d6b769cca3ca822f5ac
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoCompress and upload Javadoc for publish jobs 21/13921/3
Anil Belur [Tue, 11 Dec 2018 11:47:47 +0000 (21:47 +1000)]
Compress and upload Javadoc for publish jobs

Javadoc publish job always successfully publishes to the "javadoc"
repo, therefore dont maintain another copy of this on the logs
server. Also Javadocs contain ~15K small files which potentially
requires additional CPU cycles for pruning on the logs server.

Issue: RELENG-1215
Change-Id: I1b107a1c65db4b70096d5735c741fb32f09daedd
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd acumos hardware flavor 48/13948/1
Jessica Wagantall [Wed, 12 Dec 2018 00:34:10 +0000 (16:34 -0800)]
Add acumos hardware flavor

Add acumos-highcpu-4-avx hardware flavor to the
map list.

Change-Id: I5b320b0110d27bbb9ddf2f78244950609105046a
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoAdd puppet-verify job to lf-ci-jobs 35/13835/5
Eric Ball [Wed, 5 Dec 2018 16:01:38 +0000 (08:01 -0800)]
Add puppet-verify job to lf-ci-jobs

This job performs linting of Puppet modules.

Issue: RELENG-988
Change-Id: I8ab6e828e378eb9186187edb80487e0332ac144f
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoRefactor github-maven-sonar trigger 08/13908/3
Jeremy Phelps [Mon, 10 Dec 2018 18:25:31 +0000 (12:25 -0600)]
Refactor github-maven-sonar trigger

Configure the trigger to run on weekly cron, and also
allow a trigger via GHPRB comment only.

Issue-ID: RELENG-1508
Change-Id: I82e2ab19dc52bc29c8c6549ba753ad1e2e743994
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
5 years agoUse .xz format instead of .gz 52/13852/1
Anil Belur [Thu, 6 Dec 2018 04:32:23 +0000 (14:32 +1000)]
Use .xz format instead of .gz

xz format has faster compress ratio and better memory requirements.

Change-Id: Id26f9b292fe877f9523a43063daead32ca3486a0
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoCompress and upload the jjb-verify XML files 28/13828/3
Anil Belur [Wed, 5 Dec 2018 09:19:33 +0000 (19:19 +1000)]
Compress and upload the jjb-verify XML files

The present jjb-verify job that calls lftools maven
deploy archives for releng/builder which takes about ~8 mins to
complete uploading the archives. This is because the job generates
around ~2.3K XML files (small files) which is uploaded to Nexus.
This is possibly because the Nexus Unpack plugin in the Nexus
end unpacking the zip file we upload takes longer.

It's best to compress this into a single file and upload
to Nexus, potentially minimizing the Nexus logs from
bloating up since we have been seeing higher IOPs
and load managing these files in the backend cron jobs.

Compressing the job files before hand shaves off around
6-8 mins from the upload time, therefore reducing the
job time overall significantly.

Tested the change here:
https://jenkins.opendaylight.org/releng/job/builder-jjb-verify/4325

Change-Id: Ifb3ea6e61d28b234c79088ca128294a037ec511f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate lftools to v0.19.1 with unicode fix 26/13826/1 v0.28.2
Thanh Ha [Wed, 5 Dec 2018 03:49:11 +0000 (11:49 +0800)]
Update lftools to v0.19.1 with unicode fix

Pull in the unicode fix I88f70af17222fc7931a07e78eba23d9cc60eb9e7 in
lftools.

Change-Id: I52394f3d091a0aa54e23f5ffc654c1307bf04a89
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd a note on how to test lftools master 18/13618/4
Thanh Ha [Thu, 22 Nov 2018 04:49:05 +0000 (12:49 +0800)]
Add a note on how to test lftools master

Add a note for LF Staff for how to test lftools master branch
by tweaking global-jjb.

Change-Id: I1ecc0dc7fd251eaea6e0f53c8b9d67e9651cd839
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoBump lftools version to v0.19.0 72/13772/1 v0.28.1
Thanh Ha [Mon, 3 Dec 2018 06:48:14 +0000 (14:48 +0800)]
Bump lftools version to v0.19.0

Change-Id: Ib28022b61efe42bbad77418d4289ddf3b9afd75f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoEnable configuring the concurrency of jjb-verify 72/13672/6 v0.28.0
Trevor Bramwell [Tue, 27 Nov 2018 22:10:30 +0000 (14:10 -0800)]
Enable configuring the concurrency of jjb-verify

The JJB verify job always runs concurrently. By turning the concurrency
setting into a variable, users will be able to run the job serially,
which may be required on static nodes.

Issue: RELENG-1509
Change-Id: I4f2b51924a71674bbf2d4bbf8acc637971470eec
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
5 years agoConvert Maven Central parameter to macro 63/13663/6
Thanh Ha [Tue, 27 Nov 2018 04:45:08 +0000 (12:45 +0800)]
Convert Maven Central parameter to macro

This would make it easier for say ODL's autorelease project to
reuse this macro.

Change-Id: I73771b26041dd9e27ca470f3f9ebb54cbe67e654
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRemove deprecated lftools-install.sh from docs 55/13655/4
Anil Belur [Mon, 26 Nov 2018 07:09:47 +0000 (17:09 +1000)]
Remove deprecated lftools-install.sh from docs

lftools-install.sh has been deprecated since v0.25.0 update
the best practices docs with the latest changes.

The change is an extension to
I770eec99ae63cf92aa145124859f09d5d327c451 which missed
updating the docs.

Change-Id: I36f36d9848cd53e3e6fd9fbabcdb5b8c36aee268
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Unpin nodeenv version"
Anil Belur [Tue, 27 Nov 2018 09:40:38 +0000 (09:40 +0000)]
Merge "Unpin nodeenv version"

5 years agoRemove lf-stack-delete macro doc from lf-macros 37/13637/3
Thanh Ha [Fri, 23 Nov 2018 09:57:12 +0000 (17:57 +0800)]
Remove lf-stack-delete macro doc from lf-macros

This was moved to lf-openstack-heat.rst via change
Ie706706e24079b0b87f4537342fd3e8955f63fef but we forgot to remove
it from here.

Change-Id: I36a2a97d2c2b97220df0ffc96c2c2a9df79ab304
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoFix external link syntax 67/13667/1
Thanh Ha [Tue, 27 Nov 2018 09:00:33 +0000 (17:00 +0800)]
Fix external link syntax

We shouldn't be using :ref: for external links like this and is the
cause of the docs build failure we are seeing today.

Issue: RELENG-1518
Change-Id: I42bcaab4524b21e16e909d76ebdcea4476315350
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoUnpin nodeenv version 66/13666/1
Thanh Ha [Tue, 27 Nov 2018 08:41:32 +0000 (16:41 +0800)]
Unpin nodeenv version

This was originally pinned because nodeenv was breaking on 2nd and
onward runs of `tox -e coala` due to bug in nodeenv. This no longer
seems to be the case so unpin the version so we can get updates again.

Change-Id: I6a9906f3fcdc90eb546813026527ef63eea88c4e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRemove deprecated lftools-install.sh 36/13636/2
Thanh Ha [Fri, 23 Nov 2018 09:50:14 +0000 (17:50 +0800)]
Remove deprecated lftools-install.sh

lftools-install.sh has been deprecated since v0.25.0 it's about time
we remove it from global-jjb officially.

Change-Id: I770eec99ae63cf92aa145124859f09d5d327c451
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Hide irrelevant error message"
Thanh Ha (zxiiro) [Sat, 24 Nov 2018 01:00:18 +0000 (01:00 +0000)]
Merge "Hide irrelevant error message"

5 years agoHide irrelevant error message 98/13598/6
Tim Johnson [Tue, 20 Nov 2018 03:13:17 +0000 (19:13 -0800)]
Hide irrelevant error message

Use 'systemctl' to get status of 'sysstat' on newer releases of Unbuntu
(16.04 & 18.04).  Keep 'grep' to get status of 'sysstat' on Ubuntu
14.4. Unexpected releases will now generate an error.

Change-Id: I9745373d38213838381d3f2f0b46a6358014b268
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoPrint the final requirements file 19/13619/1
Thanh Ha [Thu, 22 Nov 2018 05:09:31 +0000 (13:09 +0800)]
Print the final requirements file

Make it clear what's being pulled in by pip install.

Change-Id: I1d659de74e1eb6e37a390285ed3b9854d19edbc9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Add details to the relnotes for submodule-timeouts"
Aric Gardner [Tue, 20 Nov 2018 13:50:18 +0000 (13:50 +0000)]
Merge "Add details to the relnotes for submodule-timeouts"

5 years agoMerge "Migrate stack create and delete scripts"
Anil Belur [Tue, 20 Nov 2018 03:09:21 +0000 (03:09 +0000)]
Merge "Migrate stack create and delete scripts"

5 years agoAdd details to the relnotes for submodule-timeouts 80/13580/2
Thanh Ha [Mon, 19 Nov 2018 09:18:07 +0000 (17:18 +0800)]
Add details to the relnotes for submodule-timeouts

When reading the release notes, there is no context to which patch
and files were changed. This makes it difficult for the reader to
know what "submodule-timeouts" refers to. Improve the release notes
to mention which macros and templates were changed.

Change-Id: Idc04dd19ea54f217cb04895d755dab71fe205351
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Allow packer-merge job to test WIP patch"
Thanh Ha (zxiiro) [Tue, 20 Nov 2018 01:30:52 +0000 (01:30 +0000)]
Merge "Allow packer-merge job to test WIP patch"

5 years agoMerge "Re-add All view tab after deleting all jobs&views"
Anil Belur [Mon, 19 Nov 2018 10:14:59 +0000 (10:14 +0000)]
Merge "Re-add All view tab after deleting all jobs&views"

5 years agoMerge "Fix literal code-block formatting"
Anil Belur [Mon, 19 Nov 2018 09:59:47 +0000 (09:59 +0000)]
Merge "Fix literal code-block formatting"

5 years agoAdd reference links for ci job-templates 79/13579/1
Thanh Ha [Mon, 19 Nov 2018 08:59:44 +0000 (16:59 +0800)]
Add reference links for ci job-templates

Add reference links for ci job-templates that are missing them.

Change-Id: I08aabb4cd1c759a4672fed5c2e60e9ec1f0a7e39
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoFix literal code-block formatting 77/13577/1
Thanh Ha [Mon, 19 Nov 2018 08:48:37 +0000 (16:48 +0800)]
Fix literal code-block formatting

Change-Id: I9b9ab53652dfcabb12190c4081e15df59ad65973
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRe-add All view tab after deleting all jobs&views 57/13457/2
Thanh Ha [Mon, 12 Nov 2018 09:20:23 +0000 (17:20 +0800)]
Re-add All view tab after deleting all jobs&views

This resolves the Jenkins Sandbox being left in a state where a
'00-Empty View' tab is left. This is confusing for Jenkins Sandbox
users as it will look as if the jobs they upload are not being added.

This patch requires JJB 2.8.0 otherwise the jenkins-sandbox-cleanup
job will report failed state. Despite this the primary purpose of the
job (deleting jobs and views) will be successful but needs the new
JJB version to replace the All view automatically.

Issue: RELENG-1450
Change-Id: I4ec106dc55789b7eab717e540491a94e47551852
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAllow packer-merge job to test WIP patch 75/13575/5
Thanh Ha [Mon, 19 Nov 2018 04:23:42 +0000 (12:23 +0800)]
Allow packer-merge job to test WIP patch

Adds the ability for the packer-merge job to be used on Jenkins
Sandbox systems to test work in progress patches from Gerrit or
GitHub.

Issue: RELENG-1495
Change-Id: I5480f17b225f462cc7802981aac9ece3ce611214
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMigrate stack create and delete scripts 13/13513/11
Thanh Ha [Wed, 14 Nov 2018 07:58:34 +0000 (15:58 +0800)]
Migrate stack create and delete scripts

Pull in OpenDaylight's opendaylight-infra-stack and
releng-openstack-stack-delete macros. This patch generalizes the
OS HEAT workflow utilized by ODL to spin up CSIT labs to make it
reusable by non-OpenDaylight projects.

Issue: RELENG-1437
Change-Id: Ie706706e24079b0b87f4537342fd3e8955f63fef
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Ignore false-neg ShellCheck SC2154 unused vars" v0.27.0
Anil Belur [Thu, 15 Nov 2018 03:46:07 +0000 (03:46 +0000)]
Merge "Ignore false-neg ShellCheck SC2154 unused vars"

5 years agoMerge "Add error handling to stack script"
Thanh Ha (zxiiro) [Thu, 15 Nov 2018 02:38:58 +0000 (02:38 +0000)]
Merge "Add error handling to stack script"

5 years agoMerge "Allow JJB Cache directory to be configured"
Thanh Ha (zxiiro) [Thu, 15 Nov 2018 02:38:46 +0000 (02:38 +0000)]
Merge "Allow JJB Cache directory to be configured"

5 years agoAdd error handling to stack script 44/13544/1
Thanh Ha [Thu, 15 Nov 2018 01:26:54 +0000 (09:26 +0800)]
Add error handling to stack script

Handle case where JENKINS_URL does not return 200 ok.

Change-Id: Ie2067c3a5f3f489e25c361a4f417a6325e4abbd1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMigrate cleanup orphaned nodes cron script 84/13484/3
Thanh Ha [Tue, 13 Nov 2018 06:03:09 +0000 (14:03 +0800)]
Migrate cleanup orphaned nodes cron script

Generalize script for reuse in global-jjb.

Issue: RELENG-1434
Change-Id: I7901e58dbc0136c339a2d488c9eef8d46cd5446a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAllow JJB Cache directory to be configured 43/13343/2
Trevor Bramwell [Tue, 6 Nov 2018 00:30:38 +0000 (16:30 -0800)]
Allow JJB Cache directory to be configured

When JJB is ran concurrently on the same system with caching enabled, a
timeout can occur when one job fails after waiting 1 minute to aquire
the lock on the JJB cache directory.

Even if caching is disabled/ignored this timeout can still occur as JJB
always tries to lock the cache directory.

Having the ability to configure the cache directory will allow projects
to override it dynamically (with Jenkins env-vars) and run concurrent
jobs on the same node.

Change-Id: I1187847f658e754621369005e627a0f33a0253bc
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
5 years agoIgnore false-neg ShellCheck SC2154 unused vars 19/13519/1
Daniel Farrell [Wed, 14 Nov 2018 15:01:41 +0000 (15:01 +0000)]
Ignore false-neg ShellCheck SC2154 unused vars

ShellCheck 0.4.4 incorrectly fails on the ignored lines, thinking they
are vars that are declared but not used. It only fails one the first
line that uses a new prefix before the hyphen.

I tried using single/double/no quotes, all of which should be valid
bash, but ShellCheck fails on all.

Change-Id: I9ccca21a9d2504ecf0ec3f09707eb82353bd8bbc
Signed-off-by: Daniel Farrell <dfarrell@redhat.com>
5 years agoMigrate cleanup-stale-stacks cron script 80/13480/6
Thanh Ha [Tue, 13 Nov 2018 03:39:45 +0000 (11:39 +0800)]
Migrate cleanup-stale-stacks cron script

Generalize script for use in global-jjb.

Issue: RELENG-1432
Change-Id: I9c450c36b0f44a5583bee1932016e7cb3d5ef834
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd orphaned-volumes feature to release notes 79/13479/1
Thanh Ha [Tue, 13 Nov 2018 03:39:08 +0000 (11:39 +0800)]
Add orphaned-volumes feature to release notes

This was missed when If84c0f4341386ed758ce9d8556d4c4e574601797
was reviewed.

Change-Id: I529cc36ef19102207594adfe28531a7a1472092a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMigrate cleanup-stale-volumes cron script 56/13456/1
Thanh Ha [Mon, 12 Nov 2018 07:48:45 +0000 (15:48 +0800)]
Migrate cleanup-stale-volumes cron script

Generalize the cleanup script for general use in global-jjb.

Issue: RELENG-1435
Change-Id: If84c0f4341386ed758ce9d8556d4c4e574601797
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Allow image protection script to be disabled"
Thanh Ha (zxiiro) [Fri, 9 Nov 2018 23:49:46 +0000 (23:49 +0000)]
Merge "Allow image protection script to be disabled"

5 years agoMerge "Migrate cleanup old images script from ODL"
Thanh Ha (zxiiro) [Fri, 9 Nov 2018 17:26:35 +0000 (17:26 +0000)]
Merge "Migrate cleanup old images script from ODL"

5 years agoMerge "Refactor the common-view and csit-view template"
Anil Belur [Thu, 8 Nov 2018 03:50:36 +0000 (03:50 +0000)]
Merge "Refactor the common-view and csit-view template"

5 years agoAllow image protection script to be disabled 09/13409/3
Thanh Ha [Wed, 7 Nov 2018 20:22:55 +0000 (04:22 +0800)]
Allow image protection script to be disabled

Change-Id: Ie15fc59fba1a4efc475136bf30ed4c94045203d2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMigrate cleanup old images script from ODL 03/13403/6
Thanh Ha [Wed, 7 Nov 2018 20:22:55 +0000 (04:22 +0800)]
Migrate cleanup old images script from ODL

Generalize the cleanup script for reuse in global-jjb.

Issue: RELENG-1436
Change-Id: I88e1bc3136b2f9b2542e441a74f26d742679c353
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoInstall 'openstack' module for lftools 97/13397/2
Thanh Ha [Wed, 7 Nov 2018 19:28:39 +0000 (03:28 +0800)]
Install 'openstack' module for lftools

The lftools optional 'openstack' module will now be installed as part
of pre-build script.

Change-Id: I05bdcf07258ffcff9242c9ced10256d1cdaeef6d
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRm openstack-install for lf-infra-pre-build 96/13396/1
Thanh Ha [Wed, 7 Nov 2018 19:15:53 +0000 (03:15 +0800)]
Rm openstack-install for lf-infra-pre-build

lf-infra-pre-build is able to manage openstack client installs
so remove the openstack-install script. No need to release note since
no project should be using openstack-cron yet as in it's current
state is incomplete.

Change-Id: Ic6ee5ae0e5815d2730b3fa6154fd8e6d5b096844
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRefactor the common-view and csit-view template 68/13268/5
Thanh Ha [Wed, 31 Oct 2018 23:21:18 +0000 (19:21 -0400)]
Refactor the common-view and csit-view template

Make the 2 views easier to use by projects. common-view is now
project-view, the intent of the project-view is that it
automatically matches jobs starting with {project-name} to
automatically create a view tab for all of a single project's jobs.

The intent of the CSIT view template is to have a common template
that loads the most useful columns for CSIT views. Since we do
not know the search pattern and view name of these views ahead of
time it has 2 required paramaters ``view-name`` and ``view-regex``.

There is no need to force configuration in defaults.yaml for these 2
templates. For project-view it's intent is to load all project jobs
into a single view using a well known regex which does not need to
be modified. For csit-view we don't know how the regex is going to
be ahead of time so this view-template should just load the common
column configuration and require the view-template user to explicitly
declare their regex and view name.

Additionally add a new 'common-view' which accepts a view-name and
view-regex for users who want to pull in a default view-template
for their view.

Move documentation from release-notes to documentation. Release notes
should be about informing users about new features where detailed
documentation should instead to to the documentation of the views.

Change-Id: Ife3cfe2e37f019dfb87ac8e90b21afb6bdf06889
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoVerify jenkins-config files have matching images 83/13283/8
Eric Ball [Thu, 25 Oct 2018 21:54:32 +0000 (14:54 -0700)]
Verify jenkins-config files have matching images

This verify job will check that for each config file in
jenkins-config/clouds/**, there is an image on the cloud provider
that matches the IMAGE_NAME field.

Issue-ID: RELENG-1133
Change-Id: I9bf79657af27e8c7ff7b63ced2bd8a0dd544e6d6
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoPull in tox implicitly through detox 18/13318/1
Thanh Ha [Mon, 5 Nov 2018 07:23:44 +0000 (15:23 +0800)]
Pull in tox implicitly through detox

detox's own requirements.txt limits the version of tox that it pulls
in. This can often conflict with the explicit version we set.
Let's use the version of tox as required by detox.

Issue: https://jira.opendaylight.org/browse/RELENG-136
Change-Id: I0b98afc6c993279c17cdd02d67471fadde23e552
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMove required params to defaults section in views 96/13196/5
Anil Belur [Fri, 26 Oct 2018 05:08:43 +0000 (15:08 +1000)]
Move required params to defaults section in views

JIRA: RELENG-1393
Change-Id: I03b7244650f30daebba246f5af7af350309079ce
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd option to stage to Maven Central 18/12318/9
Thanh Ha [Sat, 18 Aug 2018 16:26:15 +0000 (12:26 -0400)]
Add option to stage to Maven Central

Provide an option to also stage to OSSRH in preparation to release
to Maven Central.

In addition to printing the staging repo-id also print the URL to
the staging repo for convenience in stage-repo.txt.

Issue: RELENG-1398
Issue: https://jira.opendaylight.org/browse/RELENG-28
Change-Id: I0628cb5c7185e2143e448ceb4fa4752384581a5e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoFix github-maven-merge scm config 27/13227/1
Jeremy Phelps [Mon, 29 Oct 2018 19:39:35 +0000 (14:39 -0500)]
Fix github-maven-merge scm config

Specifying a refspec on this scm config causes no merge job to be
triggered on a push action.

Change-Id: I46066dc6e371f49ed58dcc4f439cb0fae63808db
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
5 years agoMerge "Add view templates to manage Jenkins views"
Andrew Grimberg [Thu, 25 Oct 2018 15:20:32 +0000 (15:20 +0000)]
Merge "Add view templates to manage Jenkins views"

5 years agoDocument Gerrit Branch Lock job 90/13190/1
Thanh Ha [Thu, 25 Oct 2018 14:57:25 +0000 (10:57 -0400)]
Document Gerrit Branch Lock job

Change-Id: I2fddf938cc45bd852bf4e4f01a6e3836a4b79338
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd view templates to manage Jenkins views 89/12989/11
Anil Belur [Tue, 16 Oct 2018 01:26:40 +0000 (11:26 +1000)]
Add view templates to manage Jenkins views

Add view templates for common views and csit jobs.
Update docs and JJB tests.

JIRA: RELENG-1259
Change-Id: I0fda1c089ef516bfd3bb3fb04ea4660d410fb094
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Add submodule-timeout for submodule operations"
Thanh Ha (zxiiro) [Fri, 19 Oct 2018 19:10:19 +0000 (19:10 +0000)]
Merge "Add submodule-timeout for submodule operations"

5 years agoAdd submodule-timeout for submodule operations 79/12879/9
Jessica Wagantall [Thu, 4 Oct 2018 00:48:05 +0000 (17:48 -0700)]
Add submodule-timeout for submodule operations

Some ONAP projects using CLM (SDC) require more submodule
timeout to let the Nexus report run successfully. Allow an
option to provide a submodule-timeout to gerrit-maven-clm
keeping the default to 10 minutes.

Add submodule-timeout parameter for both lf-infra-gerrit-scm
and lf-infra-github-scm.

Main issue and tests described in the asociated Jira.

Issue: RELENG-1297
Change-Id: Ia7d292df9bac53d760767d9ee2caeec2f84c48ac
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoMerge "Bump lftools version to v0.18.0"
Andrew Grimberg [Fri, 19 Oct 2018 17:05:23 +0000 (17:05 +0000)]
Merge "Bump lftools version to v0.18.0"

5 years agoBump lftools version to v0.18.0 88/13088/1
Thanh Ha [Fri, 19 Oct 2018 16:28:16 +0000 (12:28 -0400)]
Bump lftools version to v0.18.0

Change-Id: I999f2a79044dadfb543fe36d8a506d115d3cdf99
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoFix jenkins-init scripts dir 29/12729/2
Thanh Ha [Fri, 21 Sep 2018 23:41:20 +0000 (19:41 -0400)]
Fix jenkins-init scripts dir

Resolve issue caused by I5060dd0c1b67fa020a84844936eebb970ae2cbb6
which moved the global-jjb directory around and caused scripts
that depended on the old path to start failing.

Change-Id: I705a8c79bd69c9fdaff8dc677e0f22cb6b42d74b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMake a working publisher for Windows builds 78/13078/2 v0.26.0
Thanh Ha [Thu, 18 Oct 2018 20:22:40 +0000 (16:22 -0400)]
Make a working publisher for Windows builds

This publisher is missing sysstat / shiplogs but we will have to
develop them for Windows machines. In the meantime at least deploy
a builder that works.

Issue: RELENG-1228
Change-Id: I427779c8e580abf6126c04032ca527a5ffd627bf
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd conditional build step to use clouds.yaml 73/13073/3
Jeremy Phelps [Thu, 18 Oct 2018 19:15:30 +0000 (14:15 -0500)]
Add conditional build step to use clouds.yaml

This was missed in patch Idf23a30b111dd922120fb6195d0411e2fc993dab
when we were implementing this feature.

Issue: RELENG-1205
Change-Id: Id2017172293463c616158b79f3af826ea754fa08
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
5 years agoUpdate reno to 2.11.2 66/13066/2
Thanh Ha [Thu, 18 Oct 2018 18:18:23 +0000 (14:18 -0400)]
Update reno to 2.11.2

Issue: RELENG-1355
Change-Id: I1bbd3be83f52d63bcde956f392c3396fa236b329
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>