releng/global-jjb.git
5 years agoMerge "Add lf-infra-wss-mvn-clean-install step"
Jessica Wagantall [Thu, 11 Apr 2019 21:35:22 +0000 (21:35 +0000)]
Merge "Add lf-infra-wss-mvn-clean-install step"

5 years agoUpdate jenkins-cfg-verify to use $GERRIT_REFSPEC 63/15263/4
Anil Belur [Thu, 11 Apr 2019 12:29:45 +0000 (22:29 +1000)]
Update jenkins-cfg-verify to use $GERRIT_REFSPEC

Update jenkins-cfg to verify images names updated from the
$GERRIT_REFSPEC instead of the master branch.

Current implementation uses the 'master' branch instead of
the $GERRIT_REFSPEC, therefore failing the verify job
on false positives.

Issue: RELENG-1950
Change-Id: I6f212b28e4189ff5c7958035c7992b55d7153d02
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd lf-infra-wss-mvn-clean-install step 61/15261/3
Jessica Wagantall [Thu, 11 Apr 2019 05:14:13 +0000 (22:14 -0700)]
Add lf-infra-wss-mvn-clean-install step

Add Maven clean install conditional step
that runs before WhiteSource Unified Agent scan.
This allows to fetch additional dependencies.

Issue: RELENG-1951
Change-Id: Id3e070220e17ba995d976b3d033a16983b38fd8d
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoHardcode WSS project version 59/15259/4
Jessica Wagantall [Thu, 11 Apr 2019 00:16:16 +0000 (17:16 -0700)]
Hardcode WSS project version

Similarly to how is done in CLM, the WhiteSource
versions for projects should reflect the branch they are
running on.

Also fix script variables to be enclosed by "{}".

Change-Id: Ica0a52adcc9223e861f213bc53c67ed2102efda5
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoUse WS Agent config file internally 58/15258/3
Jessica Wagantall [Wed, 10 Apr 2019 23:37:51 +0000 (16:37 -0700)]
Use WS Agent config file internally

After checking with legal, it was agreed to not
allow tech teams to provide their own
wss-unified-agent.config file for WhiteSource scans.
Instead, this file should be managed in Jenkins Settings
Files

Change-Id: I20e330794ba28cb4c0f5206c290a734c1794b00e
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoMerge "Fixes for info votes"
Aric Gardner [Tue, 9 Apr 2019 17:31:24 +0000 (17:31 +0000)]
Merge "Fixes for info votes"

5 years agoMerge "Add missing release notes."
Aric Gardner [Tue, 9 Apr 2019 17:29:36 +0000 (17:29 +0000)]
Merge "Add missing release notes."

5 years agoFixes for info votes 01/15201/3
Aric Gardner [Thu, 4 Apr 2019 19:56:01 +0000 (15:56 -0400)]
Fixes for info votes

gerrit trigger is not working in opnfv.
we had this happen to some of the read the docs jobs...
An updgrade of gerrit is planned to hopefully resolve this.

Not sure were to test to approval category stuff.
That said I think the mistake is that the value is an int.

            approval-category: 'WHATEVER'
            approval-value: 2

I dont actually want the script to exit on first error, need to output
the vote verdict.

not sure why I thought that JENKINS_SSH_CREDENTIAL
was the correct var for the ssh user.
switched to $USER

Change-Id: I6714361423948eb1764e9c18da699dafd4cd24a9
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
5 years agoAdd missing release notes. 28/15228/2
Aric Gardner [Mon, 8 Apr 2019 17:36:03 +0000 (13:36 -0400)]
Add missing release notes.

Commit b2e2b5b7d3bb61ee9886debb4e010769c35be398
was missing release notes.

Change-Id: If6adc956f92113e80177930b82ba20872b4e009f
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
5 years agoUpdate rtd-merge to also trigger on tag changes 27/15227/1
Thanh Ha [Mon, 8 Apr 2019 03:41:07 +0000 (11:41 +0800)]
Update rtd-merge to also trigger on tag changes

When we release a tag, it should also trigger a rtd-merge build
so that the doc release note link is updated appropriately.

Change-Id: I49f6555b707018cca4042917c4193138456dafeb
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoARM64 support for update-java-alternatives 11/15211/1
Martin Klozik [Fri, 5 Apr 2019 14:05:07 +0000 (16:05 +0200)]
ARM64 support for update-java-alternatives

Java paths at Ubuntu contain host CPU architecture signature.
Script was updated to correctly detect architecture by dpdk.

Change-Id: I77c4aba1d600373edff283b6b85837fccda3af34
Signed-off-by: Martin Klozik <martin.klozik@tieto.com>
5 years agoFix bug in package-listing start script 07/15207/1
Andrew Grimberg [Thu, 4 Apr 2019 22:39:32 +0000 (15:39 -0700)]
Fix bug in package-listing start script

Change af89ef784234e7b89984d5b2995fc6d29b6d8c34 introduced a bug where
we don't make sure that the package-listing.sh script is executable
before calling it during system startup. This means that we are no
longer currently producing the needed listing start file for instances
to be able to diff against to give us a view of package changes.

Change-Id: Iea2f0b07cdf10f7819072bcb00596bdeda4665df
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoMerge "Fix global-jjb docs for installation"
Jessica Wagantall [Thu, 4 Apr 2019 21:15:38 +0000 (21:15 +0000)]
Merge "Fix global-jjb docs for installation"

5 years agoFix global-jjb docs for installation 00/15200/4
Andrew Grimberg [Thu, 4 Apr 2019 19:32:10 +0000 (12:32 -0700)]
Fix global-jjb docs for installation

The docs copy / paste script use the GitHub submodule but don't properly
name the resultant module to match the expectations of the rest of the
script.

Fix up some minor inconsistencies / errors of other parts of script

Change-Id: I66402d9ef1be80c1dd53c51d55802a864c813421
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoMerge "Add WhiteSource Scan jobs"
Jessica Wagantall [Thu, 4 Apr 2019 19:59:03 +0000 (19:59 +0000)]
Merge "Add WhiteSource Scan jobs"

5 years agoAdd WhiteSource Scan jobs 97/14997/10
Jessica Wagantall [Fri, 22 Mar 2019 00:56:15 +0000 (17:56 -0700)]
Add WhiteSource Scan jobs

WhiteSource is an open source security and license compliance
management platform.
It is used to scan a great variety of languages includding Java,
Python, C, Go, among others.

{project-name}-whitesource-scan-{stream} uses WhiteSource's
Unified Agent to perform the code scans.

Issue: RELENG-1876
Change-Id: If511da2f51e08799352ee6841338f76dd63af6b6
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoUpdate lftools to version v0.22.2 83/15183/1 v0.35.0
Aric Gardner [Wed, 3 Apr 2019 17:47:34 +0000 (13:47 -0400)]
Update lftools to version v0.22.2

Change-Id: Ic1e44a73813dd42c9b376547cc4814f1d63b6ab4
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
5 years agoMerge "INFO.yaml check vote for global-jjb"
Andrew Grimberg [Wed, 3 Apr 2019 16:26:30 +0000 (16:26 +0000)]
Merge "INFO.yaml check vote for global-jjb"

5 years agoAdd comment-to-gerrit macro 73/15173/2
Aric Gardner [Tue, 2 Apr 2019 18:28:57 +0000 (14:28 -0400)]
Add comment-to-gerrit macro

This macro will post a comment to the gerrit patchset if the build
creates a file named gerrit_comment.txt
To use this macro add it to the list of builders

Change-Id: I30e2b10fc4e09f534b161dd99775d66bdc4c3e1a
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
5 years agoINFO.yaml check vote for global-jjb 35/15035/12
Aric Gardner [Mon, 25 Mar 2019 20:35:34 +0000 (16:35 -0400)]
INFO.yaml check vote for global-jjb

Global job per gerrit, that triggers on changes
to INFO.yaml file

And auto merged if a majority of committers
have voted.

This will create a review in the internal gerrit

If a tsc vote is required, RE must manually confirm
that this has happened.

We will need to give
Non-Interactive Users merge
or perhaps specifically only the jenkins
user.

Job will only trigger when a user votes +2
cludgy but it works.

Example of Job configuration:

- project:
    name: info-vote
    build-node: centos7-builder-2c-1g
    jobs:
      - 'info-vote-verify'

Change-Id: Ib386bd2ea51be113ca59610bd3281b9ad6923bce
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
5 years agoRevert Add tox-verify job to the default ci group 63/15163/2
Bengt Thuree [Tue, 2 Apr 2019 00:45:41 +0000 (17:45 -0700)]
Revert Add tox-verify job to the default ci group

This reverts commit 059f6382bcbe1549201773eb4d881d43724ca8a9.

Merged it to quickly, need to ensure all projects has tox.ini
and .coafile linting in place first.

Change-Id: I083fcb526d1858d15decb58d4138f9dcc9cc600a
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
5 years agoMerge "Add tox-verify job to the default ci group"
Bengt Thuree [Tue, 2 Apr 2019 00:40:16 +0000 (00:40 +0000)]
Merge "Add tox-verify job to the default ci group"

5 years agoAllow jjb-workers to be configurable 88/15088/1
Thanh Ha [Wed, 27 Mar 2019 17:27:19 +0000 (13:27 -0400)]
Allow jjb-workers to be configurable

Default it to the number of CPU cores available on the system.

Issue: RELENG-917
Change-Id: I5f65ec7d7142f1a778b6e31cc6700ca2bca1ef88
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoSkip creating source jars in verify job 87/15087/1
Thanh Ha [Wed, 27 Mar 2019 17:16:03 +0000 (13:16 -0400)]
Skip creating source jars in verify job

Issue: RELENG-1655
Change-Id: Ibd2aad63afda3feb97619cef67c10c8b665cd924
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd tox-verify job to the default ci group 81/12681/2
Thanh Ha [Thu, 20 Sep 2018 15:16:01 +0000 (11:16 -0400)]
Add tox-verify job to the default ci group

This ensures our projects are running linters against the prjoect.

Change-Id: I5ce38eeecef8f1241a49dfe7da3eb04fec7ac385
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoUpdate lftools to version v0.22.1 41/15041/1 v0.34.1
Anil Belur [Tue, 26 Mar 2019 02:05:30 +0000 (12:05 +1000)]
Update lftools to version v0.22.1

Change-Id: I728e3581f956cd56cc0446f2a3360bd9e71df6af
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate lftools to version 0.22.0 96/14996/1 v0.34.0
Anil Belur [Thu, 21 Mar 2019 23:28:14 +0000 (09:28 +1000)]
Update lftools to version 0.22.0

Change-Id: I545662ee1cf31e1609971c08bb75555acf384139
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Fix JAVA_HOME for CentOS7"
Anil Belur [Thu, 21 Mar 2019 21:41:00 +0000 (21:41 +0000)]
Merge "Fix JAVA_HOME for CentOS7"

5 years agoFix JAVA_HOME for CentOS7 85/14985/4
Vanessa Rene Valderrama [Wed, 20 Mar 2019 20:15:27 +0000 (15:15 -0500)]
Fix JAVA_HOME for CentOS7

JAVA_HOME for JDK-11 on CentOS7 is set to /opt/jdk-11 but JDK-11 is
installing in /usr/lib/jvm/java-11-openjdk/.

This change is to modify JAVA_HOME to use the correct path and install
JDK-11 properly.

Change-Id: I598a063988a5afea5d6827d0940fdcbd2b83e99a
Signed-off-by: Vanessa Rene Valderrama <vvalderrama@linuxfoundation.org>
5 years agoAdd Throttle Property to JJB jobs 81/14781/4
Trevor Bramwell [Fri, 9 Nov 2018 05:43:54 +0000 (21:43 -0800)]
Add Throttle Property to JJB jobs

This adds "Throttle Builds" configuration to JJB jobs (verify, merge,
etc) to allow global-jjb users with static builders to limit the number
of jjb jobs ran on a single node.

Issue: RELENG-1510
Change-Id: I00d2950451eae8037ded407937131723b0001179
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
5 years agoAdapt maven path search for files and dirs 68/14968/3
Eric Ball [Wed, 20 Mar 2019 00:51:34 +0000 (17:51 -0700)]
Adapt maven path search for files and dirs

The "-f" maven param can specify both a directory, in which case it
will look for "pom.xml" in the directory, or a specific file. The
original version of this search was only compatible with directories
that contain a pom.xml file.

Change-Id: I81ab161f3156fc2159a4b7ca9ca084a0f82da795
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoMerge "Extract version number passed set '{java-version}'"
Anil Belur [Wed, 20 Mar 2019 21:50:01 +0000 (21:50 +0000)]
Merge "Extract version number passed set '{java-version}'"

5 years agoAdd set version conditional step 01/14901/12
Jessica Wagantall [Wed, 13 Mar 2019 01:08:21 +0000 (18:08 -0700)]
Add set version conditional step

Add conditional step for gerrit-maven-stage to optionally
run the versions maven plugin using the versions.properties
file.

Issue: RELENG-1860
Change-Id: I4bea0ae911a79db7ef238f5ece619be6270e2256
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoCleanup Jenkins init scripts 56/14756/19
Tim Johnson [Mon, 4 Mar 2019 18:33:59 +0000 (10:33 -0800)]
Cleanup Jenkins init scripts

RELENG-1856 Cleanup Jenkins init scripts Make scripts portable so the
ci-management can be installed anywhere.  Update README comments. All
changes should be backward compatible.

Change-Id: I5ce6f76cfa0483b7a1b91b72ae1264c21e769dba
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoExtract version number passed set '{java-version}' 18/14918/5
Anil Belur [Fri, 15 Mar 2019 02:05:10 +0000 (12:05 +1000)]
Extract version number passed set '{java-version}'

When the version number string is multi-digits ex: openjdk11
"${SET_JDK_VERSION: -1}" returns '1' which is incorrect.

The change fixes version number extraction for the string
passed to env variable $SET_JDK_VERSION.

Also update the regex to handle values with special chars.

Issue: RELENG-1853
Change-Id: I52c49fec007bad70d4a42d4c20a45b35fec3997c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Document required params in lf-maven-stage macro"
Bengt Thuree [Thu, 14 Mar 2019 02:22:23 +0000 (02:22 +0000)]
Merge "Document required params in lf-maven-stage macro"

5 years agoMerge "Optimize the jjb-deploy trigger regex"
Bengt Thuree [Thu, 14 Mar 2019 02:12:38 +0000 (02:12 +0000)]
Merge "Optimize the jjb-deploy trigger regex"

5 years agoUpdate lf-maven-central macro docs and templates 81/14881/2
Anil Belur [Wed, 13 Mar 2019 22:32:38 +0000 (08:32 +1000)]
Update lf-maven-central macro docs and templates

Update lf-maven-central macro docs and templates
with the missing required params.

The change 6a3a8fd3c9430a90f3b8c0c85e7e20b8a318eb8f
missed updating the docs and sample code.

Change-Id: I09b55bbf51cc3ef7c29d50b42e52982dc40e8f72
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoFix formatting in release notes files 03/14903/2
Anil Belur [Wed, 13 Mar 2019 01:41:51 +0000 (11:41 +1000)]
Fix formatting in release notes files

The change I04af514bbb237f405eb1d31ec10e3f019c85a43e breaks the rtd
verify job[1] because of the incorrect formating of the prelude section.

The prelude section can not contain a list as all prelude section are
concatenated together when the release notes are built. Merging the
preludes into their respective feature sections ensures the contex of
the comments are preserved as otherwise they will appear at the top of
the documentation apart from any other information.

Ref: https://docs.openstack.org/reno/latest/user/examples.html

[1] https://jenkins.opendaylight.org/releng/job/lf-infra-global-jjb-rtd-verify-any/475/

Change-Id: I3d23679eb4740b5fdc22fc566811d06ac6128820
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
5 years agoMerge "Update sigul macro to support sign mode" v0.33.0
Thanh Ha (zxiiro) [Mon, 11 Mar 2019 22:25:21 +0000 (22:25 +0000)]
Merge "Update sigul macro to support sign mode"

5 years agoMerge "Update lftools to version 0.21.0"
Anil Belur [Sat, 9 Mar 2019 23:26:35 +0000 (23:26 +0000)]
Merge "Update lftools to version 0.21.0"

5 years agoUpdate Jenkins Init Scripts 78/14778/5
Tim Johnson [Tue, 5 Mar 2019 22:46:52 +0000 (14:46 -0800)]
Update Jenkins Init Scripts

Support ability to specify swap size and to pre-allocate the '/w' volume
for the Hyperledger Project

Change-Id: I04af514bbb237f405eb1d31ec10e3f019c85a43e
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoUpdate sigul macro to support sign mode 12/14812/8
Thanh Ha [Wed, 6 Mar 2019 17:36:29 +0000 (12:36 -0500)]
Update sigul macro to support sign mode

Sign mode is a parameter to allow the user to choose whether or not
to sign artifacts in parallel using gnu-parallel or in serial mode
(bash for loop) methods.

Cleanup obsolete lftools_activate method.

Change-Id: Ib202f93a37e2830e5048c5a766c17d75233c6423
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoUpdate lftools to version 0.21.0 21/14821/1
Thanh Ha [Thu, 7 Mar 2019 18:28:55 +0000 (13:28 -0500)]
Update lftools to version 0.21.0

Change-Id: Ibbcd6b25247dcad926fc67342f9eac47347902e0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoOptimize the jjb-deploy trigger regex 79/14779/3
Anil Belur [Thu, 7 Mar 2019 18:04:32 +0000 (04:04 +1000)]
Optimize the jjb-deploy trigger regex

Optimize the jjb-deploy trigger regex to trigger the job _only_
when the Gerrit CR comment is intended as a command.

The `jjb-deploy` job presently gets triggered and is waste of infra
resources when the keyword is intended as comment in Gerrit.

Ref: https://git.opendaylight.org/gerrit/#/c/80358/

Issue: RELENG-1847
Change-Id: I3b35adb1284d3d9029ba025bf660b66fbb5d98e0
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoDocument required params in lf-maven-stage macro 13/14813/1
Thanh Ha [Thu, 7 Mar 2019 03:50:08 +0000 (22:50 -0500)]
Document required params in lf-maven-stage macro

Change-Id: I1f281b22f7b9454ef58e3fcc5fa49c5d333263a7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Add cmake-sonar job"
Thanh Ha (zxiiro) [Wed, 6 Mar 2019 15:44:09 +0000 (15:44 +0000)]
Merge "Add cmake-sonar job"

5 years agoMinor cleanup of openstack-protect script 67/14767/3
Tim Johnson [Tue, 5 Mar 2019 17:13:47 +0000 (09:13 -0800)]
Minor cleanup of openstack-protect script

Change-Id: I45070e0b364136b28e4130aed3972603e8c2e3c1
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoRework image code to handle no results from YAML 63/14763/4
Thanh Ha [Tue, 5 Mar 2019 01:47:38 +0000 (20:47 -0500)]
Rework image code to handle no results from YAML

When grep has 0 matches it returns a non-zero exit code. In this case
a 0 match grep is a possibility for YAML files so ignore grep's exit
code in this instance.

Change-Id: Ia7eb55364feeb42bf35681c4ae0798fe25bbfad1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoSearch for CFG & YAML files in specific dirs 62/14762/1
Thanh Ha [Tue, 5 Mar 2019 00:55:02 +0000 (19:55 -0500)]
Search for CFG & YAML files in specific dirs

These file locations are standard locations in our CI setup so be
specific of where to search for them. This prevents the scripts
from picking up data from test files in global-jjb.

Change-Id: Ia486ae8704b77f9303b5a2e54c4ffaf5de41aef8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoBump lftools to v0.20.0 58/14758/1 v0.32.0
Thanh Ha [Mon, 4 Mar 2019 20:34:29 +0000 (15:34 -0500)]
Bump lftools to v0.20.0

Change-Id: I6fa72595c1843a1955f4b492c035ebc7863163b5
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd niet needed for artifcat release job 95/14695/2
Aric Gardner [Tue, 26 Feb 2019 14:36:58 +0000 (09:36 -0500)]
Add niet needed for artifcat release job

Change-Id: I4223d00667b5701377d057e76e82e8657960d262
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
5 years agoAdd Error Handling 02/14602/1 v0.31.0
Tim Johnson [Fri, 15 Feb 2019 18:21:52 +0000 (10:21 -0800)]
Add Error Handling

Packer Cleanup Script was ignoring all errors

Change-Id: I881abf36420a1976c4f3af552aeeca4a4512c555
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoMerge "Update lf-maven-stage macro with missing parts"
Thanh Ha (zxiiro) [Wed, 13 Feb 2019 17:52:56 +0000 (17:52 +0000)]
Merge "Update lf-maven-stage macro with missing parts"

5 years agoMerge "Add job to automatically announce lf releases"
Thanh Ha (zxiiro) [Tue, 12 Feb 2019 15:42:39 +0000 (15:42 +0000)]
Merge "Add job to automatically announce lf releases"

5 years agoAdd job to automatically announce lf releases 69/13269/6
Thanh Ha [Thu, 1 Nov 2018 03:39:39 +0000 (23:39 -0400)]
Add job to automatically announce lf releases

This job triggers when a Gerrit tag is pushed and sends an email to
lf-releng@lists.linuxfoundation.org to notifiy the release.

Change-Id: I5255ac6f3afbcadfc11e9640483613bd2e84250a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoUpdate lf-maven-stage macro with missing parts 93/14393/9
Bengt Thuree [Thu, 31 Jan 2019 00:14:03 +0000 (11:14 +1100)]
Update lf-maven-stage macro with missing parts

Added the following missing parts to the lf-maven-stage macro.

* lf-provide-maven-settings
* lf-infra-create-netrc
* lf-provide-maven-settings-cleanup

This improves the macro to be a complete function that does not require
preparatory steps to complete it.

New Usage:

      - lf-maven-stage:
          mvn-global-settings: 'global-settings'
          mvn-settings: 'settings'
          mvn-staging-id: 'staging-profile-id'

This patch also updates the lf-maven-central macro as necessary to
work along side the lf-maven-stage changes.

Issue: RELENG-1554
Change-Id: Ie4fbcb52229c74c4216df4d07f077cd691ab6aa6
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
5 years agoAdd cmake-sonar job 09/14509/1
Thanh Ha [Sat, 9 Feb 2019 22:02:00 +0000 (17:02 -0500)]
Add cmake-sonar job

This job only supports SonarCloud configuration.

Change-Id: I4f2b5100e3316f21c866155199aeed0533d13b8b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd support for SonarCloud 78/14478/3
Thanh Ha [Thu, 7 Feb 2019 00:32:00 +0000 (19:32 -0500)]
Add support for SonarCloud

If SonarCloud is used the old method is automatically disabled so that
Sonar is not run twice.

Example Usage:

- project:
    name: yangtools-sonar
    jobs:
      - gerrit-maven-sonar

    project: 'yangtools'
    project-name: 'yangtools'
    branch: 'master'
    mvn-settings: 'yangtools-settings'
    mvn-opts: '-Xmx1024m'
    sonarcloud: true
    sonarcloud-project-key: KEY
    sonarcloud-project-organization: ORGANIZATION
    sonarcloud-api-token: TOKEN

Issue: RELENG-1721
Change-Id: Ica1cd29bfc9246f2452370dd6ceb8b7e5b8e5329
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoModify cron entry 77/14477/6 v0.30.0
Suresh Channamallu [Wed, 6 Feb 2019 21:27:10 +0000 (16:27 -0500)]
Modify cron entry

Modify cron entry to pick from custom config

Issue-Id: RELENG-1728
Change-Id: Icd4d97a37672fd0dece6960fa6de7d9fa2fe8a1e
Signed-off-by: Suresh Channamallu <schannamallu@linuxfoundation.org>
5 years agoVerify build node HARDWARE_ID 12/14412/4
Eric Ball [Thu, 31 Jan 2019 23:18:21 +0000 (15:18 -0800)]
Verify build node HARDWARE_ID

This script already checks each file's IMAGE_NAME for validity; this
adds essentially the same validation to check the HARDWARD_ID.
Also add the job to the ci-jobs groups, and set submodule-recursive
value for github-jenkins-cfg-verify.

Issue: RELENG-1418
Change-Id: I0266a0fcc276b2960c9c1749a89bc70e8115d152
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoAdd Release Notes for Packer Build Description 59/14359/1
Trevor Bramwell [Tue, 29 Jan 2019 18:54:09 +0000 (10:54 -0800)]
Add Release Notes for Packer Build Description

This was missed in the previous commit to add packer images to the
Jenkins build description.

Issue: RELENG-1073
Change-Id: Iced92ada13d0f576a7b560df5bc1242d83ee9689
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
5 years agoFix disable-job param typo in docs 32/14332/1
Anil Belur [Mon, 28 Jan 2019 08:56:20 +0000 (18:56 +1000)]
Fix disable-job param typo in docs

Change-Id: I28011767f45b0d29743c74cbec15f55bc48f603b
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoSet Packer Image name as Build Description 86/11786/2
Trevor Bramwell [Fri, 13 Jul 2018 23:08:36 +0000 (16:08 -0700)]
Set Packer Image name as Build Description

With the Build Description Setter plugin installed in Jenkins, the
packer image name can be extracted after a successful build.

This allows users to no longer look through the build logs when trying
to find the image name for builds.

Issue: RELENG-1073
Change-Id: Iffebd2b7e6b282d3e17a7bd6de1452e063f64603
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
5 years agoExtend ${JOB_NAME} to include {java-version} param 33/14233/3
Anil Belur [Thu, 17 Jan 2019 10:42:54 +0000 (20:42 +1000)]
Extend ${JOB_NAME} to include {java-version} param

To support javadocs jobs to build with multiple versions of
openjdk{8,11}, extend with '{java-version}' var to the ${JOB-NAME}.
Extending the job name with (java-version) var, JJB ensures
that its not passing the multiple java-versions as a list.

The job failure can be seen on the lispflowmapping jobs.
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/lispflowmapping-maven-javadoc-verify-neon/71/
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/lispflowmapping-maven-javadoc-publish-neon/19

Change-Id: If2e189f3e76f36d73863d98f1217c0b42ada52f1
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoFix coala warning 69/14169/1 v0.29.1
Anil Belur [Mon, 14 Jan 2019 12:08:25 +0000 (22:08 +1000)]
Fix coala warning

The warning was introduced with the change c90444b3
which fails the tox verify jobs.

Change-Id: I4b6241d53820f588b80f54b28bc92b635fda1b4f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd release notes for checking openjdk $VERSION 37/14137/2 v0.29.0
Anil Belur [Thu, 10 Jan 2019 23:36:50 +0000 (09:36 +1000)]
Add release notes for checking openjdk $VERSION

The release notes is for I19d77c509c90bca5fea1af53dacc35772c7ac939.

Change-Id: I81d0b2082376517ec26c978d6baa3fbd4e2e3d48
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoFix version number extraction for all cases 14/14114/1
Anil Belur [Thu, 10 Jan 2019 01:03:11 +0000 (11:03 +1000)]
Fix version number extraction for all cases

When the jobs upgrade to a later version of Java passing
"openjdk11" or "openjdk10" the string substitution would only
return the last char which would potentially fail to pick up
the correct version of Java.

This change fixes this issue to handle all cases.

Change-Id: If81100bbdd116ee071c3ecdeeadc0a3016fa2a15
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoCheck MAVEN_PARAMS for -f flag in metadata fetch 06/14106/3
Eric Ball [Wed, 9 Jan 2019 17:07:16 +0000 (09:07 -0800)]
Check MAVEN_PARAMS for -f flag in metadata fetch

maven-fetch-metadata.sh was not respecting the "-f" (for file path)
flag in MAVEN_PARAMS, causing lf-maven-merge jobs that utilize this
flag to fail.

Issue: RELENG-1630
Change-Id: I23803fd2e4a46022d85dc2befab1438853c7c3ca
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoMerge "Add documentation regarding clouds.yaml"
Thanh Ha (zxiiro) [Wed, 9 Jan 2019 16:59:29 +0000 (16:59 +0000)]
Merge "Add documentation regarding clouds.yaml"

5 years agoCheck openjdk $VERSION before setting $JAVA_HOME 05/14105/1
Anil Belur [Wed, 9 Jan 2019 02:14:44 +0000 (12:14 +1000)]
Check openjdk $VERSION before setting $JAVA_HOME

The default version of openjdk installed is openjdk-8 which
is set through the Jenkins job.

For CentOS we are installing jdk-11 and jdk-10 under /opt. So
we need to check the version of openjdk for the job to pick up
the right version of JDK.

Issue: RELENG-1454
Change-Id: I19d77c509c90bca5fea1af53dacc35772c7ac939
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Compress and upload Javadoc for publish jobs"
Thanh Ha (zxiiro) [Mon, 7 Jan 2019 22:25:00 +0000 (22:25 +0000)]
Merge "Compress and upload Javadoc for publish jobs"

5 years agoMerge "Add puppet-verify job to lf-ci-jobs"
Thanh Ha (zxiiro) [Mon, 7 Jan 2019 16:41:00 +0000 (16:41 +0000)]
Merge "Add puppet-verify job to lf-ci-jobs"

5 years agoAdd documentation regarding clouds.yaml 73/14073/2
Thanh Ha [Fri, 4 Jan 2019 20:05:08 +0000 (15:05 -0500)]
Add documentation regarding clouds.yaml

Change-Id: I7baf6a95a1552d178ba6fe48ec89448995d0f927
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Pass PYTHONPATH to tox env to fix install issues"
Thanh Ha (zxiiro) [Wed, 2 Jan 2019 13:30:16 +0000 (13:30 +0000)]
Merge "Pass PYTHONPATH to tox env to fix install issues"

6 years agoEnsure that no lingering .netrc is overriding cfg 20/14020/1
Thanh Ha [Thu, 20 Dec 2018 12:26:55 +0000 (07:26 -0500)]
Ensure that no lingering .netrc is overriding cfg

If a .netrc already exists with permissions to the same server
as logs, credentials in .netrc are read in order so the log
credentials will not be used. Ensure that no such file pre-exists
before we try to ship logs.

Change-Id: I0449702fd27837208e25a26e34a62f0b8a551c67
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoPass PYTHONPATH to tox env to fix install issues 09/14009/2
Eric Ball [Sat, 15 Dec 2018 01:20:49 +0000 (17:20 -0800)]
Pass PYTHONPATH to tox env to fix install issues

We have seen install issues that only seem to happen on build
machines. The reason these problems do not appear on local builds is
still not clear, but passing in the PYTHONPATH (set to the working
directory) is an innocuous change that fixes the problem.

Change-Id: I70217404b5a1f993d233714e2e1cac4ae9e7fd2d
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
6 years agoRevert lftools back to v0.18.0 87/13987/1 v0.28.3
Thanh Ha [Fri, 14 Dec 2018 02:00:33 +0000 (10:00 +0800)]
Revert lftools back to v0.18.0

Binary corruption issues in lftools v0.19.x stream so revert back to
v0.18.0 for now.

Change-Id: I9f9ceb94a4d5efad4307517bff72b898a10ae728
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoPin dogpile.cache to ~=0.6.8 86/13986/1
Thanh Ha [Fri, 14 Dec 2018 01:41:47 +0000 (09:41 +0800)]
Pin dogpile.cache to ~=0.6.8

dogpile.cache released a new version 0.7.x yesterday which started
breaking openstackclient. No upstream fix for this yet so pin this
library until we can upgrade.

Change-Id: I308d170fe1498bc4e03b3d6b769cca3ca822f5ac
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoCompress and upload Javadoc for publish jobs 21/13921/3
Anil Belur [Tue, 11 Dec 2018 11:47:47 +0000 (21:47 +1000)]
Compress and upload Javadoc for publish jobs

Javadoc publish job always successfully publishes to the "javadoc"
repo, therefore dont maintain another copy of this on the logs
server. Also Javadocs contain ~15K small files which potentially
requires additional CPU cycles for pruning on the logs server.

Issue: RELENG-1215
Change-Id: I1b107a1c65db4b70096d5735c741fb32f09daedd
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoAdd acumos hardware flavor 48/13948/1
Jessica Wagantall [Wed, 12 Dec 2018 00:34:10 +0000 (16:34 -0800)]
Add acumos hardware flavor

Add acumos-highcpu-4-avx hardware flavor to the
map list.

Change-Id: I5b320b0110d27bbb9ddf2f78244950609105046a
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoAdd puppet-verify job to lf-ci-jobs 35/13835/5
Eric Ball [Wed, 5 Dec 2018 16:01:38 +0000 (08:01 -0800)]
Add puppet-verify job to lf-ci-jobs

This job performs linting of Puppet modules.

Issue: RELENG-988
Change-Id: I8ab6e828e378eb9186187edb80487e0332ac144f
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
6 years agoRefactor github-maven-sonar trigger 08/13908/3
Jeremy Phelps [Mon, 10 Dec 2018 18:25:31 +0000 (12:25 -0600)]
Refactor github-maven-sonar trigger

Configure the trigger to run on weekly cron, and also
allow a trigger via GHPRB comment only.

Issue-ID: RELENG-1508
Change-Id: I82e2ab19dc52bc29c8c6549ba753ad1e2e743994
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
6 years agoUse .xz format instead of .gz 52/13852/1
Anil Belur [Thu, 6 Dec 2018 04:32:23 +0000 (14:32 +1000)]
Use .xz format instead of .gz

xz format has faster compress ratio and better memory requirements.

Change-Id: Id26f9b292fe877f9523a43063daead32ca3486a0
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoCompress and upload the jjb-verify XML files 28/13828/3
Anil Belur [Wed, 5 Dec 2018 09:19:33 +0000 (19:19 +1000)]
Compress and upload the jjb-verify XML files

The present jjb-verify job that calls lftools maven
deploy archives for releng/builder which takes about ~8 mins to
complete uploading the archives. This is because the job generates
around ~2.3K XML files (small files) which is uploaded to Nexus.
This is possibly because the Nexus Unpack plugin in the Nexus
end unpacking the zip file we upload takes longer.

It's best to compress this into a single file and upload
to Nexus, potentially minimizing the Nexus logs from
bloating up since we have been seeing higher IOPs
and load managing these files in the backend cron jobs.

Compressing the job files before hand shaves off around
6-8 mins from the upload time, therefore reducing the
job time overall significantly.

Tested the change here:
https://jenkins.opendaylight.org/releng/job/builder-jjb-verify/4325

Change-Id: Ifb3ea6e61d28b234c79088ca128294a037ec511f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoUpdate lftools to v0.19.1 with unicode fix 26/13826/1 v0.28.2
Thanh Ha [Wed, 5 Dec 2018 03:49:11 +0000 (11:49 +0800)]
Update lftools to v0.19.1 with unicode fix

Pull in the unicode fix I88f70af17222fc7931a07e78eba23d9cc60eb9e7 in
lftools.

Change-Id: I52394f3d091a0aa54e23f5ffc654c1307bf04a89
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd a note on how to test lftools master 18/13618/4
Thanh Ha [Thu, 22 Nov 2018 04:49:05 +0000 (12:49 +0800)]
Add a note on how to test lftools master

Add a note for LF Staff for how to test lftools master branch
by tweaking global-jjb.

Change-Id: I1ecc0dc7fd251eaea6e0f53c8b9d67e9651cd839
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump lftools version to v0.19.0 72/13772/1 v0.28.1
Thanh Ha [Mon, 3 Dec 2018 06:48:14 +0000 (14:48 +0800)]
Bump lftools version to v0.19.0

Change-Id: Ib28022b61efe42bbad77418d4289ddf3b9afd75f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoEnable configuring the concurrency of jjb-verify 72/13672/6 v0.28.0
Trevor Bramwell [Tue, 27 Nov 2018 22:10:30 +0000 (14:10 -0800)]
Enable configuring the concurrency of jjb-verify

The JJB verify job always runs concurrently. By turning the concurrency
setting into a variable, users will be able to run the job serially,
which may be required on static nodes.

Issue: RELENG-1509
Change-Id: I4f2b51924a71674bbf2d4bbf8acc637971470eec
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
6 years agoConvert Maven Central parameter to macro 63/13663/6
Thanh Ha [Tue, 27 Nov 2018 04:45:08 +0000 (12:45 +0800)]
Convert Maven Central parameter to macro

This would make it easier for say ODL's autorelease project to
reuse this macro.

Change-Id: I73771b26041dd9e27ca470f3f9ebb54cbe67e654
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRemove deprecated lftools-install.sh from docs 55/13655/4
Anil Belur [Mon, 26 Nov 2018 07:09:47 +0000 (17:09 +1000)]
Remove deprecated lftools-install.sh from docs

lftools-install.sh has been deprecated since v0.25.0 update
the best practices docs with the latest changes.

The change is an extension to
I770eec99ae63cf92aa145124859f09d5d327c451 which missed
updating the docs.

Change-Id: I36f36d9848cd53e3e6fd9fbabcdb5b8c36aee268
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoMerge "Unpin nodeenv version"
Anil Belur [Tue, 27 Nov 2018 09:40:38 +0000 (09:40 +0000)]
Merge "Unpin nodeenv version"

6 years agoRemove lf-stack-delete macro doc from lf-macros 37/13637/3
Thanh Ha [Fri, 23 Nov 2018 09:57:12 +0000 (17:57 +0800)]
Remove lf-stack-delete macro doc from lf-macros

This was moved to lf-openstack-heat.rst via change
Ie706706e24079b0b87f4537342fd3e8955f63fef but we forgot to remove
it from here.

Change-Id: I36a2a97d2c2b97220df0ffc96c2c2a9df79ab304
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoFix external link syntax 67/13667/1
Thanh Ha [Tue, 27 Nov 2018 09:00:33 +0000 (17:00 +0800)]
Fix external link syntax

We shouldn't be using :ref: for external links like this and is the
cause of the docs build failure we are seeing today.

Issue: RELENG-1518
Change-Id: I42bcaab4524b21e16e909d76ebdcea4476315350
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoUnpin nodeenv version 66/13666/1
Thanh Ha [Tue, 27 Nov 2018 08:41:32 +0000 (16:41 +0800)]
Unpin nodeenv version

This was originally pinned because nodeenv was breaking on 2nd and
onward runs of `tox -e coala` due to bug in nodeenv. This no longer
seems to be the case so unpin the version so we can get updates again.

Change-Id: I6a9906f3fcdc90eb546813026527ef63eea88c4e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRemove deprecated lftools-install.sh 36/13636/2
Thanh Ha [Fri, 23 Nov 2018 09:50:14 +0000 (17:50 +0800)]
Remove deprecated lftools-install.sh

lftools-install.sh has been deprecated since v0.25.0 it's about time
we remove it from global-jjb officially.

Change-Id: I770eec99ae63cf92aa145124859f09d5d327c451
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Hide irrelevant error message"
Thanh Ha (zxiiro) [Sat, 24 Nov 2018 01:00:18 +0000 (01:00 +0000)]
Merge "Hide irrelevant error message"

6 years agoHide irrelevant error message 98/13598/6
Tim Johnson [Tue, 20 Nov 2018 03:13:17 +0000 (19:13 -0800)]
Hide irrelevant error message

Use 'systemctl' to get status of 'sysstat' on newer releases of Unbuntu
(16.04 & 18.04).  Keep 'grep' to get status of 'sysstat' on Ubuntu
14.4. Unexpected releases will now generate an error.

Change-Id: I9745373d38213838381d3f2f0b46a6358014b268
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
6 years agoPrint the final requirements file 19/13619/1
Thanh Ha [Thu, 22 Nov 2018 05:09:31 +0000 (13:09 +0800)]
Print the final requirements file

Make it clear what's being pulled in by pip install.

Change-Id: I1d659de74e1eb6e37a390285ed3b9854d19edbc9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add details to the relnotes for submodule-timeouts"
Aric Gardner [Tue, 20 Nov 2018 13:50:18 +0000 (13:50 +0000)]
Merge "Add details to the relnotes for submodule-timeouts"