releng/global-jjb.git
5 years agoCleanup tox warnings 13/61713/3
Tim Johnson [Thu, 12 Sep 2019 22:58:33 +0000 (15:58 -0700)]
Cleanup tox warnings

Issue: RELENG-2365
Change-Id: I5a739a4446bfeabf73d020e6f72a8440b9d1c913
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoUpdate release container info 04/61704/4
Jessica Wagantall [Wed, 11 Sep 2019 23:28:17 +0000 (16:28 -0700)]
Update release container info

- Update schema to require pull and push container
registries
  - This will fix issues where for example ONAP pulls from
    snapshots exclusively where other projects like O-RAN
    use both snapshots or staging.
  - Allows for future push to DockerHub
  - Benefits projects like O-RAN and Acumos where they already
    provide this information for other docker jobs.
  - Pull and push registries need to be defined as Jenkins
    global variables.
  - Can be overwritten in releases files.
- Update some echos to allow users to find information faster.

Change-Id: Ibd0088c6419bacca16ded4ae86d70304c5bad45c
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoFix WS CLI Agent calls 09/61709/1
Jessica Wagantall [Thu, 12 Sep 2019 18:22:42 +0000 (11:22 -0700)]
Fix WS CLI Agent calls

Remove quotes from optional variable
WSS_UNIFIED_AGENT_OPTIONS which, if empty, will
cause WS Unified Agent CLI issues.
Upgrade WS Unified Agent CLI to latest version 19.8.1

Change-Id: I3d21bad21bceee0a0c626b145c1ca74e0cd21ddc
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoFix import key release jobs 99/61699/1 v0.43.0
Jessica Wagantall [Wed, 11 Sep 2019 20:17:37 +0000 (13:17 -0700)]
Fix import key release jobs

Import GPG signing key before verifying Gerrit
tag details.

Change-Id: I132fb8dbba51de995b0e42765344bf218340415c
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoMerge "Add python templates to run tox on merge"
Aric Gardner [Wed, 11 Sep 2019 15:22:00 +0000 (15:22 +0000)]
Merge "Add python templates to run tox on merge"

5 years agoAdd python templates to run tox on merge 21/16621/6
Lott, Christopher (cl778h) [Tue, 20 Aug 2019 15:46:31 +0000 (11:46 -0400)]
Add python templates to run tox on merge

Makes the Python job groups and templates parallel to
the maven job groups and templates.  Although no build
is required by most Python projects, re-testing after
merge is a chance to reveal failures in the new tree.

Change-Id: I1fb793e08deb435a838d76d634dbbad730a3d7fa
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
5 years agoMerge "Use configured python version to install tox-pyenv"
Aric Gardner [Wed, 11 Sep 2019 11:52:55 +0000 (11:52 +0000)]
Merge "Use configured python version to install tox-pyenv"

5 years agoUse configured python version to install tox-pyenv 63/61663/4
Lott, Christopher (cl778h) [Mon, 9 Sep 2019 15:08:37 +0000 (11:08 -0400)]
Use configured python version to install tox-pyenv

Revise the tox-install.sh script to use PYTHON env var to invoke pip,
instead of hardcoded 'python'.
Add python-version parameter to Read-The-Docs doc, macro and templates.

Change-Id: Ib0fe98dca62ce67f0ad002036b54f02faf27e888
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
Issue: RELENG-2361

5 years agoMerge "Fix self release script indent"
Jessica Wagantall [Mon, 9 Sep 2019 20:54:53 +0000 (20:54 +0000)]
Merge "Fix self release script indent"

5 years agoRemove passive voice in doc to silence warnings 50/61650/2
Lott, Christopher (cl778h) [Sun, 8 Sep 2019 11:04:37 +0000 (07:04 -0400)]
Remove passive voice in doc to silence warnings

The write-good feature as invoked by tox coala WriteGoodLintBear
complains about usage of passive voice e.g., 'is applied'.
These changes silence all but two warnings about use of the word
'validate' which is a well-known Maven goal.
This makes no functional change to any templates.

Change-Id: I5a68a475997cbeeae5ce41a2e487d67cc3c2644f
Issue: RELENG-2365
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
5 years agoFix self release script indent 49/61649/1
Anil Belur [Sun, 8 Sep 2019 03:56:42 +0000 (09:26 +0530)]
Fix self release script indent

Use 4 spaces indent. This makes the code easily readable
and consistant with rest of the code in global-jjb.

Change-Id: I62edca97517fe456d0fe688e2826b0d0fce8432a
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoRepair tox-sonar behavior by restoring mvn goal 12/61612/3
Lott, Christopher (cl778h) [Fri, 6 Sep 2019 00:38:46 +0000 (20:38 -0400)]
Repair tox-sonar behavior by restoring mvn goal

Use existing builder lf-infra-maven-sonar, drop the locally
defined builder lf-infra-tox-sonar, to get desired behavior
of generating code analysis then publishing to Sonar. This
broke when hardcoded goals were removed from maven-sonar.sh.

Change-Id: I862c4857d60361d77b390425d50f3c7c77e3919d
Issue: RELENG-2359
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
5 years agoUpdate release vars functions 83/61583/7
Jessica Wagantall [Wed, 4 Sep 2019 00:25:52 +0000 (17:25 -0700)]
Update release vars functions

Release schema verification needs to happen
first before we attempt to assign values to
the variables.

Organize variable setup into functions.
Maven release files expects different variables
than container release files.

Rename "version" variable in container release
files to "container_release_tag" which is a
better user friendly name given the fact that
container versions are rather called tags.
Internally, we still process it as "version"
to allow reuse of the tag function.

Issue: RELENG-2353
Change-Id: Ie0456a1fa87ea62855cb5d1140f6ae9f32b3e566
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoMerge "Clarify limit of one release yaml file"
Aric Gardner [Wed, 4 Sep 2019 16:21:48 +0000 (16:21 +0000)]
Merge "Clarify limit of one release yaml file"

5 years agoClarify limit of one release yaml file 78/61578/3
Lott, Christopher (cl778h) [Tue, 3 Sep 2019 19:06:03 +0000 (15:06 -0400)]
Clarify limit of one release yaml file

Specify that Centos + Sigul + Docker are required.
Rewrap all paragraphs to eliminate long lines.

Change-Id: I57f46405cb962aed31c578354147b2b414da811f
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
5 years agoRemove the unused MAVEN_CENTRAL_URL variable 86/61586/1
Anil Belur [Wed, 4 Sep 2019 11:48:26 +0000 (17:18 +0530)]
Remove the unused MAVEN_CENTRAL_URL variable

Remove the unused MAVEN_CENTRAL_URL variable. The self-release job is
designed to work with any Nexus repository info published in
`staging-repo.txt.gz`, which makes the `MAVEN_CENTRAL_URL` redundant,
hence remove the unused variable.

Issue: RELENG-2129
Change-Id: Iabb772d908f8001a6626d728a9140ff14e3088f2
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMinor rewording and reformatting of release doc 76/61576/3
Lott, Christopher (cl778h) [Tue, 3 Sep 2019 17:53:27 +0000 (13:53 -0400)]
Minor rewording and reformatting of release doc

Change-Id: I4d91d42bc2bd5cee4ae1e8dcebaba659bc168b6a
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
5 years agoMerge "INFO validate job improvements"
Anil Belur [Sat, 31 Aug 2019 03:04:35 +0000 (03:04 +0000)]
Merge "INFO validate job improvements"

5 years agoAdd Sonar plug-in configurations to docs 48/61548/1
Houa Yang [Fri, 30 Aug 2019 14:23:08 +0000 (09:23 -0500)]
Add Sonar plug-in configurations to docs

Add documentation for configuring the Jenkins Sonarqube Scanner plug-in.

Signed-off-by: Houa Yang <hyang@contractor.linuxfoundation.org>
Change-Id: I21b21efc6cefa3f4f643a0235d670c6e1ec4ac08

5 years agoINFO validate job improvements 36/61536/3
Aric Gardner [Thu, 29 Aug 2019 18:54:03 +0000 (14:54 -0400)]
INFO validate job improvements

removed +x trace as job is too verbose

verify now checks that repositories matches $PROJECT
Catches projects that replace / with - in their INFO file
Also ensures that repositories only has one entry.
We are not supporting multiple projects with a single INFO.yaml
file.

ISSUE: RELENG-2344
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Id847b7c10ef0515fd6f6a4b7d6809ebd37a9b51d

5 years agoMerge "Remove detox in favor of tox --parallel"
Anil Belur [Thu, 29 Aug 2019 14:49:46 +0000 (14:49 +0000)]
Merge "Remove detox in favor of tox --parallel"

5 years agoUpdate lftools version to v0.26.2 20/61520/1 v0.42.1
Anil Belur [Wed, 28 Aug 2019 18:52:30 +0000 (00:22 +0530)]
Update lftools version to v0.26.2

Change-Id: I069dda979d203b1960821e4d112a5591843fd441
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd DRY_RUN param to the release job 20/16720/5
Anil Belur [Wed, 28 Aug 2019 18:02:03 +0000 (23:32 +0530)]
Add DRY_RUN param to the release job

Add DRY_RUN build param to do a test run the job with
publishing artifacts.

Change-Id: I1b015db4be8ed3e8c39bec2cefe022819e31e59f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Verify both repos before attempting release"
Anil Belur [Wed, 28 Aug 2019 16:31:17 +0000 (16:31 +0000)]
Merge "Verify both repos before attempting release"

5 years agoVerify both repos before attempting release 19/16719/3
Aric Gardner [Wed, 28 Aug 2019 15:39:00 +0000 (11:39 -0400)]
Verify both repos before attempting release

We have run into a case where the repo on ODL
was good, and the repo on nexus was failed.
Cover this case by running the loop twice.

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I995a34918481db9e41d5486c06995f57100c1199

5 years agoRemove detox in favor of tox --parallel 16/16716/2
Thanh Ha [Tue, 27 Aug 2019 22:13:38 +0000 (18:13 -0400)]
Remove detox in favor of tox --parallel

Tox officially supports parallel mode as of Tox 3.7 so drop the
usage of detox. Set parallel mode to 'auto' which will run as many
threads as available CPUs.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: Id59ad10ed9b521a5c0ca3fef3e7e15053c2dc32c

5 years agoFix cmake docs 15/16715/2
Houa Yang [Tue, 27 Aug 2019 21:32:58 +0000 (16:32 -0500)]
Fix cmake docs

Remove unnecessary info for :install-prefix parameter.

Issue: RELENG-2210

Signed-off-by: Houa Yang <hyang@contractor.linuxfoundation.org>
Change-Id: I6cebee91b62f240c9cd8ade4b80573621b060ef0

5 years agoConfigure basepython to python3 06/16706/2
Thanh Ha [Tue, 27 Aug 2019 19:08:54 +0000 (15:08 -0400)]
Configure basepython to python3

More and more libraries that we are pulling in require Python 3 so
to avoid having to play the pin dependencies whack-a-mole game let's
bump or base python where possible to python3.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I65208f80b9f0479a10b1a8b6a4ba52956c2c3e3c

5 years agoUpgrade lftools to v0.26.1 77/16677/1 v0.42.0
Anil Belur [Tue, 27 Aug 2019 03:25:00 +0000 (08:55 +0530)]
Upgrade lftools to v0.26.1

Change-Id: I66a72fcb7cff0148eb0f680df13e8744b17c7025
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoFix use-release-file varaible 73/16673/3
Jessica Wagantall [Mon, 26 Aug 2019 20:59:33 +0000 (13:59 -0700)]
Fix use-release-file varaible

Rename "RELEASE_FILE" parameter to "USE_RELEASE_FILE" in
release-jobs. This will match the actual varaible default value
better and will not collide with the local "release_file" in
the script.

Fix "USE_RELEASE_FILE" if statement. We are now using a bool
instead of a string.

Changing the if statements to evaluate bools.

Change-Id: I9909a16487fabacc0261f6489ac81a00cbf0f965
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoPin more-itertools~=5.0.0 69/16669/4
Anil Belur [Mon, 26 Aug 2019 19:12:17 +0000 (00:42 +0530)]
Pin more-itertools~=5.0.0

Pin more-itertools ~= 5.0.0, since version 6.0.0 requires Python 3.4
https://github.com/erikrose/more-itertools/releases

Error:
more-itertools requires Python '>=3.4' but the running Python is 2.7.5

Pin tox~<3.5.0

Change-Id: I1f2eefefc3f5b58a14441db78fb3f0bf9061e711
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Add pre-build scripts to Python templates"
Eric Ball [Sat, 24 Aug 2019 00:25:49 +0000 (00:25 +0000)]
Merge "Add pre-build scripts to Python templates"

5 years agoAdd support for container releases 70/16470/47
Aric Gardner [Wed, 7 Aug 2019 18:05:51 +0000 (14:05 -0400)]
Add support for container releases

- Add support for distribution_type container
- Add conditional build step.
If {build-node} is ^.*-docker-.*
docker login will occur for verify and merge.
- Trigger updated to support hidden releases
directory. (releases\/.*\.yaml|\.releases\/.*\.yaml)
- Container support:
    container_release_file example:

        containers:
            - name: example
            version: 1.5.1-20190806T184921Z

    - Verify pulls container
    - Grab the image_id
    - Merge
    - Apply tag
    - Push the container

ISSUE: RELENG-2292
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ibc6cf1a22bd6c0136a94d2b434c6cd332c12b43f

5 years agoAdd pre-build scripts to Python templates 52/16652/3
Lott, Christopher (cl778h) [Fri, 23 Aug 2019 12:36:56 +0000 (08:36 -0400)]
Add pre-build scripts to Python templates

New parameter pre-build-script accepts a shell script
that can install prerequisite libraries, rearrange the
source tree to cope with nested projects, etc.

Change-Id: I1fd7c2c094a664244e697ff5a61b82a2e7752faa
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
5 years agoMerge "Add lf-sonar-common job-template"
Eric Ball [Fri, 23 Aug 2019 22:25:38 +0000 (22:25 +0000)]
Merge "Add lf-sonar-common job-template"

5 years agoMerge "Allow Build with Params releases"
Aric Gardner [Fri, 23 Aug 2019 18:13:56 +0000 (18:13 +0000)]
Merge "Allow Build with Params releases"

5 years agoAllow Build with Params releases 47/16647/3
Aric Gardner [Fri, 23 Aug 2019 00:02:05 +0000 (17:02 -0700)]
Allow Build with Params releases

Add support for "Build with Parameters" for projects
that do not want to use a release file for maven builds.
Restructure shell/release-job.sh into functions.

Change-Id: Iada9c9a50710c24fbfe87b2ea0def10f3dd04bd7
Authored-By: Aric Gardner <agardner@linuxfoundation.org>
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoAdd lf-sonar-common job-template 71/16571/8
Houa Yang [Thu, 15 Aug 2019 21:40:44 +0000 (16:40 -0500)]
Add lf-sonar-common job-template

Add lf-sonar-common job-template for sonar-project.properties

Issue: RELENG-1304

Signed-off-by: Houa Yang <hyang@contractor.linuxfoundation.org>
Change-Id: I6788bf9afb3b25fb751b806f08fddcf2fed42d26

5 years agoFix release-schema.yaml 46/16646/1
Jessica Wagantall [Thu, 22 Aug 2019 23:23:16 +0000 (16:23 -0700)]
Fix release-schema.yaml

Fix missing extension in ID for release-schema.yaml.

Make "distribution_type" mandatory in future
release files.

Change-Id: I813f6b863fabfafcc9560df753c380675e8cc09e
Co-Authored-By: Aric Gardner <agardner@linuxfoundation.org>
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoMerge "Upgrade lftools to v0.26.0" v0.41.0
Andrew Grimberg [Mon, 19 Aug 2019 21:52:32 +0000 (21:52 +0000)]
Merge "Upgrade lftools to v0.26.0"

5 years agoMerge "Update packer-version to 1.4.3"
Andrew Grimberg [Mon, 19 Aug 2019 21:42:57 +0000 (21:42 +0000)]
Merge "Update packer-version to 1.4.3"

5 years agoHard code release job compare pattern. 02/16602/6
Aric Gardner [Mon, 19 Aug 2019 19:07:54 +0000 (15:07 -0400)]
Hard code release job compare pattern.

This change:
https://gerrit.acumos.org/r/c/ci-management/+/4979

The project section has:
gerrit_trigger_file_paths:
  - compare-type: REG_EXP
  pattern: '^((?!migrate-cms-to-cds|\/COMMIT_MSG).)*$'

This is overwriting the default parameters of this job.

gerrit_trigger_file_paths:
  - compare-type: REG_EXP
    pattern: "releases/*.yaml"

I belive I should hard code this value so that this
does not happen.

Pattern going forward will actually support
pattern: '(releases\/.*\.yaml|\.releases\/.*\.yaml)'

I will update 16470 and hard code as well, if this is
merged.

ISSUE: RELENG-2316
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ib520672a548f34762c723972152abea898cd98a8

5 years agoUpdate packer-version to 1.4.3 01/16601/3
Suresh Channamallu [Mon, 19 Aug 2019 18:52:59 +0000 (14:52 -0400)]
Update packer-version to 1.4.3

This new packer version fixed an issues in
docker image, where its unable to install the
packages into docker containers due to checking
of wrong container OS.

Fix in 1.4.3:
builder/docker: Check container os, not host os,
when creating container dir default [GH-7939]

Issue: RELENG-2180
Signed-off-by: Suresh Channamallu <schannamallu@linuxfoundation.org>
Change-Id: I3fe36e5669942d1f7fadfd2b711f9d7a3ca1e5a9

5 years agoUpgrade lftools to v0.26.0 73/16573/1
Anil Belur [Fri, 16 Aug 2019 02:48:13 +0000 (08:18 +0530)]
Upgrade lftools to v0.26.0

Change-Id: Iaa97109a097a85d0847372b8d7c9515960a4cd2e
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate release-jobs docs. 91/16491/2
Aric Gardner [Thu, 8 Aug 2019 17:02:19 +0000 (13:02 -0400)]
Update release-jobs docs.

Feedback has allowed me to
Clarify the procedure

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I421cb6eebfccb48bde02d2a9d75ff020f895fde3

5 years agoMerge "Resolve ShellCheck errors in shell scripts"
Andrew Grimberg [Thu, 8 Aug 2019 16:44:56 +0000 (16:44 +0000)]
Merge "Resolve ShellCheck errors in shell scripts"

5 years agoUse --config-xml option for JJB archives 79/16479/2
Thanh Ha [Thu, 8 Aug 2019 02:20:22 +0000 (22:20 -0400)]
Use --config-xml option for JJB archives

The --config-xml option will produce actual config xml files in the
format 'job_name/config.xml' similar to the jobs directory in Jenkins
master instead of creating generically named 'job_name' files.

This allows the jobs to be dropped into a Jenkins jobs directory
path and Jenkins will be able to pick it up.

This is potentially useful for the ability to quickly drop in the
config into a Jenkins test environment.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I59517d905a2448e46c50d98a471e10d7ca0fb808

5 years agoResolve ShellCheck errors in shell scripts 80/16480/1
Thanh Ha [Thu, 8 Aug 2019 02:32:05 +0000 (22:32 -0400)]
Resolve ShellCheck errors in shell scripts

* SC2086: Double quote to prevent globbing and word splitting.
* SC2027: The surrounding quotes actually unquote this.
          Remove or escape them.
* SC2016: Expressions don't expand in single quotes,
          use double quotes for that.
* SC2236: Use -n instead of ! -z.
* SC2233: Remove superfluous (..) around condition.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I73207aa87646472bdb1c7848ba9608ac702bda53

5 years agoMerge "Use release credentials only for merge job wrappers" v0.40.4
Jessica Wagantall [Tue, 6 Aug 2019 23:18:55 +0000 (23:18 +0000)]
Merge "Use release credentials only for merge job wrappers"

5 years agoMerge "Spell check fixes in release-jobs docs"
Aric Gardner [Tue, 6 Aug 2019 15:00:49 +0000 (15:00 +0000)]
Merge "Spell check fixes in release-jobs docs"

5 years agoUse release credentials only for merge job wrappers 51/16451/2
Anil Belur [Tue, 6 Aug 2019 03:38:38 +0000 (13:38 +1000)]
Use release credentials only for merge job wrappers

Release creds are only required for promoting the repo, which uses diff ACL
as compared to normal user.

Therefore dont use the release creds for the verify jobs and the
scm sections in both the job templates.

Issue: RELENG-2129
Change-Id: Ibb36d11ba57c5e59a63c414b0b338846eefd8983
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoHad an if where I needed a fi 52/16452/2
Aric Gardner [Tue, 6 Aug 2019 02:08:41 +0000 (22:08 -0400)]
Had an if where I needed a fi

Also the git fetch needs the dashed version
git fetch "$PATCH_DIR/${PROJECT//\//-}.bundle"

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ib4245be2b46645fca93225a5c32efb904acd68c8

5 years agoUpgrade lftools version to v0.25.4 50/16450/1
Anil Belur [Tue, 6 Aug 2019 00:15:37 +0000 (10:15 +1000)]
Upgrade lftools version to v0.25.4

Change-Id: I10460e230bccbf69cf8e8e12244f6c7a88cdccca
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoSpell check fixes in release-jobs docs 30/16430/2
Anil Belur [Sun, 4 Aug 2019 01:10:28 +0000 (11:10 +1000)]
Spell check fixes in release-jobs docs

Change-Id: I76aacf289bd46a1578c0741d58f2f3be4f48f28f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoExit 0 if release job has already tagged a repo. 41/16441/14
Aric Gardner [Mon, 5 Aug 2019 17:32:04 +0000 (13:32 -0400)]
Exit 0 if release job has already tagged a repo.

The release merge job is a one way operation.
Given this, Release jobs should only exist in the format
{project-name}-release-verify
and
{project-name}-release-merge

They should trigger from a change to any branch/**
As a developer working on {example-branch} will not want to
check out master to push a new release file.

Edge Cases:
Inevitably a release file will be pulled in from master to {example-branch}
or a remerge will be requested
This will again trigger the release jobs.
since in this case the repo is already tagged, the job should not report
a failure.

This is solved by having the verfiy and merge exit 0 when the repo is already tagged

if git tag -v "$VERSION"; then
  echo "Repo already tagged $VERSION"
  echo "This job has already run exit 0"
  exit 0
if

Example Workflow - Milestone 0 is released from master:
releases/1.4.0.yaml (on branch master)
tag: 1.4.0
log_dir: 'optf-osdf-maven-stage-master/97/'

Repo is tagged and artifact is pushed!

Milestone 3 is released from dublin:
releases/1.4.3.yaml (on branch dublin)
tag: 1.4.3
log_dir: 'optf-osdf-maven-stage-dublin/20/'

Repo is tagged and artifact is pushed!

Note:
the ref is determined by the log file.
So We could check if this ref can be found on the {branch}
to reduce human errors.
example:
odlparent-maven-stage-4.0.x/152/patches/taglist.log.gz
has:
odlparent c8964051cf81e2bd5d831fe6e5d568c5eedeeb8e

Psudo code:
STREAM=${STREAM:-'nostream'}
echo "--> Checking if $ref is on stable/$STREAM"
if ! (git branch -a --contains $ref | grep "stable/$STREAM"); then
    echo "--> ERROR: $ref for $repo is not on stable/$STREAM!"
fi

Also in this patch:
To reduce possible errors:
Don't get project name from the release file:
PROJECT="$(niet ".project" "$release_file")"
we can just use ${PROJECT//\//-} when we need
the /'s changed to -'s

It will allways be correct when passed to jenkins
from gerrit

Also remove quiet from wget,
if there is a failure I want to see it on the console.

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I1e95e864bc2b49a62b3f7016cb2f1292a6687d9a

5 years agoMerge "Add release notes for lftools v0.25.3 release" v0.40.3
Aric Gardner [Sun, 4 Aug 2019 23:26:45 +0000 (23:26 +0000)]
Merge "Add release notes for lftools v0.25.3 release"

5 years agoImprove the release job script 32/16432/1
Anil Belur [Sun, 4 Aug 2019 22:55:50 +0000 (08:55 +1000)]
Improve the release job script

Change-Id: Ie62e2f3771632012423e0b791aca1551b4b31112
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate the release job tigger 31/16431/2
Anil Belur [Sun, 4 Aug 2019 08:35:30 +0000 (18:35 +1000)]
Update the release job tigger

Update release job template to tigger on any branch name, and not just
'**/master'. ODL projects branches are version '4.0.x' which requires
passing the branch name to the template.

Issue: RELENG-2129
Change-Id: I9c29d1c986fee73f6bad4fe39af8f9ffbda201c6
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd release notes for lftools v0.25.3 release 29/16429/1
Anil Belur [Sun, 4 Aug 2019 01:08:22 +0000 (11:08 +1000)]
Add release notes for lftools v0.25.3 release

This was missed in CR Id64b36e8f59d500800d6b9078f56b314348ed1f2

Change-Id: Ie5412ea5724db51fe1eb73fd5f9370b2b6e3b6d2
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoVerify job needs target/variable switch 28/16428/1 v0.40.2
Aric Gardner [Sat, 3 Aug 2019 15:59:44 +0000 (11:59 -0400)]
Verify job needs target/variable switch

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I46501ed93f873c87a7d297398c1eb0dc9d23700f

5 years agoFix release-job.sh 26/16426/1 v0.40.1
Jessica Wagantall [Fri, 2 Aug 2019 23:04:08 +0000 (16:04 -0700)]
Fix release-job.sh

Use {GERRIT_PROJECT} when calling Gerrit in
release-merge job.

Change-Id: I0d62d5c713858e7e43b2fc6a6462813b01bea2f8
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoFix signing pubkey declaration 25/16425/1
Andrew Grimberg [Fri, 2 Aug 2019 22:42:47 +0000 (15:42 -0700)]
Fix signing pubkey declaration

Change-Id: Id9fb1cfa8c992949266622e3a2ed2100e8bc78f6
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoMerge "Update lftools version to v0.25.3"
Aric Gardner [Fri, 2 Aug 2019 21:48:04 +0000 (21:48 +0000)]
Merge "Update lftools version to v0.25.3"

5 years agoHandle multiple repos listed in staging-repo. 13/16413/19
Anil Belur [Fri, 2 Aug 2019 09:34:31 +0000 (19:34 +1000)]
Handle multiple repos listed in staging-repo.

Handle multiple repos listed in staging-repo for ODL.

Also multiple release files in the same commit will not work.
exiting if this case is detected

Also
git diff HEAD^1 --name-only
is bad. if there is another merge it fails.

Change to
git diff-tree --no-commit-id -r $GERRIT_PATCHSET_REVISION \
    --name-only -- "releases/"

StrictHostKeyChecking is enabled on the builder.
disabled

Also needed to change

{jenkins-ssh-credential}
to
{jenkins-ssh-release-credential}

Also
Give group ``self-serve-release`` Forge Committer rights
on refs/tags/
and allow on Create Signed Tag Create Annotated Tag

Also
- file-id: signing-pubkey
target: SIGNING_PUBKEY

Also fix nexus url needs https://

Issue: RELENG-2129
Change-Id: I55dc0a4022f93a6c3c0f389c04c88c38c53eff64
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate lftools version to v0.25.3 24/16424/1
Anil Belur [Fri, 2 Aug 2019 21:34:35 +0000 (07:34 +1000)]
Update lftools version to v0.25.3

Change-Id: Id64b36e8f59d500800d6b9078f56b314348ed1f2
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd upgrade information 11/16411/1
Jessica Wagantall [Fri, 2 Aug 2019 02:11:16 +0000 (19:11 -0700)]
Add upgrade information

Add upgrade information for commit
729b2e3535f009a146719df7d7b52e036b2505cb

Change-Id: Id27a14aaff31153dad534794ef70534d0bfe8bf7
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoImport the pubkey and check the tag 10/16410/2
Aric Gardner [Fri, 2 Aug 2019 01:27:18 +0000 (21:27 -0400)]
Import the pubkey and check the tag

with git tag -v $VERSION

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I954dfa1c8ca09f64ff8a35c138290565de8de4b9

5 years agoProject pattern was incorrectly set to ** 05/16405/2
Aric Gardner [Thu, 1 Aug 2019 18:09:34 +0000 (14:09 -0400)]
Project pattern was incorrectly set to **

must be {project}

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ic2db72185a3b2e2eb22011b2bc5e8f6012c235c3

5 years agoMerge "Self serve release docs" v0.40.0
Anil Belur [Wed, 31 Jul 2019 23:39:20 +0000 (23:39 +0000)]
Merge "Self serve release docs"

5 years agoSelf serve release docs 44/15944/20
Aric Gardner [Wed, 19 Jun 2019 18:10:42 +0000 (14:10 -0400)]
Self serve release docs

And code fixes for new:
jenkins-ssh-release-credential

Fix in release-job.sh
These were overlooked for merge:

git config user.name
git config user.email
gerrit_ssh=$(echo "$GERRIT_URL" | awk -F"/" '{print $3}')
git remote set-url origin ssh://"$RELEASE_USERNAME"@"$gerrit_ssh":29418/$PROJECT

Also included a small capitalization fix in lf-info-vote.rst

ISSUE: RELENG-2127
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Change-Id: I11ad122153d71a3d25d6b8839e09435f8b27b19d

5 years agoMerge "Log $JOB_NAME with the jjb-deploy"
Andrew Grimberg [Wed, 31 Jul 2019 22:17:17 +0000 (22:17 +0000)]
Merge "Log $JOB_NAME with the jjb-deploy"

5 years agoMerge "Update lf-info-yaml-verify"
Jessica Wagantall [Wed, 31 Jul 2019 22:15:21 +0000 (22:15 +0000)]
Merge "Update lf-info-yaml-verify"

5 years agoLog $JOB_NAME with the jjb-deploy 48/16348/3
Anil Belur [Mon, 29 Jul 2019 01:33:00 +0000 (11:33 +1000)]
Log $JOB_NAME with the jjb-deploy

Change-Id: Ifb2715402dc1dcb21708e4dc8851e0c621d59145
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Consolidate lf-infra-jjbini with JJB 2.0"
Andrew Grimberg [Wed, 31 Jul 2019 22:03:18 +0000 (22:03 +0000)]
Merge "Consolidate lf-infra-jjbini with JJB 2.0"

5 years agoConsolidate lf-infra-jjbini with JJB 2.0 50/16350/6
Anil Belur [Mon, 29 Jul 2019 03:35:20 +0000 (13:35 +1000)]
Consolidate lf-infra-jjbini with JJB 2.0

There is no need to have two separate versions of the
macro anymore since we are already on JJB 2.0

Change-Id: I2295489f8d0bebf9e1628db91c38e74d3839b8a0
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate lf-info-yaml-verify 70/16370/5
Jessica Wagantall [Tue, 30 Jul 2019 01:05:00 +0000 (18:05 -0700)]
Update lf-info-yaml-verify

Move info-schema to schema/info-schema.yaml
to keep schemas consistency.
Download only needed files for lf-info-yaml-verify
rather than cloning the entire repo.

Issue: RELENG-2242
Change-Id: Ie135c761d1401957be33c98634467967cc085ce2
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoMerge "Extend sonatype-clm.sh goals"
Jessica Wagantall [Tue, 30 Jul 2019 21:05:21 +0000 (21:05 +0000)]
Merge "Extend sonatype-clm.sh goals"

5 years agoFix wget release-schema.yaml 69/16369/2
Jessica Wagantall [Tue, 30 Jul 2019 00:32:05 +0000 (17:32 -0700)]
Fix wget release-schema.yaml

Download raw version of release-schema.yaml to
compare against release files using lftools.

Issue: RELENG-2131
Change-Id: I82b505260ff172ddb47ad0e58e8721f91ed98581
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoFix release-job.sh Nexus proxy var - 2 68/16368/1
Jessica Wagantall [Tue, 30 Jul 2019 00:14:25 +0000 (17:14 -0700)]
Fix release-job.sh Nexus proxy var - 2

Avoid the usage of project specific variables.
Do not use ODLNEXUSPROXY var, but instead use
a generalized variable.

Commit 118b7cbf171aca498d1a0a3a485bad990ad2e7b6
missed this variable.

Issue: RELENG-2131
Change-Id: I3adee1f41616495aab3afe262e8fa70902d619f7
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoExtend sonatype-clm.sh goals 46/16346/4
Jessica Wagantall [Sat, 27 Jul 2019 02:08:16 +0000 (19:08 -0700)]
Extend sonatype-clm.sh goals

Allow to configure maven goals in
lf-maven-clm to allow running test in
some projects.

Issue: RELENG-2236
Change-Id: I83c0efb8b3749483554bf4c1db04ea77c2d17cd7
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoUpdate staging jobs triggers 11/16311/12
Jessica Wagantall [Tue, 23 Jul 2019 19:39:46 +0000 (12:39 -0700)]
Update staging jobs triggers

- Allow lf-maven-stage jobs to be triggered
using either "stage-release" or "stage-maven-release".
- Allow lf-maven-docker-stage jobs to be triggered
using either "stage-release" or "stage-docker-release".

Change-Id: I133063d0cf99ab9b6a06b452ac392959bc678b39
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoPin numpy to 1.16.4 44/16344/2
Jessica Wagantall [Fri, 26 Jul 2019 21:27:43 +0000 (14:27 -0700)]
Pin numpy to 1.16.4

numpy numpy-1.17.0 requires python 3.5

Change-Id: I4bfffcf2bd7b4cf8c9246d15b6fbe819d86e695d
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoMerge "Add packer image $NAME in description setting"
Andrew Grimberg [Fri, 26 Jul 2019 19:51:55 +0000 (19:51 +0000)]
Merge "Add packer image $NAME in description setting"

5 years agoUpgrade wss-unified-agent 40/16340/1
Jessica Wagantall [Fri, 26 Jul 2019 17:54:44 +0000 (10:54 -0700)]
Upgrade wss-unified-agent

Upgrade to the WhiteSource Unified Agent
version 19.7.1.

Issue: RELENG-2237
Change-Id: Iac87fc09597a79586718cc3b07ec7034e4a95c13
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoAdd packer image $NAME in description setting 15/16315/3
Anil Belur [Wed, 24 Jul 2019 12:17:27 +0000 (22:17 +1000)]
Add packer image $NAME in description setting

Add packer image $NAME in description setting so that the image name
is displayed above the job logs URL. This saves a little time from
looking for the image name in the console logs.

Tested:
https://jenkins.opendaylight.org/releng/view/packer/job/builder-packer-merge-centos-7-builder/57/

Issue: RELENG-2230
Change-Id: Id86ee9795e1bfd50850c4833fc6f90545367f113
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Add option to pass JAVA_OPTIONS"
Andrew Grimberg [Thu, 18 Jul 2019 16:03:54 +0000 (16:03 +0000)]
Merge "Add option to pass JAVA_OPTIONS"

5 years agoAdd option to pass JAVA_OPTIONS 26/16226/9
Suresh Channamallu [Tue, 16 Jul 2019 17:36:29 +0000 (13:36 -0400)]
Add option to pass JAVA_OPTIONS

JAVA_OPTIONS are required to execute sonar jobs
Currently there is no options to pass this variable
so adding this option to pass from the project.yaml

Issue: RELENG-2152
Signed-off-by: Suresh Channamallu <schannamallu@linuxfoundation.org>
Change-Id: I0842f222fc7abf11b258b0691167b701bb319458

5 years agoFix release-job schema and central url 15/16215/4
Jessica Wagantall [Tue, 16 Jul 2019 01:06:39 +0000 (18:06 -0700)]
Fix release-job schema and central url

Perform "lftools schema verify" command to validate the release files
against schema/release-schema.yaml
Obtain optional maven central URL inside the loop that scans release
files.

Issue: RELENG-2131
Change-Id: I6af1cbb32f25ad33385bac176dc57021d94a5043
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoMerge "Fix release-job.sh version var"
Andrew Grimberg [Fri, 12 Jul 2019 18:02:26 +0000 (18:02 +0000)]
Merge "Fix release-job.sh version var"

5 years agoFix release-job.sh version var 25/16025/3
Jessica Wagantall [Fri, 28 Jun 2019 01:17:24 +0000 (18:17 -0700)]
Fix release-job.sh version var

Allow only semantic release versions like
"v${SEMVER}" or "${SEMVER}".
Fail the script if the version is not valid.
Do not append any additional characters to the
release version during tagging and pushing steps.

In this case, if the user wants to use the prefixed
"v", they can define it in the releases/*.yaml as:
version: 'v1.0.0'

Issue: RELENG-2131
Change-Id: I71f32600d3e17ec117776be6764c2e45ee934bdd
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoMerge "Set $JAVA_HOME for openjdk12 on CentOS7"
Andrew Grimberg [Fri, 12 Jul 2019 15:15:29 +0000 (15:15 +0000)]
Merge "Set $JAVA_HOME for openjdk12 on CentOS7"

5 years agoUse prettier for automatic formatting for YAML 41/16141/3
Thanh Ha [Wed, 10 Jul 2019 02:12:48 +0000 (22:12 -0400)]
Use prettier for automatic formatting for YAML

No more debate on style formatting, prettier is an opinionated
style formatter that will automatically format to the right
style.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: If9aae2087c3299ae5d54728cd17d86c2b0dcb8a0

5 years agoFix formatting to prettier style 40/16140/3
Thanh Ha [Wed, 10 Jul 2019 02:14:36 +0000 (22:14 -0400)]
Fix formatting to prettier style

Refer to If9aae2087c3299ae5d54728cd17d86c2b0dcb8a0 for details
on the implementation.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: If845d7ecf8ee229bdd2407038cfe6bb0e2a9194f

5 years agoAdd release-schema 40/16040/3
Jessica Wagantall [Sat, 29 Jun 2019 01:37:20 +0000 (18:37 -0700)]
Add release-schema

Add schema file used to verify releases yaml file
contents during lf-release-jobs

Issue: RELENG-2131
Change-Id: I8466269333f6f6597ee2a3b1c55b3140c1ea1e23
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoMerge "Standardize "release" wording"
Jessica Wagantall [Wed, 10 Jul 2019 18:56:05 +0000 (18:56 +0000)]
Merge "Standardize "release" wording"

5 years agoSet $JAVA_HOME for openjdk12 on CentOS7 46/16146/1
Anil Belur [Wed, 10 Jul 2019 11:17:58 +0000 (21:17 +1000)]
Set $JAVA_HOME for openjdk12 on CentOS7

Issue: RELENG-2202
Change-Id: Ib2a53613bfb3b6f1ee388d2f621356b1cefb185e
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Add workaround for javadoc verify jobs"
Anil Belur [Wed, 10 Jul 2019 08:50:17 +0000 (08:50 +0000)]
Merge "Add workaround for javadoc verify jobs"

5 years agoMerge "Fix script indent to 4 spaces"
Anil Belur [Wed, 10 Jul 2019 08:49:49 +0000 (08:49 +0000)]
Merge "Fix script indent to 4 spaces"

5 years agoMerge "Handle stack removal with the force option"
Anil Belur [Wed, 10 Jul 2019 00:12:22 +0000 (00:12 +0000)]
Merge "Handle stack removal with the force option"

5 years agoStandardize "release" wording 38/16038/5
Jessica Wagantall [Fri, 28 Jun 2019 22:14:52 +0000 (15:14 -0700)]
Standardize "release" wording

Using "releases" and "release" in different places
is becoming a little confusing. Standardize to "release"
to match lftools command and the majority of the exisiting
wording.
Using "releases" for the list of tech team releases does
makes sense to be kept that way, for example:
releases/1.1.1.yaml

Issue: RELENG-2131
Change-Id: I9a9a548947cc1acbe3e61bb8fa6532a89fb17e16
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>