releng/global-jjb.git
7 years agoMerge "Fix docker-login to conditionally use -e" v0.8.1
Andrew Grimberg [Tue, 5 Sep 2017 18:51:47 +0000 (18:51 +0000)]
Merge "Fix docker-login to conditionally use -e"

7 years agoFix docker-login to conditionally use -e 18/6318/1
Jeremy Phelps [Tue, 5 Sep 2017 18:37:57 +0000 (13:37 -0500)]
Fix docker-login to conditionally use -e

The -e/--email flag was removed in docker version
17.06.0.  Do not use if equal or less than that version.

Change-Id: I69335d8ff068fbb180de284317d02f21b8b7e552
JIRA: RELENG-452
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
7 years agoRelease global-jjb v0.8.0 13/6313/2 v0.8.0
Thanh Ha [Tue, 5 Sep 2017 16:01:03 +0000 (12:01 -0400)]
Release global-jjb v0.8.0

Change-Id: Ife11163e99194bd9b43efee47f9601e3adbc5710
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoGeneralize odl rtd jobs for global-jjb 43/5643/51
Jeremy Phelps [Wed, 26 Jul 2017 21:25:05 +0000 (16:25 -0500)]
Generalize odl rtd jobs for global-jjb

Change-Id: I35bdd1629c4f1079131048a8a8f41f09949796ff
JIRA: RELENG-240
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoMerge "Add update-expected-xml call"
Anil Shashikumar Belur [Fri, 1 Sep 2017 04:10:24 +0000 (04:10 +0000)]
Merge "Add update-expected-xml call"

7 years agoMerge "Add lf-pip-install macro to handle pypi packages"
Anil Shashikumar Belur [Fri, 1 Sep 2017 04:08:48 +0000 (04:08 +0000)]
Merge "Add lf-pip-install macro to handle pypi packages"

7 years agoAdd update-expected-xml call 70/6270/1
Thanh Ha [Thu, 31 Aug 2017 21:43:32 +0000 (17:43 -0400)]
Add update-expected-xml call

Simplify the process of updating the expected-xml by adding a command to
tox.

Change-Id: I02a9f07272baab3fd0dd4407cdb0afb3ffd6e3c7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoRe-order maven jobs alphabetically 46/6246/1
Thanh Ha [Thu, 31 Aug 2017 02:54:52 +0000 (22:54 -0400)]
Re-order maven jobs alphabetically

Change-Id: Ibc3d91e21d59614fc8e0ed3402e43b986a185759
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoAdd lf-pip-install macro to handle pypi packages 44/6244/1
Thanh Ha [Wed, 30 Aug 2017 23:08:53 +0000 (19:08 -0400)]
Add lf-pip-install macro to handle pypi packages

This creates a generically reusable pip-install macro that can be used
to install pypi packages into a virtualenv. The script uses the first
package name from {pip-packages} as the name of the virtualenv it
creates.

Change-Id: Iabe4005efcb0a79af48d3c83d854f4e101414737
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoMerge "Fix tox-run.sh print statement"
Andrew Grimberg [Tue, 29 Aug 2017 16:04:13 +0000 (16:04 +0000)]
Merge "Fix tox-run.sh print statement"

7 years agoMigrate sonar job 19/6119/4
Thanh Ha [Wed, 23 Aug 2017 23:15:38 +0000 (19:15 -0400)]
Migrate sonar job

This patch uses OpenDaylight Maven Sonar job as a base and modifies it
to be more generic for use in global-jjb.

Changes from original:

- Support overriding Sonar Gerrit Triggers.
- Use maven deploy goal rather than install. End result is the same but
  deploy sends artifacts to a staging repo which we can check if needed.

Issue: RELENG-243
Change-Id: Ided78015ced8b2d4b42ed452b704e1fe65770a01
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoFix tox-run.sh print statement 59/6159/2
Thanh Ha [Fri, 25 Aug 2017 17:42:53 +0000 (13:42 -0400)]
Fix tox-run.sh print statement

Change-Id: I11721f0e292cddfd7eb05c26866ed274fa7116c8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoMerge "Update .gitignore"
Andrew Grimberg [Tue, 29 Aug 2017 15:39:57 +0000 (15:39 +0000)]
Merge "Update .gitignore"

7 years agoDont prefix PATH with hash built-in command. 59/6059/5
Anil Belur [Fri, 18 Aug 2017 23:03:11 +0000 (09:03 +1000)]
Dont prefix PATH with hash built-in command.

The changes are tested with the below job
https://jenkins.hyperledger.org/job/ci-management-packer-verify/4/console

Also supress wget verbose output.

Jira: releng-442
Change-Id: Icab1214c19e68346a84c37179a1d8c1540ffc657
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
7 years agoUpdate .gitignore 20/6120/1
Anil Belur [Thu, 24 Aug 2017 04:39:03 +0000 (14:39 +1000)]
Update .gitignore

Remove path/to/files not to ignore

Change-Id: I844cc63779c29f85f4d57e7a8bd180a30c9ad085
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
7 years agoRelease global-jjb v0.7.0 75/5875/8 v0.7.0
Thanh Ha [Fri, 11 Aug 2017 17:15:09 +0000 (13:15 -0400)]
Release global-jjb v0.7.0

Change-Id: Ia57686ff8dc8ca45048d81211463052a96ee7be9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoAdd .jjb-test to tox jjb command 46/6046/1
Thanh Ha [Fri, 18 Aug 2017 19:22:27 +0000 (15:22 -0400)]
Add .jjb-test to tox jjb command

This directory needs to be explicitly added for tests to run when
calling `tox -e jjb`.

Change-Id: I38fc6fff2bab79d03298acff768ecffbb6c25c9a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoRevert "Exclude the jjb-test directory" 42/6042/2
Thanh Ha [Fri, 18 Aug 2017 19:10:27 +0000 (15:10 -0400)]
Revert "Exclude the jjb-test directory"

No longer necessary due to I9bd85a43d76cf1dfcb02e327f097cd6551366d1b.

This reverts commit 854da1e33dd03f1569d0766dbedda123dd5af4ea.

Change-Id: Iba0569d8fb1931af6dbacbd0bec8fd03b5d4e775
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoMove tests to .jjb-test 39/6039/1
Thanh Ha [Fri, 18 Aug 2017 18:56:32 +0000 (14:56 -0400)]
Move tests to .jjb-test

JJB by default ignores paths that start with '.' so take advantage of
that and move our tests there. This way end users do not need to make
any changes to their local environment to handle the new jjb-test
directory.

Change-Id: I9bd85a43d76cf1dfcb02e327f097cd6551366d1b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoAdd all release notes up to date 36/5936/1
Thanh Ha [Wed, 16 Aug 2017 18:38:04 +0000 (14:38 -0400)]
Add all release notes up to date

Keeping a history for our release notes would be useful for folks to
understand what changes since the version they are currently using.

Change-Id: Ic0e06f742ec1609664329cdd70c809655d3c9db2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoUpdate expected-xml due to missing changes 32/5932/1
Thanh Ha [Wed, 16 Aug 2017 18:11:03 +0000 (14:11 -0400)]
Update expected-xml due to missing changes

The patch I52d05808f1f84fee37e41f68677136ee105ba48c was merged after
patch I165c8309d4aaded6830f21eea10bc5a64cad2dff so is missing the
lftools changes in it's expected-xml.

Change-Id: I6a524cf9a0ba4251e3e2b2ad1036de52c4f2a48b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoMerge "Migrate Maven Verify job template"
Andrew Grimberg [Tue, 15 Aug 2017 15:17:07 +0000 (15:17 +0000)]
Merge "Migrate Maven Verify job template"

7 years agoMerge "Verify JJB when global-jjb is updated"
Anil Shashikumar Belur [Tue, 15 Aug 2017 00:38:19 +0000 (00:38 +0000)]
Merge "Verify JJB when global-jjb is updated"

7 years agoMerge changes I7d7323b8,Ib796cfee
Anil Shashikumar Belur [Tue, 15 Aug 2017 00:38:04 +0000 (00:38 +0000)]
Merge changes I7d7323b8,Ib796cfee

* changes:
  Add default refspec reference
  Refactor packer ci templates

7 years agoMerge "Refactor jjb ci templates"
Anil Shashikumar Belur [Tue, 15 Aug 2017 00:37:02 +0000 (00:37 +0000)]
Merge "Refactor jjb ci templates"

7 years agoVerify JJB when global-jjb is updated 96/5896/2
Thanh Ha [Mon, 14 Aug 2017 15:53:27 +0000 (11:53 -0400)]
Verify JJB when global-jjb is updated

Change-Id: Ic9b9bb7f8869a4580dc0f6c69290c1c76f96ecba
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoAdd default refspec reference 83/5883/1
Thanh Ha [Fri, 11 Aug 2017 21:52:09 +0000 (17:52 -0400)]
Add default refspec reference

Missed this when we ported over the infra-parameters. Refspec should
default to refs/heads/{branch} so that a manual run of a job will always
work against the specified branch for verify jobs. Merge jobs ignore
this by defining a blank refspec.

Change-Id: I7d7323b85b61639eef0e1b4491d6ba94d934b714
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoRefactor packer ci templates 82/5882/1
Thanh Ha [Fri, 11 Aug 2017 21:26:32 +0000 (17:26 -0400)]
Refactor packer ci templates

Improve maintainability by refactoring some common code.

- Improve documentation
- Reorganize anchors for better maintainability
- Remove dependency on jjb_common anchor
  (updates to jjb code should not not break packer code)
- Do not run packer merge jobs concurrently
  (merge jobs should not be configured to run concurrently)

Change-Id: Ib796cfee44ae7b01b42ee400a04be79b243cac3b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoRefactor jjb ci templates 81/5881/1
Thanh Ha [Fri, 11 Aug 2017 21:02:46 +0000 (17:02 -0400)]
Refactor jjb ci templates

- Add Documentation
- Rearrange anchors for improved maintainability

Change-Id: If69127cb4a26942bcc3f57c5bfc407964c817393
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoMigrate Maven Verify job template 37/5837/5
Thanh Ha [Thu, 10 Aug 2017 20:54:34 +0000 (16:54 -0400)]
Migrate Maven Verify job template

This patch uses the OpenDaylight Maven Verify job as the original base
and modifies it to be more generically used in global-jjb.

Changes from original template:

- Support overriding the default Gerrit Triggers.
- Use the mvn deploy goal rather than install goal. End result is the
  same only the job copies artifacts to a staging directory in case we
  need it.
- File paths can now be overrided in case a project wants to be more
  specific on which files to build against.

Issue: RELENG-244
Change-Id: I52d05808f1f84fee37e41f68677136ee105ba48c
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoMake lftools version configurable 69/5869/7
Thanh Ha [Fri, 11 Aug 2017 16:40:04 +0000 (12:40 -0400)]
Make lftools version configurable

Add a new global default variable 'lftools-version' which can be used to
specify a version of lftools to install.

Issue: RELENG-391
Change-Id: I165c8309d4aaded6830f21eea10bc5a64cad2dff
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoAdd method to test dev versions of lftools 16/5816/7
Thanh Ha [Wed, 9 Aug 2017 20:48:11 +0000 (16:48 -0400)]
Add method to test dev versions of lftools

Can be used to test dev versions of lftools. Instructions are purposely
only documented in lftools install script such that end consumers of
global-jjb should not see this and do not need to know about it's
existance.

Issue: RELENG-362
Change-Id: I95605d578f0295edc34d6f782fb0a6b44b2f295a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoAdd Maven CLM tests 45/5845/2
Thanh Ha [Fri, 11 Aug 2017 03:35:03 +0000 (23:35 -0400)]
Add Maven CLM tests

Change-Id: Ib38677b946f6b50c165a571cecd1ab93468afd78
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoRemove lftools install step from clm job 44/5844/2
Thanh Ha [Fri, 11 Aug 2017 03:26:57 +0000 (23:26 -0400)]
Remove lftools install step from clm job

Missed this in code review. The CLM job does not do anything with
lftools so it shouldn't waste cycles installing it.

Change-Id: I6ecb765c03f10a03397af7583cc2bde5c990b348
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoReorganize template definition structure 43/5843/2
Thanh Ha [Fri, 11 Aug 2017 02:48:56 +0000 (22:48 -0400)]
Reorganize template definition structure

Having maintained this repo for awhile we can now see some common
patterns emerge from our yaml structures. Typically for a given job
there is a common builder anchor followed by 2 template sections, 1
for Gerrit and 1 for GitHub. Plus some common macros / anchors that
may be used by multiple job types.

This patch reorganizes the structure to treat a set of
anchor-gerrit-github job definitions as a single job definition. This
allows us to keep the job template close to their anchor definition so
that looking up what the main anchor does will not cause as much back
scroll.

Additionally this patch refactors the confusingly named verify_merge
anchors to a more understandable 'common' name.

Change-Id: I95eb4609274ec21b50aef059b03ac4598894f671
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoMerge "Make Merge and Verify job triggers overrideable"
Andrew Grimberg [Thu, 10 Aug 2017 20:56:36 +0000 (20:56 +0000)]
Merge "Make Merge and Verify job triggers overrideable"

7 years agoMake Merge and Verify job triggers overrideable 35/5835/2
Thanh Ha [Thu, 10 Aug 2017 20:32:01 +0000 (16:32 -0400)]
Make Merge and Verify job triggers overrideable

This makes merge and verify jobs that use Gerrit Trigger be able to
override the default trigger defined by the job. This is useful in cases
where a project may have topic branches that they do not wish to
configure their build rules differently. For example in OpenDaylight the
NetVirt project has a need for topic branches but they  don't want to
run testing against every patch submission in the topic branches and
would prefer to run them manually. This patch would allow them to
reconfigure the triggers in that case for the topic branch.

Based on a POC from:
https://git.opendaylight.org/gerrit/61385

Change-Id: Idf1d301572bb33f5b540f1011658cdd3f3fc69f5
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoFix python-tox incorrect default stream setting 23/5823/5
Thanh Ha [Thu, 10 Aug 2017 03:43:41 +0000 (23:43 -0400)]
Fix python-tox incorrect default stream setting

Change-Id: Ief29e966267efac9634e51bec3c201c3ee4351c9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoRefactor tests into jjb-tests directory 22/5822/5
Thanh Ha [Wed, 9 Aug 2017 23:01:52 +0000 (19:01 -0400)]
Refactor tests into jjb-tests directory

Issue: RELENG-250
Change-Id: Idc2ddc9c28a8ab835b3558f4076305053ee43235
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoExclude the jjb-test directory 21/5821/2
Thanh Ha [Wed, 9 Aug 2017 23:24:01 +0000 (19:24 -0400)]
Exclude the jjb-test directory

When running jenkins-jobs we should exclude the jjb-test directory. That
way global-jjb can maintain tests in that directory while not affecting
users of global-jjb's tests jobs from picking up the tests.

Change-Id: I5101e4627291817d5cf9538f85a9f3db3e6d7b28
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoMerge "Generalize deploy maven-file builder"
Thanh Ha [Thu, 10 Aug 2017 03:17:12 +0000 (03:17 +0000)]
Merge "Generalize deploy maven-file builder"

7 years agoGeneralize deploy maven-file builder 53/5753/6
Anil Belur [Sun, 6 Aug 2017 23:59:48 +0000 (09:59 +1000)]
Generalize deploy maven-file builder

The builder hard codes the URL path which makes the builder
limit deploy capabilites only to external nexus repos and not
local file systems.

The change fixes the above issue by allowing the calling job
to define the repository path.
ex:

    "${NEXUS_URL}/content/repositories/$REPO_ID"
    "file://$WORKSPACE/m2repo"

Renamed variables:
'files-dir' to 'upload-files-dir'
'nexus-url-path' to 'maven-repo-url'
'$NEXUS_URL_PATH' to '$MAVEN_REPO_URL'

Change-Id: If0beb7f6dff76c18988c9a75b52c80358f16dff6
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
7 years agoAdd license header test for shell scripts 34/5734/2
Thanh Ha [Thu, 3 Aug 2017 21:23:32 +0000 (17:23 -0400)]
Add license header test for shell scripts

Add a test to check license headers exist in shell scripts.

Change-Id: I4f6f9874f79d55f5ab52b17a4dafa9d836f90d34
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoAdd timer to packer merge job to build new images 25/5725/1 v0.6.0
Anil Belur [Thu, 3 Aug 2017 02:20:05 +0000 (12:20 +1000)]
Add timer to packer merge job to build new images

Change-Id: I5d3d76ba4350d71e051a3f1642060ebb0bd0aa2d
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
7 years agoMerge "Make packer-cloud-env variable configurable"
Thanh Ha [Wed, 2 Aug 2017 18:58:50 +0000 (18:58 +0000)]
Merge "Make packer-cloud-env variable configurable"

7 years agoMake packer-cloud-env variable configurable 99/5699/3
Anil Belur [Mon, 31 Jul 2017 05:30:30 +0000 (15:30 +1000)]
Make packer-cloud-env variable configurable

The packer-cloud-env can have a default in job templates but we should
allow it to be configurable in case a downstream consumer names their
configuration file differently.

Jira: releng-341
Change-Id: Ifc89dec39d3b93b75df58d8d37c7299a9dccf29d
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
7 years agopacker should validate only .json files 06/5706/2
Anil Belur [Tue, 1 Aug 2017 05:39:55 +0000 (15:39 +1000)]
packer should validate only .json files

Some of the projects could have README or .txt file
which should be excluded, without which packer validate
would fail.

Change-Id: I6cc388a851ae2731a113d47e3719da7edf519c63
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
7 years agoMerge "Rename settings-file to mvn-settings"
Thanh Ha [Wed, 2 Aug 2017 13:49:54 +0000 (13:49 +0000)]
Merge "Rename settings-file to mvn-settings"

7 years agoRename settings-file to mvn-settings 16/5716/1
Thanh Ha [Wed, 2 Aug 2017 13:42:40 +0000 (09:42 -0400)]
Rename settings-file to mvn-settings

I7970aae80a807e359dd34c52353007f13642fb01 did not rename the test
settings and caused some build failures.

Issue: RELENG-345
Change-Id: Ief9b45a000eb43600531a9ac923d12b4e4aa8c91
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoMerge "Pass common maven options to deploy file builder"
Thanh Ha [Fri, 28 Jul 2017 01:00:31 +0000 (01:00 +0000)]
Merge "Pass common maven options to deploy file builder"

7 years agoRename settings-file to mvn-settings in templates 37/5637/3 v0.5.5
Thanh Ha [Wed, 26 Jul 2017 15:30:57 +0000 (11:30 -0400)]
Rename settings-file to mvn-settings in templates

For Maven Templates we should be using mvn-* when referring to settings
files (and any other variables provided by user related to mvn cli).

Issue: RELENG-345
Change-Id: I7970aae80a807e359dd34c52353007f13642fb01
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoBuild CLM weekly on Saturdays 36/5636/1 v0.5.4
Thanh Ha [Wed, 26 Jul 2017 15:08:24 +0000 (11:08 -0400)]
Build CLM weekly on Saturdays

Change-Id: I92f7bf21f66d48f2f073a09e444f16b909d22c27
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoFix GitHub CLM job-group 35/5635/1
Thanh Ha [Wed, 26 Jul 2017 14:43:36 +0000 (10:43 -0400)]
Fix GitHub CLM job-group

Incorrectly referred to a Gerrit job.

Change-Id: Id0a48befb5631ea4b192a81e15b0c57659e1b65e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoAdd missing maven parameters to CLM job 34/5634/1
Thanh Ha [Wed, 26 Jul 2017 03:32:36 +0000 (23:32 -0400)]
Add missing maven parameters to CLM job

lf-infra-maven-parameters expects some parameters to be passed to it.
Add the missing ones to the CLM job and set sensible defaults.

Change-Id: I7d5fe5e2c306dd988ba023c78b86ce32db1d30b2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoAdd missing stream declaration in clm job 32/5632/1
Thanh Ha [Wed, 26 Jul 2017 03:06:15 +0000 (23:06 -0400)]
Add missing stream declaration in clm job

Change-Id: Ie2e0540f79c10b0960405056d697a305c5520d27
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoFix docker login for custom registry 29/5629/1 v0.5.3
Andrew Grimberg [Tue, 25 Jul 2017 20:30:23 +0000 (13:30 -0700)]
Fix docker login for custom registry

Custom registry detection fails to be detected if the corresponding
Jenkins environment variable is not properly referenced.

Change-Id: I618836ca4e9364045b01c7d710ac00f359a1e025
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
7 years agoSet sensible default timeouts for packer jobs 25/5625/2
Anil Belur [Tue, 25 Jul 2017 03:01:23 +0000 (13:01 +1000)]
Set sensible default timeouts for packer jobs

Set the build timeout for verify job to 10 minutes and merge job
to 45 minutes based on evaluation of average build timeouts in
OpenDaylight's packer job runs.

https://jenkins.opendaylight.org/releng/view/packer

Change-Id: If97eab52f23952fd141888a7618b4bd8a502dd70
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoRemove unneeded JJB variable from packer templates 26/5626/2 v0.5.2
Andrew Grimberg [Tue, 25 Jul 2017 13:22:49 +0000 (06:22 -0700)]
Remove unneeded JJB variable from packer templates

Gerrit Iaccaa906980eb077676a3dc4b9100a91ec65f8c0 /
commit 91d975d5c426a08e82d8a6bbe1e298fa82f2b7b5 removed an unneeded
variable from the packer scripts but it wasn't removed from the top
level macros.

Change-Id: I893e25429c5aab45fcabdf3853e752e2f12dcf36
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
7 years agoRemove unnecessary openstack configuration file 18/5618/2
Thanh Ha [Mon, 24 Jul 2017 22:10:08 +0000 (18:10 -0400)]
Remove unnecessary openstack configuration file

The packer jobs do not use openstack so do not require a clouds.yaml
file.

Change-Id: Iaccaa906980eb077676a3dc4b9100a91ec65f8c0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoGitHub triggers need distinct status contexts 15/5615/1 v0.5.1
Andrew Grimberg [Mon, 24 Jul 2017 20:47:15 +0000 (13:47 -0700)]
GitHub triggers need distinct status contexts

For GitHub to register each of the different jobs that are running for
verify (and merge?) we need distinct status context strings.

Change-Id: Ie240797dcb383b9f6a09319db902df9a6f46c942
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
7 years agoPass common maven options to deploy file builder 94/5594/6
Anil Belur [Wed, 19 Jul 2017 10:14:49 +0000 (20:14 +1000)]
Pass common maven options to deploy file builder

Uploading large files on nexus prints lots of unwanted
messages in the logs, therefore disable this by passing
predefiend options through $MAVEN_OPTIONS. Also pass any
parameters to maven provided through job parameters.

Change-Id: I2e11adcb26d07097fa43cc289175aaa1cda897f9
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
7 years agoAdd generic packer templates 88/5488/18 v0.5.0
Anil Belur [Mon, 3 Jul 2017 11:54:03 +0000 (21:54 +1000)]
Add generic packer templates

- Packer jobs are designed to validate packer files and build
  custom images on the cloud. The packer templates can be used for
  and LF projects which clone global-jjb submodule.
- The jobs check if packer binaries are unavailable and install packer
  on the build node. The default version of packer installed is 1.0.2
  while more recent version can be defined in the job.
- Update test templates according to new packer jobs introduced.

Change-Id: I3657621e30d402655b84973ea4b4c48fa21cea2d
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
7 years agoMerge "Cleanup Coala config"
Jamo Luhrsen [Wed, 19 Jul 2017 21:49:57 +0000 (21:49 +0000)]
Merge "Cleanup Coala config"

7 years agoCleanup Coala config 99/5599/1
Daniel Farrell [Wed, 19 Jul 2017 21:43:16 +0000 (17:43 -0400)]
Cleanup Coala config

* Use new inheritance semantics
* Add ignores so checks run faster
* Use cleaner multi-dir regexes

Change-Id: I18e0fa2b3033c658a297cd71e368f0e272b081b0
Signed-off-by: Daniel Farrell <dfarrell@redhat.com>
7 years agoAdd whitelist and admin-list params to gh trigger 31/5531/38
Jeremy Phelps [Thu, 13 Jul 2017 16:46:19 +0000 (11:46 -0500)]
Add whitelist and admin-list params to gh trigger

This enables users in the white list to trigger any job
that uses the lf-infra-github-trigger macro.
In addition you can set users in the admin-list to have
admin rights on a pr.

Change-Id: Ib69095008780e8ba3a752aaeb498e7d6f8582cb5
JIRA: RELENG-174
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoUpdate SPDX License header to new format 73/5573/2
Thanh Ha [Tue, 18 Jul 2017 17:54:16 +0000 (13:54 -0400)]
Update SPDX License header to new format

According to SPDX's website https://spdx.org/using-spdx the correct
format for specifying SPDX is now:

    # SPDX-License-Identifier: EPL-1.0

Issue: RELENG-278
Change-Id: I261e220b0b509eac12a9d2b3bbef82dd1c3a7c4c
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoValidate URLs using Jenkins env variable 36/5536/1
Jessica Wagantall [Fri, 14 Jul 2017 07:02:39 +0000 (00:02 -0700)]
Validate URLs using Jenkins env variable

Since we are adding in Jenkins a variable
JIRA_URL, we want to make the logic of this
script so that it picks up its value to look
for related URLs in the commit text rather
than using a static value.

Change-Id: Id6d1df4cfefa0d37f7f74a48d8d5149379ede3eb
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
7 years agoCheck for JIRA URLs on JIRA based projects 18/5518/6
Jessica Wagantall [Tue, 11 Jul 2017 20:40:36 +0000 (13:40 -0700)]
Check for JIRA URLs on JIRA based projects

Check if the project has a JIRA_URL defined
in Jenkins and, if so, check if there are
JIRA URLS in the commit text.
Ignore checks for projects that don't use JIRA

Change-Id: I2267aa98e1b6a9c727fb9a54426aef5c5a7131e6
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
7 years agoAdd missing content/repositories path to URL 81/5381/3
Anil Belur [Tue, 4 Jul 2017 00:47:28 +0000 (10:47 +1000)]
Add missing content/repositories path to URL

NEXUS_REPO_URL was missing the content/repositories path creating an
invalid upload path for Nexus.

Change-Id: I09f6ff06e71355aafcb90cb2d011203bfc6584c0
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
7 years agoAdd Nexus IQ CLM scan job 23/5223/9
Jessica Wagantall [Fri, 16 Jun 2017 06:29:09 +0000 (23:29 -0700)]
Add Nexus IQ CLM scan job

This template will be used to perform code scans
and report the results in nexus-iq server.
A plugin for nexus-iq will need to be installed and
configured in Jenkins before using this job.

JIRA: RELENG-176
Change-Id: I813da293728615a42b43ec785385a38237c57213
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
7 years agoMerge "Fix typo in deploy file builder loop"
Andrew Grimberg [Mon, 3 Jul 2017 16:01:01 +0000 (16:01 +0000)]
Merge "Fix typo in deploy file builder loop"

7 years agoMerge "Expose stream as a env variable" v0.4.2
Jamo Luhrsen [Fri, 30 Jun 2017 02:43:40 +0000 (02:43 +0000)]
Merge "Expose stream as a env variable"

7 years agogrep: DO NOT return exit code 1 on no match 63/5363/1
Thanh Ha [Fri, 30 Jun 2017 02:02:00 +0000 (22:02 -0400)]
grep: DO NOT return exit code 1 on no match

This is causing build failures in OpenDaylight for our patches :(

Change-Id: I1e5c90a0507756103e602a4fd298164b127189ec
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoExpose stream as a env variable 62/5362/3
Thanh Ha [Thu, 29 Jun 2017 22:55:53 +0000 (18:55 -0400)]
Expose stream as a env variable

Shell scripts might be interested in using STREAM so let's expose it via
env variable so that they can pick it up. Default all job types to
stream=master.

Change-Id: Ia61d00f88649ef9c01cd6a15cbc248bb4fae1106
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoFix maven-release job choosing strategy 50/5350/1 v0.4.1
Thanh Ha [Thu, 29 Jun 2017 15:38:36 +0000 (11:38 -0400)]
Fix maven-release job choosing strategy

The choosing strategy should be default otherwise the wrong commit is
checked out.

Change-Id: I398e51440c9fffd58538fea5f12665685c97c725
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoFix typo in deploy file builder loop 48/5348/2
Anil Belur [Thu, 29 Jun 2017 09:36:58 +0000 (19:36 +1000)]
Fix typo in deploy file builder loop

Change-Id: I22400fdd7a15897f3648881c1ab74868248113ee
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
7 years agoMerge "Add maven-deploy-file builder for packaging jobs" v0.4.0
Anil Shashikumar Belur [Wed, 28 Jun 2017 07:22:22 +0000 (07:22 +0000)]
Merge "Add maven-deploy-file builder for packaging jobs"

7 years agoAdd maven-deploy-file builder for packaging jobs 00/5200/24
Anil Belur [Wed, 14 Jun 2017 08:12:40 +0000 (18:12 +1000)]
Add maven-deploy-file builder for packaging jobs

Change-Id: I243d13c257756f653b702c57e5d8c700411c53e9
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
7 years agoMerge "Add a check before running the sysstat script"
Jessica Wagantall [Tue, 27 Jun 2017 18:58:20 +0000 (18:58 +0000)]
Merge "Add a check before running the sysstat script"

7 years agoAdd write-good plugin to lint Markdown 20/5320/1
Thanh Ha [Mon, 26 Jun 2017 22:01:30 +0000 (18:01 -0400)]
Add write-good plugin to lint Markdown

Change-Id: Ib96d2abbfff893ef70d084f3b55ec656a4b374be
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoAdd Markdown linting 17/5317/1
Thanh Ha [Mon, 26 Jun 2017 21:25:07 +0000 (17:25 -0400)]
Add Markdown linting

Change-Id: I6232763c39c2369e5352033ffbffc831aa5e148e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoReorganize .coafile 16/5316/1
Thanh Ha [Mon, 26 Jun 2017 21:14:35 +0000 (17:14 -0400)]
Reorganize .coafile

- Alphabetical ordering
- More specific ignores

Change-Id: I1663a044be845ee1edc65e65192074548212ccae
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoAdd jenkins-admin scripts to global-jjb 15/5315/1
Thanh Ha [Mon, 26 Jun 2017 21:10:13 +0000 (17:10 -0400)]
Add jenkins-admin scripts to global-jjb

Change-Id: I3419d1af343d04dc85f777b61d820f7c4f27276e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoMake global-jjb verify JJB files 62/5262/5
Thanh Ha [Tue, 20 Jun 2017 19:49:12 +0000 (15:49 -0400)]
Make global-jjb verify JJB files

This adds to tox.ini some code to make it actually test the global-jjb
yaml files by using a test.yaml that is copied from test.template by
tox. This is so that downstream projects do not pull in the fake job
data.

Also archives the test job output to job-configs on the log server.

Change-Id: Ie0dcc99d2df9ac90410be4a5b03d6d409027f34a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoMerge "Add an appendix to the README"
Andrew Grimberg [Thu, 22 Jun 2017 13:33:53 +0000 (13:33 +0000)]
Merge "Add an appendix to the README"

7 years agoMake sure docker-login is include-raw-escape 74/5274/1
Andrew Grimberg [Wed, 21 Jun 2017 20:00:58 +0000 (13:00 -0700)]
Make sure docker-login is include-raw-escape

The docker login script has several curly brace variables that it needs.
Using include-raw instead of include-raw-escape causes JJB to break.

Change-Id: I345e9256f46eef2bf1fd245b985ebf5ecad77616
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
7 years agoAdd an appendix to the README 69/5269/1
Thanh Ha [Wed, 21 Jun 2017 15:45:55 +0000 (11:45 -0400)]
Add an appendix to the README

Explain the minimum ShellCheck version that is required if ShellCheck is
being used by a ci-management project to lint their JJB directory.

Change-Id: I5b08f6d29fee189b9cbc7714f2dc0ba646c5e917
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoMerge "lf-ci-jobs: Add git-validate-jira-urls.sh"
Thanh Ha [Mon, 19 Jun 2017 18:36:55 +0000 (18:36 +0000)]
Merge "lf-ci-jobs: Add git-validate-jira-urls.sh"

7 years agolf-ci-jobs: Add git-validate-jira-urls.sh 94/5194/7
Jessica Wagantall [Tue, 13 Jun 2017 17:43:39 +0000 (10:43 -0700)]
lf-ci-jobs: Add git-validate-jira-urls.sh

Across projects, we have commit messages being
merged with jira URLs instead of the accepted
format: Issue: <JIRAKEY>-<ISSUE#>
Jira URLs break the its-jira plugin making the
link unaccessible from gerrit to the jira issue.

Change-Id: Iecc82b34275d640549f974312bfba65fe3fa56f7
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
7 years agoAdd lf-infra-docker-login macro and script 26/5226/1
Andrew Grimberg [Fri, 16 Jun 2017 20:39:26 +0000 (13:39 -0700)]
Add lf-infra-docker-login macro and script

Provide a way to utilize a maven settings file to store the login
credentials to our custom hosted docker registries as well as for
DockerHub (docker.io)

Change-Id: Ifb519b09afecf29c44bedc75a3a81a9b174a978a
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
7 years agoAdd a check before running the sysstat script 22/5222/1
Anil Belur [Fri, 16 Jun 2017 02:51:33 +0000 (12:51 +1000)]
Add a check before running the sysstat script

Systat is not enabled by default on Ubuntu, which requires
this check for projects which have not configured sysstat in
their provisioning scripts.

Change-Id: Idcde47ab389ff2b0bb2f5d43ee716a4a5c926fd0
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
7 years agoFully render GitHub SCM clone URLs 95/5195/1
Andrew Grimberg [Tue, 13 Jun 2017 17:48:51 +0000 (10:48 -0700)]
Fully render GitHub SCM clone URLs

After additional testing it has been proven that the GitHub plugin
itself, which is used for the final merge (push) operation will only
register as a handler if the SCM URL is fully realized without any
variables.

Issue: RELENG-191
Change-Id: I3492dac96e85a3d81819fd2ba810629f179d0c04
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
7 years agoMake GitHub JJB verify jobs actually verify 93/5193/1 v0.3.5
Andrew Grimberg [Tue, 13 Jun 2017 17:03:35 +0000 (10:03 -0700)]
Make GitHub JJB verify jobs actually verify

The JJB verify jobs were found to be doing merge instead of just verify.
This means that every JJB PR was actually is causing Jenkins changes.

Issue: RELENG-190
Change-Id: If7984725e0319f558738dce7735d688df2cc5092
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
7 years agoFix shellcheck errors SC2086 89/5189/1
Anil Belur [Tue, 13 Jun 2017 03:03:32 +0000 (13:03 +1000)]
Fix shellcheck errors SC2086

SC2086: Double quote to prevent globbing and word splitting.

Change-Id: Ib9fd89a72d3c3f4788f9b1c02a882ace0250bd09
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
7 years agoInstall virtualenvs into /tmp/v 88/5188/1 v0.3.4
Thanh Ha [Tue, 13 Jun 2017 02:34:30 +0000 (22:34 -0400)]
Install virtualenvs into /tmp/v

The kernel has a limit on how long a shebang can be and causes build
issues when we store virtualenvs in $WORKSPACE for jobs that have very
long names. Move virtualenv installs into /tmp/v will solve this issue
for good. Also no need for workarounds like `$PYTHON -m pip` anymore.

Change-Id: I72be5f42c2196e88f22262688e3425f6a83e807c
Jira: RELENG-156
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoFix typo in WORKSPACE name 86/5186/1 v0.3.3
Thanh Ha [Mon, 12 Jun 2017 23:23:04 +0000 (19:23 -0400)]
Fix typo in WORKSPACE name

Change-Id: I27608a7db9ec47b4bd0c95226a08d5346f236d8b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoJenkins cannot handle ${} vars in GitHub project 81/5181/4
Andrew Grimberg [Mon, 12 Jun 2017 19:51:37 +0000 (12:51 -0700)]
Jenkins cannot handle ${} vars in GitHub project

The GitHub project requires a rendered URL to the project. It does not
apparently properly render variables that are {} enclosed which means
that any vars needs to fully path separated properly

Issue: RELENG-188
Change-Id: Ie616752899453a0e614f1b6500eb2886e796d7a9
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
7 years agoFix alias linking for lf_tox_verify 80/5180/1 v0.3.2
Thanh Ha [Mon, 12 Jun 2017 18:49:07 +0000 (14:49 -0400)]
Fix alias linking for lf_tox_verify

Change-Id: I1c80c87fcb591af1d00f0d480615e58da0a68ab7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
7 years agoGitHub JJB verifies should be concurrent 79/5179/2
Andrew Grimberg [Mon, 12 Jun 2017 18:42:55 +0000 (11:42 -0700)]
GitHub JJB verifies should be concurrent

While doing some work pushing multiple different patches to a GitHub
project it was discovered that the GitHub JJB verify jobs were not
configured for concurrency.

Change-Id: I48af8b1722a8223916324f49151fec88aa8e32b3
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
7 years agoMerge "Add support for GitHub to Python templates" v0.3.0
Andrew Grimberg [Mon, 12 Jun 2017 18:03:12 +0000 (18:03 +0000)]
Merge "Add support for GitHub to Python templates"