releng/common-packer.git
4 years agoAdd support for Ubuntu 20.04 and newer 83/65083/1
Thanh Ha [Thu, 13 Aug 2020 22:17:38 +0000 (18:17 -0400)]
Add support for Ubuntu 20.04 and newer

Ubuntu 20.04 no longer has python-* packages and instead the packages
are now more explicitly either python2-* or python3-* with packages
named python-is-python[23] to set the default. This patch when run
on Ubuntu systems will check if python-is-python3 is available and
installs that as the system python version, otherwise installs
python-minimal.

Also adjust the comments since they are no longer accurate or in the
right places.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I830e0250ddbee395fdeddd78fcc7824f7855d75d

4 years agoUpdate Ubuntu 18.04 base image 15/64815/1 v0.6.2
Aric Gardner [Thu, 30 Jul 2020 03:10:00 +0000 (13:10 +1000)]
Update Ubuntu 18.04 base image

Change-Id: I4b40ff1c84cfa2f64235b3a8132e6d43baea1f3a
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoUpdate Ubuntu base image to 2020-07-28 snapshot 97/64797/4 v0.6.1
Aric Gardner [Tue, 28 Jul 2020 19:22:24 +0000 (15:22 -0400)]
Update Ubuntu base image to 2020-07-28 snapshot

base image was uploaded in qcow2 format.
this casued timeout in packer builds.
new image is in raw format.

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I0151e6b8677625e4477d93faa67cb1abdf84515c

4 years agoAdd release notes for sigul client on CentOS 7 21/64621/1 v0.6.0
Eric Ball [Thu, 23 Jul 2020 18:48:55 +0000 (11:48 -0700)]
Add release notes for sigul client on CentOS 7

Issue: RELENG-3077
Change-Id: I3314b0770a0f0280ee162bbb646f95312cff6196
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
4 years agoAdd install of Sigul client on CentOS 7 10/64610/1
Eric Ball [Thu, 23 Jul 2020 01:27:12 +0000 (18:27 -0700)]
Add install of Sigul client on CentOS 7

Our current signing scripts download and install Sigul only when
executing the signing scripts. However, the servers hosting this
package are not reliable, and we should see significant performance
improvements by having the package pre-loaded on CentOS 7 builders
(the only builders capable of using Sigul).

Issue: RELENG-3077
Change-Id: I35ac34a3619f4018331f280745f4ff3d5c864fa9
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
4 years agoUpdate the CentOS 7 base image to the latest 77/64277/2
Anil Belur [Tue, 16 Jun 2020 00:37:09 +0000 (10:37 +1000)]
Update the CentOS 7 base image to the latest

https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-2003.raw.tar.gz

Issue: IT-19958
Change-Id: Ib6f84fc36d6644c286540abd11c8446bea78497f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoFix provisioner to install Git 2 from IUS 61/64261/2 v0.5.2
Thanh Ha [Fri, 12 Jun 2020 00:37:12 +0000 (20:37 -0400)]
Fix provisioner to install Git 2 from IUS

While patch Ibace4cdf89d0615a2e5117f9ea3ca50730d5c93d fixed the setup
for the IUS repo. It failed to ensure that Git 2 is installed from the
IUS repo.

In the refactored IUS repo git2u no longer exists and instead the
latest Git package appears to be git224 so let's update to that
version.

Also class cast ansible_distribution_major_version|int as this variable
normally returns a type AnsibleUnsafeText which cannot be compared to
an int type.

Change-Id: I1cd815fb0440be996ba66c3a48eab9d6da02f4bb
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
4 years agoMerge "Run docs and docs-linkcheck in verify"
Andrew Grimberg [Sat, 6 Jun 2020 14:32:43 +0000 (14:32 +0000)]
Merge "Run docs and docs-linkcheck in verify"

4 years agoMerge "Add .editorconfig file"
Andrew Grimberg [Sat, 6 Jun 2020 14:26:48 +0000 (14:26 +0000)]
Merge "Add .editorconfig file"

4 years agoMerge "Print git-diff when pre-commit fails"
Andrew Grimberg [Sat, 6 Jun 2020 14:25:54 +0000 (14:25 +0000)]
Merge "Print git-diff when pre-commit fails"

4 years agoRun docs and docs-linkcheck in verify 30/64230/1
Thanh Ha [Sat, 6 Jun 2020 14:21:06 +0000 (10:21 -0400)]
Run docs and docs-linkcheck in verify

Not sure why we weren't running docs build tests as part of the
common-packer verify build but this patch updates tox to enable
those build tests.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I7d28bdf8ace6946bcf1b6a2d08bc1408875fce96

4 years agoAdd .editorconfig file 25/64225/1
Thanh Ha [Sat, 6 Jun 2020 14:04:39 +0000 (10:04 -0400)]
Add .editorconfig file

EditorConfig is a universal editor configuration file supported by many
IDEs and Editors (although some might require installing a plugin) to
share consistent editor configuration with developers using different
editors.

Ref: https://editorconfig.org/
Change-Id: I59a4a57298e295dcfe7d21591a8c0fb501b0cef6
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
4 years agoPrint git-diff when pre-commit fails 24/64224/1
Thanh Ha [Sat, 6 Jun 2020 14:01:53 +0000 (10:01 -0400)]
Print git-diff when pre-commit fails

When pre-commit fails and automatically fixes issues for us in Jenkins
we are not able to see what pre-commit automatically fixed. Adding
--show-diff-on-failure makes pre-commit perform a git-diff immediately
after failing to ensure more useful output is printed to inform the
contributor what changes are needed to pass the check.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: Ic2ad49fd6222bee0341c4826e6ccc463d05fda5f

4 years agoUpdate pre-commit hooks 2020-06 23/64223/1
Thanh Ha [Sat, 6 Jun 2020 14:00:27 +0000 (10:00 -0400)]
Update pre-commit hooks 2020-06

It's been awhile since we last updated the pre-commit hooks so let's
catch up with the world.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: Ibf5e945a2bb6324c73ea5b2d7a683bc0f926384b

5 years agoFix PowerTools enablement and IUS install 42/64042/1 v0.5.1
Andrew Grimberg [Wed, 20 May 2020 20:08:50 +0000 (13:08 -0700)]
Fix PowerTools enablement and IUS install

* Use correct syntax for verifying the major version(s) we
  should be setting IUS up on
* Fix the fact we're checking to know if should be enabling
  the PowerTools repo

Change-Id: I3340d5aae147a2c5881d973442fba4e9543d8c1e
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoFix version comparison for PowerTools installation 05/63705/1
Andrew Grimberg [Fri, 17 Apr 2020 21:42:42 +0000 (14:42 -0700)]
Fix version comparison for PowerTools installation

Apparently ansible_distribution_major_version returns a string and
cannot therefore properly be tested in an integer based test. Use the
version() function instead.

https://docs.ansible.com/ansible/latest/user_guide/playbooks_tests.html#version-comparison

Change-Id: I7abb42f94b1543aed5fe3ca548184ac1082612d7
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoSupport CentOS 8 builds 31/63531/3 v0.5.0
Andrew Grimberg [Fri, 27 Mar 2020 22:14:36 +0000 (15:14 -0700)]
Support CentOS 8 builds

Changes needed to run the packer against CentOS 8 systems

Issue: RELENG-2823
Change-Id: I7748d98150630bc5327d407eab5421a2ec21e7bf
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoAdd Centos 8 base image 00/63500/2
Lott, Christopher (cl778h) [Wed, 25 Mar 2020 15:06:45 +0000 (11:06 -0400)]
Add Centos 8 base image

Issue-ID: RELENG-2823
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
Change-Id: I6c00ef45e269028f60a06f7f4aa9c38729ad952a

5 years agoUpdate ARM base images 32/63432/1
Jessica Wagantall [Tue, 17 Mar 2020 20:20:04 +0000 (13:20 -0700)]
Update ARM base images

Update the latest (tested) baseimages for ARM
Ubuntu 16.40 and Ubuntu 18.04

Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: Ide6115ef7f9d3e42935011aaf37872adc759e589

5 years agoInstall netselect depending on the Ubuntu arch 47/63147/12 v0.4.4
Bengt Thuree [Wed, 19 Feb 2020 01:01:27 +0000 (12:01 +1100)]
Install netselect depending on the Ubuntu arch

Change-Id: Ie93b7b5d510ff1e067db4fcfc1020f9ae93dadcf
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAllow the jenkins user to sudo as any user 25/63125/1
Thanh Ha [Fri, 14 Feb 2020 03:12:17 +0000 (22:12 -0500)]
Allow the jenkins user to sudo as any user

Previously the Jenkins user could only sudo and run commands as root.
This change allows the Jenkins user to `sudo -u another_user` without
providing a password. This is useful for test suites that need to
perform commands as different user accounts and not just root.

Change-Id: Ib7864e8b9370ae7f1a2a8bec82cf609e7dcccead
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
5 years agoMinor Cleanup 97/62997/1
Tim Johnson [Thu, 30 Jan 2020 16:12:45 +0000 (08:12 -0800)]
Minor Cleanup

Change-Id: Ia826901dbbc1a3fd6f5df4af22418015b9dbc78c
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoDo not fail the build when a mirror is not found 81/62981/1 v0.4.3
Anil Belur [Wed, 29 Jan 2020 10:09:38 +0000 (20:09 +1000)]
Do not fail the build when a mirror is not found

Forgot to include this check in the change 4c9db8e41b3b546a52.
netselect checks for the mirrors from the list and when one
or more mirrors are not available returns the below error
causing the build to fail.

error:
netselect: unknown host gpl.savoirfairelinux.net/pub/mirrors/ubuntu/

Issue: RELENG-2642
Change-Id: I4351a40be996092febe4efe956b18a9a575c994e
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Use netselect to choose a package mirror" v0.4.2
Anil Belur [Wed, 29 Jan 2020 01:27:01 +0000 (01:27 +0000)]
Merge "Use netselect to choose a package mirror"

5 years agoMerge "Revert "Use apt-get to install python-minimal""
Anil Belur [Wed, 29 Jan 2020 01:26:42 +0000 (01:26 +0000)]
Merge "Revert "Use apt-get to install python-minimal""

5 years agoAdd GitLint which replaces GitCommitBear 76/62976/2
Thanh Ha [Tue, 28 Jan 2020 15:39:05 +0000 (10:39 -0500)]
Add GitLint which replaces GitCommitBear

The GitCommitBear was removed via
I8d782822ca87add9108a044fd6e19b1c23686717 this replaces it with
GitLint which does similar linting for git commit messages.

Issue: RELENG-2642
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I6e47db45a6c5c519ee795adf15c4243fb0bfb6dd

5 years agoUse netselect to choose a package mirror 68/62968/1
Anil Belur [Tue, 28 Jan 2020 02:56:48 +0000 (12:56 +1000)]
Use netselect to choose a package mirror

Use netselect to choose a package mirror to install python-minimal in a
reliable manner.

apt{-get} does not refresh the package mirrors (for packer builds run
within Jenkins), therefore fails with "E: Unable to locate package
python-minimal" while installing python-minimal.

Issue: RELENG-2548
Change-Id: Iab90212657816a9b66db07b356c32ae7e90272d8
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoRevert "Use apt-get to install python-minimal" 67/62967/1
Anil Belur [Tue, 28 Jan 2020 03:00:51 +0000 (13:00 +1000)]
Revert "Use apt-get to install python-minimal"

This reverts commit 9eca5443d48cde7a5f16e16cb16d89e3cd3ab66c.

This only works intermittently, therefore reverting the commit.

Change-Id: If7a2f9850e1054e251e596d104e6c4ba9eb52338
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUse apt-get to install python-minimal 30/62930/1 v0.4.1
Anil Belur [Fri, 24 Jan 2020 06:54:13 +0000 (16:54 +1000)]
Use apt-get to install python-minimal

apt does not refresh the package mirrors for the jobs
run within the Jenkins, and fails to install python-minimal.

With apt issue is reproducable:
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/builder-packer-merge-ubuntu-16.04-docker/74

With apt-get as a workaround resolves the packages with the mirrors sussessfully:
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/builder-packer-merge-ubuntu-16.04-docker/73

Issue: RELENG-2548
Change-Id: I3436fa250bff5b90e2b4c390a97c0b989b52d7af
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd pre-commit hook for shellcheck 29/62829/3
Anil Belur [Thu, 16 Jan 2020 08:09:24 +0000 (18:09 +1000)]
Add pre-commit hook for shellcheck

Use pre-commit to run the shellcheck hook rather than coala as
we want to eventually phase out coala.

Issue: RELENG-2642
Change-Id: Ie06bbb0a06ba517bcc378480ea39320da00403c6
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpgrade pre-commit hooks version to the latest 31/62831/2
Anil Belur [Thu, 16 Jan 2020 02:18:55 +0000 (12:18 +1000)]
Upgrade pre-commit hooks version to the latest

Change-Id: I10dade2daa3b0379ff402875c1333d9457dd22c2
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoResolve shellcheck SC2196: egrep is non-standard 30/62830/1
Anil Belur [Thu, 16 Jan 2020 02:13:53 +0000 (12:13 +1000)]
Resolve shellcheck SC2196: egrep is non-standard

SC2196: egrep is non-standard and deprecated. Use grep -E instead.

Change-Id: I25f083bca244580ec01a8a1e000b13cc5c3ba2e5
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Update Ubuntu 18.04 arm64 base image to the latest" v0.4.0
Anil Belur [Tue, 7 Jan 2020 10:42:28 +0000 (10:42 +0000)]
Merge "Update Ubuntu 18.04 arm64 base image to the latest"

5 years agoMerge "Update Ubuntu 16.04 amd64 base image to latest"
Anil Belur [Tue, 7 Jan 2020 10:42:12 +0000 (10:42 +0000)]
Merge "Update Ubuntu 16.04 amd64 base image to latest"

5 years agoMerge "Update Ubuntu 16.04 arm64 base image to latest"
Anil Belur [Tue, 7 Jan 2020 10:42:08 +0000 (10:42 +0000)]
Merge "Update Ubuntu 16.04 arm64 base image to latest"

5 years agoUpdate Ubuntu 16.04 amd64 base image to latest 42/62642/3
Anil Belur [Mon, 16 Dec 2019 22:40:04 +0000 (08:40 +1000)]
Update Ubuntu 16.04 amd64 base image to latest

Change-Id: Iec964db3b3652379641e3336bfe3df18ee796300
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate Ubuntu 16.04 arm64 base image to latest 41/62641/2
Anil Belur [Mon, 16 Dec 2019 22:01:53 +0000 (08:01 +1000)]
Update Ubuntu 16.04 arm64 base image to latest

Issue: RELENG-2548
Change-Id: I88b8c6fa37dba294c7caec09d862963f77c146ef
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate Ubuntu 18.04 amd64 base image to the latest 40/62640/2
Anil Belur [Mon, 16 Dec 2019 22:14:08 +0000 (08:14 +1000)]
Update Ubuntu 18.04 amd64 base image to the latest

Issue: RELENG-2548
Change-Id: Iac6f3a9bc56172ad607ead417b4f538724e5fcc4
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate Ubuntu 18.04 arm64 base image to the latest 39/62639/2
Anil Belur [Mon, 16 Dec 2019 22:17:29 +0000 (08:17 +1000)]
Update Ubuntu 18.04 arm64 base image to the latest

Issue: RELENG-2548
Change-Id: I5d520ef30747f2e842c64057a7060a68782ebb80
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoRemove coala from common-packer 79/62679/4
Anil Belur [Thu, 19 Dec 2019 22:18:47 +0000 (08:18 +1000)]
Remove coala from common-packer

Move away from coala since that project refuses to produce any
new release and the version we're using is quite dated at this
point.

Coala does not seem to be working well with prettier pre-commit
hook in place, since prettier is auto-formating the yaml code
to make the code terse which fails the tox verify job.

Issue: RELENG-2641
Change-Id: I8d782822ca87add9108a044fd6e19b1c23686717
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoChange IUS configuration 44/62244/1
Andrew Grimberg [Wed, 13 Nov 2019 15:31:55 +0000 (07:31 -0800)]
Change IUS configuration

While trying to troubleshoot why a recent CentOS7 system was failing to
get past the installation of yum-plugin-replace from IUS it was
discovered that IUS has changed up the keys that they are signing with.

Along with this change are changes to their official distribution point
according to the standard release package.

While we don't actually install the release package, we do have reason
to configure the rest of the repository correctly, as such we need to
follow the changes that were made back in May of 2019!

Change-Id: Ibace4cdf89d0615a2e5117f9ea3ca50730d5c93d
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoNeed become true for yum_repository 93/61793/1
Aric Gardner [Thu, 19 Sep 2019 22:37:11 +0000 (18:37 -0400)]
Need become true for yum_repository

Don't set priority

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I5d0449dd983acf0fee2b69b8f2994fa709ddf3a6

5 years agoAnsible yum_repository has includepkgs option 91/61791/1
Aric Gardner [Thu, 19 Sep 2019 21:40:28 +0000 (17:40 -0400)]
Ansible yum_repository has includepkgs option

Limit IUS repo ro git2u and yum replace plugin
Something about the order of the yum package install
causes yum to try to pull down python36u

This can be reproduced in docker with
https://gist.github.com/Aricg/73a872fcaaa5eae1302d71eb2237d208

ISSUE: IT-17631
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Id968757b41fe6d99f9a22bcbaf98be74b5d05e3e

5 years agoOnly use IUS for git2u 84/61784/4
Aric Gardner [Thu, 19 Sep 2019 18:40:21 +0000 (14:40 -0400)]
Only use IUS for git2u

Limit IUS repo ro git2u and yum replace plugin
Something about the order of the yum package install
causes yum to try to pull down python36u

This can be reproduced in docker with
https://gist.github.com/Aricg/73a872fcaaa5eae1302d71eb2237d208

ISSUE: IT-17631
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I65a8b388340e28ceea45d6b3ad33bcc84f31e7ac

5 years agoConfigure basepython to python3 10/16710/1
Thanh Ha [Tue, 27 Aug 2019 19:18:34 +0000 (15:18 -0400)]
Configure basepython to python3

More and more libraries that we are pulling in require Python 3 so
to avoid having to play the pin dependencies whack-a-mole game let's
bump or base python where possible to python3.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I1cfd7eebb06d0651a1fcefefb81f900aecdef057

5 years agoMerge changes from topic "update-U1804-arm64-base-image"
Andrew Grimberg [Fri, 12 Jul 2019 14:59:20 +0000 (14:59 +0000)]
Merge changes from topic "update-U1804-arm64-base-image"

* changes:
  Fix Ubuntu 18.04 version in var file
  Update Ubuntu 18.04 arm64 base image to latest

5 years agoMerge changes from topic "prettier"
Andrew Grimberg [Fri, 12 Jul 2019 14:56:07 +0000 (14:56 +0000)]
Merge changes from topic "prettier"

* changes:
  Use prettier for auto formatting of JSON/YAML
  Reformat JSON and YAML files with prettier

5 years agoMerge "Update CentOS 7 base image to latest"
Andrew Grimberg [Fri, 12 Jul 2019 14:54:31 +0000 (14:54 +0000)]
Merge "Update CentOS 7 base image to latest"

5 years agoMerge "Update Ubuntu 18.04 amd64 base images"
Andrew Grimberg [Fri, 12 Jul 2019 14:54:20 +0000 (14:54 +0000)]
Merge "Update Ubuntu 18.04 amd64 base images"

5 years agoMerge "Update Ubuntu 16.04 base images to the latest"
Andrew Grimberg [Fri, 12 Jul 2019 14:54:09 +0000 (14:54 +0000)]
Merge "Update Ubuntu 16.04 base images to the latest"

5 years agoMerge "Update Ubuntu 16.04 arm64 base image to latest"
Andrew Grimberg [Fri, 12 Jul 2019 14:53:35 +0000 (14:53 +0000)]
Merge "Update Ubuntu 16.04 arm64 base image to latest"

5 years agoUse prettier for auto formatting of JSON/YAML 43/16143/1
Thanh Ha [Wed, 10 Jul 2019 02:24:40 +0000 (22:24 -0400)]
Use prettier for auto formatting of JSON/YAML

No more debate on which style to use, prettier is an opinionated
style formatter for JSON and YAML files.

Also standardize the pre-commit env name with other lfit projects.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I2655382ba6fb099970418475ad4cf939b11752e5

5 years agoReformat JSON and YAML files with prettier 42/16142/1
Thanh Ha [Wed, 10 Jul 2019 02:25:56 +0000 (22:25 -0400)]
Reformat JSON and YAML files with prettier

See I2655382ba6fb099970418475ad4cf939b11752e5 for implementation
details.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I88c7cd50a03f8816bb78ff54dcf574bb65276794

5 years agoUpdate Virtual HW to v2-highcpu-1 65/16065/5
Tim Johnson [Wed, 3 Jul 2019 16:40:33 +0000 (09:40 -0700)]
Update Virtual HW to v2-highcpu-1

Specify the volume size build agent to 20(GB). The defualt values for
v1-standard-1 were adequate (40GB), but the default values for
v2-highcpu-1 were not.

Issue: RELENG-2158
Change-Id: I6db62229d90573b899947171e3dba83fb00efe71
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoScript Cleanup 96/15996/3
Tim Johnson [Tue, 25 Jun 2019 20:46:48 +0000 (13:46 -0700)]
Script Cleanup

Improved error handling, more succinct output (removed -x flag). Minor
enhancements tox config (.coafile). Changed local variables to
lower-case.

Change-Id: Ice1768f01ed513b7236b7cfa42d4b32886a3c83e
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoMerge "Make git2u swap idempotent"
Anil Belur [Thu, 27 Jun 2019 23:02:54 +0000 (23:02 +0000)]
Merge "Make git2u swap idempotent"

5 years agoMake git2u swap idempotent 82/15982/4
Andrew Grimberg [Tue, 25 Jun 2019 00:51:18 +0000 (10:51 +1000)]
Make git2u swap idempotent

Issue: RELENG-2143
Change-Id: Icba128bf8d775edd2b750f05c7662162fcdf9371
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd reno note for aws template fix 99/15999/1 v0.3.1
Andrew Grimberg [Wed, 26 Jun 2019 13:46:16 +0000 (06:46 -0700)]
Add reno note for aws template fix

Change-Id: I4da1dc30a0b8d668f6b5397421bf56bbda241221
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoRemove unknown config value for aws template 98/15998/1
Andrew Grimberg [Wed, 26 Jun 2019 13:28:09 +0000 (06:28 -0700)]
Remove unknown config value for aws template

Change c0dc5f030e259b70533a4d626eae5ab176fdd560 added in an
instance_name variable to the aws template. Unfortunately the
verification which happens in a cloud not using aws as the cloud itself
apparently couldn't pick up the fact that it's not valid with an EC2
definition. Trying to upgrade the LF internal Jenkins to latest
common-packer uncovered this problem.

Change-Id: Id96671cbe674c2af0df7ad4f4babed2b1ee8a4ed
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoFix Ubuntu 18.04 version in var file 95/15995/1
Anil Belur [Tue, 25 Jun 2019 22:23:33 +0000 (08:23 +1000)]
Fix Ubuntu 18.04 version in var file

Change-Id: I5c339d72a97cccc173a30cb1c8f7366ef64d3459
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate Ubuntu 18.04 arm64 base image to latest 87/15987/2
Anil Belur [Tue, 25 Jun 2019 06:48:49 +0000 (16:48 +1000)]
Update Ubuntu 18.04 arm64 base image to latest

Issue: RELENG-2166
Change-Id: I36fc5cc5289039a6d723ff7740d64d90b2d8364e
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate Ubuntu 18.04 amd64 base images 89/15989/1
Anil Belur [Tue, 25 Jun 2019 01:43:21 +0000 (11:43 +1000)]
Update Ubuntu 18.04 amd64 base images

Issue: RELENG-2166
Change-Id: I164e65d8db2b9b40a590cdf9e341309f937193f9
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate Ubuntu 16.04 arm64 base image to latest 91/15991/1
Anil Belur [Tue, 25 Jun 2019 01:37:47 +0000 (11:37 +1000)]
Update Ubuntu 16.04 arm64 base image to latest

Issue: RELENG-2166
Change-Id: I4812d8f764080942733e3671674c26da08b58eae
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate Ubuntu 16.04 base images to the latest 90/15990/1
Anil Belur [Tue, 25 Jun 2019 01:29:07 +0000 (11:29 +1000)]
Update Ubuntu 16.04 base images to the latest

Issue: RELENG-2166
Change-Id: Icd4247a02e2839b4681234f58a2aa5ec3c4f4661
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate CentOS 7 base image to latest 88/15988/1
Anil Belur [Tue, 25 Jun 2019 01:00:06 +0000 (11:00 +1000)]
Update CentOS 7 base image to latest

LF - CentOS 7.1905 (2019-06-04)

Issue: RELENG-2166
Change-Id: I4a11b8106fefe5663f3298176a5ee463b176ee88
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoPin pygments~=2.3.1 and request=2.21.0 76/15976/5
Anil Belur [Mon, 24 Jun 2019 23:49:48 +0000 (09:49 +1000)]
Pin pygments~=2.3.1 and request=2.21.0

Pygments release 2.4.0 which added a python requires that excludes all
versions of Python < 3.5. The LFCI's default 3 version is 3.4 so causes
build failure.

Request-2.22.0 does not work with python-3.4.9, so pin requests to
v2.21.0 to address the tox failures.

Change-Id: I0eca07029e71b55ecff993dac39c4dd0a83bf575
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoRemove with_items for package installs 64/15464/2
Anil Belur [Thu, 2 May 2019 11:28:45 +0000 (21:28 +1000)]
Remove with_items for package installs

Ansible supports lists passed to package install.
With this we can avoid using with_items since this makes
multiple calls to the packages manager slowing down
the performance.

Change-Id: Ieb09ed634533bcbbc60275aafd2c22b3d5c4086d
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoMerge "Add release notes and required upgrade info" v0.3.0
Anil Belur [Sat, 20 Apr 2019 09:51:26 +0000 (09:51 +0000)]
Merge "Add release notes and required upgrade info"

6 years agoAdd release notes and required upgrade info 43/15343/2
Andrew Grimberg [Fri, 19 Apr 2019 18:19:02 +0000 (11:19 -0700)]
Add release notes and required upgrade info

New options for volume attachment means a requirement of packer v1.3.2
is now the minimum supported by common-packer

Change-Id: I1c352d054a01cd14bce365c5a17e3467dff6d25a
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
6 years agoMerge "Fix volume options"
Andrew Grimberg [Fri, 19 Apr 2019 17:54:20 +0000 (17:54 +0000)]
Merge "Fix volume options"

6 years agoFix volume options 36/15336/1
Andrew Grimberg [Fri, 19 Apr 2019 04:41:02 +0000 (21:41 -0700)]
Fix volume options

Change I358c414adb49bba9349f30a1190c2f8b90f13d14 had a few issues. Of
note, packer variables can only be strings, even if they get
interpolated to booleans where needed.

Additionally the volume format can only be specified if
use_block_storage is true and we default to false. This means that if
volume storage is being set then the volume format must be set and it
needs to be set to raw!

Finally, the windows builder did not contain all the extra variables
being defined.

Change-Id: Icb2de795f0ba185a2d758123ec5dd705c86d8980
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
6 years agoMerge "Enable selection of instance knobs"
Jessica Wagantall [Thu, 18 Apr 2019 23:51:02 +0000 (23:51 +0000)]
Merge "Enable selection of instance knobs"

6 years agoEnable selection of instance knobs 33/15333/1
Andrew Grimberg [Thu, 18 Apr 2019 19:56:34 +0000 (12:56 -0700)]
Enable selection of instance knobs

We now operate in multiple Openstack clouds (even if from the same
provider) and thus need to properly enable selectability of the region,
availability zone, instance flavor size, and if a volume needs to be
created (and at what size) for builds. We default to our current
standard configuration for Vexxhost Montreal.

If vm_use_block_storage is set true then vm_volume_size will be used
(along with the image_disk_format option which must be "raw"). If it is
false (default) then the volume size and format will be ignored. This is
needed for building on flavors that do not predefine any attached disk
as is the case for all flavors in the SJC cloud for Vexxhost.

Change-Id: I358c414adb49bba9349f30a1190c2f8b90f13d14
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
6 years agoUpdate the Ubuntu 16.04 arm64 image 54/15254/1
Anil Belur [Wed, 10 Apr 2019 10:30:51 +0000 (20:30 +1000)]
Update the Ubuntu 16.04 arm64 image

Update the Ubuntu 16.04 arm64 image to more recent version.
The image dated '2018-09-06' is no longer available.

Change-Id: I9434cf5ef36b1dbda8b4ae35656966b6a25e4c65
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoAdd lf-network note regarding ssh-agents 08/10508/4
Trevor Bramwell [Tue, 8 May 2018 01:50:10 +0000 (18:50 -0700)]
Add lf-network note regarding ssh-agents

When building packer images in AWS, I found I wasn't easily able to
provision them with the ssh_proxy configurations without also disabling
my ssh-agent.

Even setting all at once: ssh_agent_auth=false,
ssh_disable_agent_forwarding=true, and passing '-a' to the ssh proxy
command did not help.

Change-Id: I01df33f1e0f52082ce25098d52b5b50bee0dafa3
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>
6 years agoUpdate to CentOS 7.6 1811 base image 69/14369/2
Anil Belur [Wed, 30 Jan 2019 12:54:54 +0000 (22:54 +1000)]
Update to CentOS 7.6 1811 base image

The CentOS 7.6 1811 image bundles OpenJDK 11
required for ODL.

Change-Id: I6e34bf69933a90a6a9c1bdd8d1d04ef95a2f5d71
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoRevert "Document ansible min requirement" 40/13540/1 v0.2.0
Anil Belur [Wed, 14 Nov 2018 21:39:43 +0000 (07:39 +1000)]
Revert "Document ansible min requirement"

This reverts commit 60e311b6ad0aa31e22c955fb63c136eeb8844f8c.

As the change If3bcce8f2e0c285609b0efbde293d4e1bf53b787
is reverted, we no longer have a dependency on 2.7 version
of Ansible.

Change-Id: I7d889f9334398d7971d9125831d64e6822b77877
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoUpdate README link to use the lf domain 55/13455/1
Thanh Ha [Mon, 12 Nov 2018 04:32:34 +0000 (12:32 +0800)]
Update README link to use the lf domain

Change-Id: Iac9f441b316296a6fdf17e2031d5f52946e4826b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoDocument ansible min requirement 91/13391/1
Thanh Ha [Wed, 7 Nov 2018 18:35:21 +0000 (02:35 +0800)]
Document ansible min requirement

Patch I9c79063fa21f19356d9b8e0e5f2bb0ca0ed94064 introduced a change
which requires a newer Ansible but missed updating the documenation.

Ref: https://docs.ansible.com/ansible/latest/modules/yum_module.html
Change-Id: Id120b3156aff441b10d20d7ab15b465c7b03371f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Use replace module to update /etc/nsswitch.conf"
Anil Belur [Wed, 7 Nov 2018 11:45:14 +0000 (11:45 +0000)]
Merge "Use replace module to update /etc/nsswitch.conf"

6 years agoDisable apt-daily.* systemd services 18/13218/2
Anil Belur [Sun, 28 Oct 2018 09:40:45 +0000 (19:40 +1000)]
Disable apt-daily.* systemd services

Disable apt-daily.* systemd services that call unattended-upgrades.
Also disable the default configuration for periodic updates.

JIRA: RELENG-1037
Change-Id: Ic9e5e0f8a1543e0d978c0f8528c1a30646a71370
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoUse replace module to update /etc/nsswitch.conf 19/13219/1
Anil Belur [Mon, 29 Oct 2018 02:55:47 +0000 (12:55 +1000)]
Use replace module to update /etc/nsswitch.conf

Use the replace module instead of calling the sed regex
directly from shell.

Change-Id: I2408c49ed2578552c924a484a58979d7b0c24ed6
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoPrevent unattended-upgrades from being installed 87/13187/2
Anil Belur [Thu, 25 Oct 2018 02:41:20 +0000 (12:41 +1000)]
Prevent unattended-upgrades from being installed

Uninstalling unattended-upgrade package is not enough, also mark
the package on hold. This will stop the package being pulled in
on any upgrades.

Change-Id: I8f418c23cb946bdb163805e4412cd33637cc79ae
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoUse pre-commit hook version 1.11.2 88/13188/1
Anil Belur [Thu, 25 Oct 2018 03:22:09 +0000 (13:22 +1000)]
Use pre-commit hook version 1.11.2

The latest version, 1.12.0, is causing problems with the
lf-infra-common-packer-tox-verify-any Jenkins jobs.

Change-Id: Ia9beea08d4f07683acc83a2538ff133c152a5adc
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoMerge "Parse version number from git tags" v0.1.0
Jeremy Phelps [Fri, 19 Oct 2018 15:30:11 +0000 (15:30 +0000)]
Merge "Parse version number from git tags"

6 years agoNote that global-jjb v0.26.0 is required 80/13080/1
Thanh Ha [Thu, 18 Oct 2018 22:02:14 +0000 (18:02 -0400)]
Note that global-jjb v0.26.0 is required

Mention that this version of common-packer requires global-jjb
v0.26.0 first.

Change-Id: I438e52ec6fb332b1c9c2dc10fb2e01c3185d17d5
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoParse version number from git tags 50/13050/3
Thanh Ha [Thu, 18 Oct 2018 15:47:50 +0000 (11:47 -0400)]
Parse version number from git tags

Base documentation version numbers from git tags automatically.

Change-Id: I205b4bb24073de69f135024e9612f9bd28009a93
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoUse reno for release notes 49/13049/2
Thanh Ha [Thu, 18 Oct 2018 15:41:23 +0000 (11:41 -0400)]
Use reno for release notes

After we tag v0.1.0 we will start requiring reno notes moving forward
for patches in this repo.

Change-Id: I0f4aa54c3523197db14f9b9fec46ffe5230c3060
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRemove OpenStack config found in clouds.yaml 47/12547/2
Thanh Ha [Tue, 11 Sep 2018 22:34:48 +0000 (18:34 -0400)]
Remove OpenStack config found in clouds.yaml

We can now reuse configuration already existing in our clouds.yaml to
run packer builds against instead of duplicating that effort in var
files. Packer added this feature in version 1.2.5.

Packer does not read profiles so we have to explicitly pass
some variables that would normally be pulled from profiles such
as auth_url, user_domain_name, and project_domain_name.

  linuxfoundation:
    profile: vexxhost
    auth:
      auth_url: 'https://auth.vexxhost.net/v3/'
      project_name: abc
      username: 123
      password: v1.2.3
      user_domain_name: Default
      project_domain_name: Default
    region_name: ca-ymq-1

Issue: RELENG-1205
Change-Id: I9f05ec1c430a9261e45dc801e5e5aab52a22cfba
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBuild and Install zlib 57/11757/16
Thanh Ha [Thu, 12 Jul 2018 19:55:34 +0000 (15:55 -0400)]
Build and Install zlib

zlib libraries are needed to build several open source tools
on Windows.

Issue: RELENG-999
Change-Id: I137a07c3f9269c521f28f6f08d52c60a72bc1c76
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add Ubuntu 18.04 x86_64 var file"
Andrew Grimberg [Wed, 12 Sep 2018 18:33:06 +0000 (18:33 +0000)]
Merge "Add Ubuntu 18.04 x86_64 var file"

6 years agoMerge "Update CentOS 7 and Ubuntu 16.04 images"
Andrew Grimberg [Wed, 12 Sep 2018 18:31:36 +0000 (18:31 +0000)]
Merge "Update CentOS 7 and Ubuntu 16.04 images"

6 years agoAdd vars for ubuntu-16.04-arm64 53/12553/2
Jessica Wagantall [Wed, 12 Sep 2018 03:09:43 +0000 (20:09 -0700)]
Add vars for ubuntu-16.04-arm64

Add vars to support Ubuntu 16.04 with arm64
architecture.

Change-Id: I53dbd5ad76df6d4601c98f2ce57532cfa3ffd4ed
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoAdd Ubuntu 18.04 x86_64 var file 96/12396/3
Anil Belur [Sun, 26 Aug 2018 02:17:07 +0000 (07:47 +0530)]
Add Ubuntu 18.04 x86_64 var file

ODL would require supporting Ubuntu 18.04 x86_64 vm soon.

Issue: RELENG-1201
Change-Id: Ie440938e6b7545f1b573d0303885d23a2d9439d5
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoUpdate CentOS 7 and Ubuntu 16.04 images 01/12501/2
Thanh Ha [Fri, 7 Sep 2018 20:10:43 +0000 (16:10 -0400)]
Update CentOS 7 and Ubuntu 16.04 images

Issue: RELENG-1201
Change-Id: Ib5d1faee2363a97fe3eb3d1be274df0798d290ad
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Fail ansible-galaxy install step"
Thanh Ha (zxiiro) [Tue, 21 Aug 2018 23:36:14 +0000 (23:36 +0000)]
Merge "Fail ansible-galaxy install step"

6 years agoFail ansible-galaxy install step 55/12355/1
Jessica Wagantall [Tue, 21 Aug 2018 23:24:37 +0000 (16:24 -0700)]
Fail ansible-galaxy install step

Fail packer if ansible-galaxy install step fails to
start due to ~/.netrc being too permissive.

Change-Id: I67b922688e738986dfbae85225b78df342d792ce
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoAdd mimimum requirements for Ansible and Packer 16/12316/6
Anil Belur [Sat, 18 Aug 2018 05:44:20 +0000 (11:14 +0530)]
Add mimimum requirements for Ansible and Packer

Also fix toctree error by adding lf-network to toctree.

Change-Id: I23c1adb2a8beecf3f99a495a554d8a59f6c65a67
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoUse new lf images for arm images 46/12146/1
Jeremy Phelps [Tue, 7 Aug 2018 19:51:00 +0000 (14:51 -0500)]
Use new lf images for arm images

These are v2 flavors which have 40gb root volume attached and
have been renamed to lf.

Change-Id: I0375e8b639e086d6188fe05eb5c47a3f22fffa98
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
6 years agoEnable YAML linter and fix lint issues 10/12110/1
Thanh Ha [Fri, 3 Aug 2018 19:39:44 +0000 (15:39 -0400)]
Enable YAML linter and fix lint issues

Resolve relevant lint issues and enable the YAML linter.

Change-Id: I11cb7b6a1edb72f165ba9ba051bc19794e4684bd
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>