releng/common-packer.git
3 years agoFeat: Add support for CentOS Streams 8 92/69792/1 v0.10.0
Anil Belur [Thu, 3 Mar 2022 23:18:53 +0000 (09:18 +1000)]
Feat: Add support for CentOS Streams 8

Issue-ID: RELENG-4141
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Change-Id: I172aa7399e5f235082f058c7ab4690d83c5ad381

3 years agoChore: Update base images for Ubuntu 75/68975/1 v0.9.4
Jessica Wagantall [Thu, 14 Oct 2021 19:35:11 +0000 (12:35 -0700)]
Chore: Update base images for Ubuntu

Update base images for Ubuntu 18.04 and 20.04
ARM and x86

Issue: RELENG-4010
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I243b0547173afb79f38755c74dc4891a71b551ea

3 years agoChore: Update Ubuntu 18.04 68/68968/2
Jessica Wagantall [Tue, 12 Oct 2021 20:27:07 +0000 (13:27 -0700)]
Chore: Update Ubuntu 18.04

Update base image for Ubuntu 18.04

Issue: RELENG-3989
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I1821b6678745c89842936ced72e8cda06abf939b

3 years agoChore: Upgrade pre-commit hooks 81/68681/1 v0.9.3
Andrew Grimberg [Wed, 1 Sep 2021 18:20:39 +0000 (11:20 -0700)]
Chore: Upgrade pre-commit hooks

* github.com/pre-commit/pre-commit-hooks: v3.4.0 -> v4.0.1
* github.com/pre-commit/mirrors-prettier: v2.2.1 -> v2.3.2

Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I5499abc30a514568d7a4fe30f679d10c4fff5a22

3 years agoFix: Add missing Release Notes for latest packer 89/68589/2 v0.9.2
Jessica Wagantall [Thu, 19 Aug 2021 22:22:57 +0000 (15:22 -0700)]
Fix: Add missing Release Notes for latest packer

Add Release Notes for packer requirements changed
recently.

Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I459c7da09c3a5262e4093d98c763a21dd73df0b3

3 years agoFix: format of requirements.yaml 87/68587/1
quentinDERORY [Thu, 19 Aug 2021 21:51:16 +0000 (14:51 -0700)]
Fix: format of requirements.yaml

Signed-off-by: quentinDERORY <quentin.derory@ecole.ensicaen.fr>
Change-Id: Ibaef85987792566938a0ea7ef397148701c371ca

3 years agoChore: Add new galaxy requirement for magma project 78/68578/1
quentinDERORY [Wed, 18 Aug 2021 21:11:53 +0000 (14:11 -0700)]
Chore: Add new galaxy requirement for magma project

Signed-off-by: quentinDERORY <quentin.derory@ecole.ensicaen.fr>
Change-Id: I6a884a6d3c831222a095d1f75f958f1eed18c815

3 years agoFix: Use Vexx provided base images 34/68434/1
Jessica Wagantall [Wed, 4 Aug 2021 22:18:08 +0000 (15:18 -0700)]
Fix: Use Vexx provided base images

Use base images provided by Vexx instead of
LF's ones.

Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: If2ecdf622235ae82211bd0051314713775676c23

3 years agoFix: Enable powertools repo for CentOS 8.* vers 39/68039/5 v0.9.1
Anil Belur [Thu, 17 Jun 2021 01:23:55 +0000 (11:23 +1000)]
Fix: Enable powertools repo for CentOS 8.* vers

The CR I43b943ddfc38faca does not work as expected since
ansible_facts does not match/return the minor versions of
the Repoid correctly.

For CentOS 8.2.2004 and earlier versions uses repoid as
'PowerTools' while CentOS 8.3.2011 and later versions uses repoid as
'powertools'. To handle this, check the repo file name under
/etc/yum.repos.d/ and enable the correct repository.

Ref: https://wiki.centos.org/Manuals/ReleaseNotes/CentOS8.2011\
Issue: RELENG-3710
Change-Id: Ib6a8d78664bbb5f20d2889fae3ae3b4238007141
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
3 years agoFix: AMI product codes need to be defined 56/67956/4 v0.9.0
Andrew Grimberg [Thu, 10 Jun 2021 20:05:27 +0000 (13:05 -0700)]
Fix: AMI product codes need to be defined

While trying to upgrade the internal Jenkins to latest global-jjb and
common-packer it's been found that all of our configurations for AWS
systems are missing vital filters.

The Ubuntu product codes were found by doing the following:

aws ec2 describe-images --region us-west-2 \
  --owners aws-marketplace --filters \
  "Name=architecture,Values=arm64" \
  "Name=name,Values=*ubuntu*20.04*" \
  "Name=manifest-location,Values=aws-marketplace/ubuntu/*"

This is per documentation found at
https://ubuntu.com/server/docs/cloud-images/amazon-ec2

Issue: RELENG-3757
Change-Id: I9391f6bec59318e00bc1b097c7a27181defeccff
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
3 years agoCI: Configure gitlint for Conventional Commits 58/67958/1
Andrew Grimberg [Thu, 10 Jun 2021 20:55:09 +0000 (13:55 -0700)]
CI: Configure gitlint for Conventional Commits

Turn on Conventional Commit message subject validation. This will affect
CI specifically, but developers can avoid having CI fail by assuring
that they have installed pre-commit and that they have also run

pre-commit install --hook-type commit-msg

The above is needed because pre-commit does not install commit-msg hooks
by default

Change-Id: I99779c6188ab2a4e57022a4ac552cbc3749accc2
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
3 years agoCI: Update pre-commit and make it work properly in CI 57/67957/1
Andrew Grimberg [Thu, 10 Jun 2021 20:53:56 +0000 (13:53 -0700)]
CI: Update pre-commit and make it work properly in CI

Change-Id: Idf2dbf580ca14af8a3c868a385658b4d51c4de9e
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
3 years agoFeat: Add support for Docker builder on Windows 07/67907/2
Anil Belur [Mon, 7 Jun 2021 22:23:47 +0000 (08:23 +1000)]
Feat: Add support for Docker builder on Windows

The original change e24c07369afd514abdf3efb0f596f772261412ed
missed updating Windows var files, while the builder templates were
updated. This breaks the packer verify jobs.

Error:
    required variable not set: docker_source_image'

Change-Id: Iac258a0ff132e12e00a49d9d02ce0f1806bef016
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
3 years agoFeat: Add support for Docker builder on arm64 78/67878/1
Anil Belur [Fri, 4 Jun 2021 02:50:17 +0000 (12:50 +1000)]
Feat: Add support for Docker builder on arm64

The original change e24c07369afd514abdf3efb0f596f772261412ed
missed updating arm64 var files and Ubuntu 20:04 var file,
while the builder template were updated. This breaks the
packer verify jobs.

Error:
* required variable not set: docker_source_image'

Issue: RELENG-3717
Change-Id: I1f3acf1479bf7a4ab271f747da5261ee2619e566
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
3 years agoFix: Change powertools repoid to lowercase 44/67844/4 v0.8.0
Anil Belur [Sat, 29 May 2021 03:47:14 +0000 (13:47 +1000)]
Fix: Change powertools repoid to lowercase

The repoids are renamed with CentOS 8.3 update breaks the packer
builds. Now the correct repoid to use is 'powertools' instead
of 'PowerTools'

Error: No matching repo to modify: PowerTools.

Ref:
https: //wiki.centos.org/Manuals/ReleaseNotes/CentOS8.2011#Yum_repo_file_and_repoid_changes
Issue: RELENG-3710
Change-Id: I43b943ddfc38facaa8c02f2d6ee98254e7ee3241
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoAdd docker-aws template 85/67485/1
Eric Ball [Tue, 20 Apr 2021 18:38:27 +0000 (11:38 -0700)]
Add docker-aws template

We now have at least one project on AWS that needs a docker template.

Issue: IT-21917
Change-Id: Ia08a1a32eb2ffaab46b1f01468018de688423852
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
4 years agoMerge "Use official CentOS & Ubuntu images via prodId"
Andrew Grimberg [Mon, 12 Apr 2021 16:06:58 +0000 (16:06 +0000)]
Merge "Use official CentOS & Ubuntu images via prodId"

4 years agoUse official CentOS & Ubuntu images via prodId 65/63865/2
Thanh Ha [Sat, 2 May 2020 02:20:16 +0000 (22:20 -0400)]
Use official CentOS & Ubuntu images via prodId

The previous filter method could result in getting an unexpected image
even one that's entirely different than what is intended. This patch
uses a product-code search of the marketplace for the official upstream
supported images rather than the custom ones provided by Amazon.

This patch is based on Zack William's suggestion on the lf-releng
mailinglist.

Reference:
https://lists.linuxfoundation.org/pipermail/lf-releng/2020-February/000135.html

Change-Id: Ia6208bd7a61036eb210d07533e3ceedbbc787678
Co-authored-by: Zack Williams <zdw@opennetworking.org>
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
4 years agoAdd support for a Docker builder 91/65091/6
Thanh Ha [Fri, 14 Aug 2020 14:23:29 +0000 (10:23 -0400)]
Add support for a Docker builder

Adding a Docker builder will allow us to more easily test build images
locally in a container before pushing a patch to production to build
the VMs themselves. While not guarenteed to be exactly the same it's
a close enough approximation that many issues can be caught before
patches are pushed.

Additionally this allows projects to also use the same packer templates
to build container based builder nodes if there's any projects using
Docker for their Jenkins jobs.

Issue: RELENG-1052
Change-Id: If89801cea16046900e7e880265ad859a96595fa1
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
4 years agoMerge "Update CentOS8 image" v0.7.7
Andrew Grimberg [Fri, 26 Mar 2021 14:59:43 +0000 (14:59 +0000)]
Merge "Update CentOS8 image"

4 years agoFixing netselect in install-python 37/67137/9
Bengt Thuree [Thu, 18 Mar 2021 11:24:37 +0000 (22:24 +1100)]
Fixing netselect in install-python

Actually use the result from netselect

Issue-ID: CIMAN-30

Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Change-Id: I31b5858dc4b348f383f3b801b17c1aa8547296bd

4 years agoUpdate CentOS8 image 10/67110/1
Anil Belur [Wed, 17 Mar 2021 03:03:19 +0000 (13:03 +1000)]
Update CentOS8 image

Change-Id: If34cb35b7b54b7015e7c20a53c9ace9c10db5500
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoMerge "Migrate jobs from lf-standard-1 flavor to v3" v0.7.6
Anil Belur [Tue, 23 Feb 2021 08:00:51 +0000 (08:00 +0000)]
Merge "Migrate jobs from lf-standard-1 flavor to v3"

4 years agoUpdate shellcheck pre-commit hook to 2.1.5 09/66809/1 v0.7.5
Anil Belur [Thu, 18 Feb 2021 01:59:18 +0000 (11:59 +1000)]
Update shellcheck pre-commit hook to 2.1.5

With 2.1.5 release the workaround added in
d7d264e733e5 is no longer required.

Change-Id: I7e890d03844a2fb47159ad471f166310290fbf9f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoMigrate jobs from lf-standard-1 flavor to v3 93/66793/2
Anil Belur [Tue, 16 Feb 2021 13:27:38 +0000 (23:27 +1000)]
Migrate jobs from lf-standard-1 flavor to v3

v3 flavors guarantees jobs spin on newer hardware that
are faster and cost-efficient than the v2 flavors.

Issue: RELENG-3329
Change-Id: I97f8d540f02fdccb3115bc624b167eb12464833b
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoAdd workaround to for shellcheck failure 94/66794/1
Anil Belur [Wed, 17 Feb 2021 00:21:12 +0000 (10:21 +1000)]
Add workaround to for shellcheck failure

The shellcheck hook breaks with the latest version of pre-commit (2.1.1)

[ERROR] The hook `shellcheck` specifies `additional_dependencies`
but is using language `script` which does not install an environment.

Ref:
https://github.com/jumanjihouse/pre-commit-hooks/pull/81

Change-Id: I19d8b26a7589d0bde7f1d3c6f768b738289ab937
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoRevert "Set packer builder type with env variable" 96/66596/1 v0.7.4
Anil Belur [Sat, 23 Jan 2021 01:45:35 +0000 (11:45 +1000)]
Revert "Set packer builder type with env variable"

This reverts commit cc035619ac1673a5b5ea2f6bfb843eac75ea4a9d.

Based on the input provided on the Gihub packer issue
hard-coded for our parser to be able to decode the rest of the config
properly."

Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Change-Id: If63b83eae7baa8ab6b30d8b9b187aa219ec2ba44

4 years agoRevert "Set packer builder type with env variable" 95/66595/1
Anil Belur [Sat, 23 Jan 2021 01:45:03 +0000 (11:45 +1000)]
Revert "Set packer builder type with env variable"

This reverts commit 8041971ef211709579f919bdeb9a77fef0a29433.

Based on the input provided on the Gihub packer issue
hard-coded for our parser to be able to decode the rest of the config
properly."

Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Change-Id: I52ef1448fc53b2b9777786994683fbd705a1f8ba

4 years agoSet packer builder type with env variable 59/66559/1 v0.7.3
Anil Belur [Thu, 21 Jan 2021 08:32:12 +0000 (18:32 +1000)]
Set packer builder type with env variable

Using the -only CLI option for packer build does not work as
expected. The #CR 90dd675df482a8 (global-jjb) submitted with
common-packer v0.7.0 fails to build packer jobs because of a
bug in the packer build CLI.

The `-only` option packer build CLI does not set the cloud
provider name/type as per the documentation. The alternative
is to use an environment variable that is passed through the
packer job and can be reference in the packer templates.

The CR cc035619ac16 missed including the changes to the other
templates.

Issue: https://github.com/hashicorp/packer/issues/10495
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Change-Id: I14976aa77312a3d887d30b7d6e2cc32ab8073b72

4 years agoSet packer builder type with env variable 28/66528/3 v0.7.2
Anil Belur [Sat, 16 Jan 2021 05:11:31 +0000 (15:11 +1000)]
Set packer builder type with env variable

Using the -only CLI option for packer build does not work as
expected. The #CR 90dd675df482a8 (global-jjb) submitted with
common-packer v0.7.0 fails to build packer jobs because of a
bug in the packer build CLI.

The `-only` option packer build CLI does not set the cloud
provider name/type as per the documentation. The alternative
is to use an environment variable that is passed through the
packer job and can be reference in the packer templates.

Issue: https://github.com/hashicorp/packer/issues/10495
Change-Id: I1c1e2e7c5545df5a2a374a02714634a0f4086378
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoRemove openstack AZ from templates 73/66373/4 v0.7.1
Eric Ball [Tue, 29 Dec 2020 20:06:23 +0000 (12:06 -0800)]
Remove openstack AZ from templates

The availability zone started causing issues with packer builds. Per
Vexxhost help, there is only one AZ, and removing this line
alleviates the issue.

Issue: RELENG-3344
Change-Id: Icf60fe527ae328d7883ef38c3efba9972d2f2669
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
4 years agoUpdate prettier URL for pre-commit 74/66374/2
Eric Ball [Tue, 29 Dec 2020 20:22:21 +0000 (12:22 -0800)]
Update prettier URL for pre-commit

Issue: RELENG-3344
Change-Id: I07daf63bcb03252367b11a64f44c938bf807eea6
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
4 years agoMerge "Workflow for updating images."
Aric Gardner [Mon, 31 Aug 2020 18:33:59 +0000 (18:33 +0000)]
Merge "Workflow for updating images."

4 years agoWorkflow for updating images. 81/61681/15
Aric Gardner [Tue, 10 Sep 2019 19:08:20 +0000 (15:08 -0400)]
Workflow for updating images.

documenation on:
"Update Packer Base Image"
and
"Update Packer Images to reflect changes
in LF ansible galaxy roles"

ISSUE: RELENG-2363
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ia2bac9c336772742bf58058114b1668498cfcbd3

4 years agoAdd Ubuntu 20.04 arm64 image 37/65137/1 v0.7.0
Anil Belur [Wed, 19 Aug 2020 04:55:20 +0000 (14:55 +1000)]
Add Ubuntu 20.04 arm64 image

Issue: JSD IT-20435
Change-Id: I5f21934ec15754892d3ebea826965167edb84601
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoAdd Ubuntu 20.04 amd64 image 36/65136/1
Anil Belur [Wed, 19 Aug 2020 04:50:38 +0000 (14:50 +1000)]
Add Ubuntu 20.04 amd64 image

Issue: JSD IT-20435
Change-Id: Iec986b683fe168c417c40c0815d743d28c288dd6
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoMerge "Add support for Ubuntu 20.04 and newer"
Andrew Grimberg [Tue, 18 Aug 2020 17:13:24 +0000 (17:13 +0000)]
Merge "Add support for Ubuntu 20.04 and newer"

4 years agoEnsure sudo is installed as Ansible requirement 94/65094/1
Thanh Ha [Fri, 14 Aug 2020 14:44:14 +0000 (10:44 -0400)]
Ensure sudo is installed as Ansible requirement

Ansible requires sudo to be available and Docker images do not
typically come with sudo pre-installed so add code to ensure it
get's pulled in.

Issue: RELENG-1052
Change-Id: I8e67f7048f2199f208c7596cb7b7c6182e61ed51
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
4 years agoRename vexxhost to openstack for builder names 90/65090/2
Thanh Ha [Fri, 14 Aug 2020 14:15:42 +0000 (10:15 -0400)]
Rename vexxhost to openstack for builder names

Considering the AWS builders are named AWS after the service let's
rename the vexxhost ones to openstack. As far as I am aware we are
not supporting multiple providers with the same backend at the same
time so this will make things consistent.

When we later add docker support it will just be called "docker" as
well.

Issue: RELENG-1052
Change-Id: I8f46d544961e508cdeee28d4e761c2feae5e744e
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
4 years agoAdd support for Ubuntu 20.04 and newer 83/65083/1
Thanh Ha [Thu, 13 Aug 2020 22:17:38 +0000 (18:17 -0400)]
Add support for Ubuntu 20.04 and newer

Ubuntu 20.04 no longer has python-* packages and instead the packages
are now more explicitly either python2-* or python3-* with packages
named python-is-python[23] to set the default. This patch when run
on Ubuntu systems will check if python-is-python3 is available and
installs that as the system python version, otherwise installs
python-minimal.

Also adjust the comments since they are no longer accurate or in the
right places.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I830e0250ddbee395fdeddd78fcc7824f7855d75d

4 years agoUpdate Ubuntu 18.04 base image 15/64815/1 v0.6.2
Aric Gardner [Thu, 30 Jul 2020 03:10:00 +0000 (13:10 +1000)]
Update Ubuntu 18.04 base image

Change-Id: I4b40ff1c84cfa2f64235b3a8132e6d43baea1f3a
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoUpdate Ubuntu base image to 2020-07-28 snapshot 97/64797/4 v0.6.1
Aric Gardner [Tue, 28 Jul 2020 19:22:24 +0000 (15:22 -0400)]
Update Ubuntu base image to 2020-07-28 snapshot

base image was uploaded in qcow2 format.
this casued timeout in packer builds.
new image is in raw format.

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I0151e6b8677625e4477d93faa67cb1abdf84515c

4 years agoAdd release notes for sigul client on CentOS 7 21/64621/1 v0.6.0
Eric Ball [Thu, 23 Jul 2020 18:48:55 +0000 (11:48 -0700)]
Add release notes for sigul client on CentOS 7

Issue: RELENG-3077
Change-Id: I3314b0770a0f0280ee162bbb646f95312cff6196
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
4 years agoAdd install of Sigul client on CentOS 7 10/64610/1
Eric Ball [Thu, 23 Jul 2020 01:27:12 +0000 (18:27 -0700)]
Add install of Sigul client on CentOS 7

Our current signing scripts download and install Sigul only when
executing the signing scripts. However, the servers hosting this
package are not reliable, and we should see significant performance
improvements by having the package pre-loaded on CentOS 7 builders
(the only builders capable of using Sigul).

Issue: RELENG-3077
Change-Id: I35ac34a3619f4018331f280745f4ff3d5c864fa9
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
4 years agoUpdate the CentOS 7 base image to the latest 77/64277/2
Anil Belur [Tue, 16 Jun 2020 00:37:09 +0000 (10:37 +1000)]
Update the CentOS 7 base image to the latest

https://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud-2003.raw.tar.gz

Issue: IT-19958
Change-Id: Ib6f84fc36d6644c286540abd11c8446bea78497f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoFix provisioner to install Git 2 from IUS 61/64261/2 v0.5.2
Thanh Ha [Fri, 12 Jun 2020 00:37:12 +0000 (20:37 -0400)]
Fix provisioner to install Git 2 from IUS

While patch Ibace4cdf89d0615a2e5117f9ea3ca50730d5c93d fixed the setup
for the IUS repo. It failed to ensure that Git 2 is installed from the
IUS repo.

In the refactored IUS repo git2u no longer exists and instead the
latest Git package appears to be git224 so let's update to that
version.

Also class cast ansible_distribution_major_version|int as this variable
normally returns a type AnsibleUnsafeText which cannot be compared to
an int type.

Change-Id: I1cd815fb0440be996ba66c3a48eab9d6da02f4bb
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
4 years agoMerge "Run docs and docs-linkcheck in verify"
Andrew Grimberg [Sat, 6 Jun 2020 14:32:43 +0000 (14:32 +0000)]
Merge "Run docs and docs-linkcheck in verify"

4 years agoMerge "Add .editorconfig file"
Andrew Grimberg [Sat, 6 Jun 2020 14:26:48 +0000 (14:26 +0000)]
Merge "Add .editorconfig file"

4 years agoMerge "Print git-diff when pre-commit fails"
Andrew Grimberg [Sat, 6 Jun 2020 14:25:54 +0000 (14:25 +0000)]
Merge "Print git-diff when pre-commit fails"

4 years agoRun docs and docs-linkcheck in verify 30/64230/1
Thanh Ha [Sat, 6 Jun 2020 14:21:06 +0000 (10:21 -0400)]
Run docs and docs-linkcheck in verify

Not sure why we weren't running docs build tests as part of the
common-packer verify build but this patch updates tox to enable
those build tests.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I7d28bdf8ace6946bcf1b6a2d08bc1408875fce96

4 years agoAdd .editorconfig file 25/64225/1
Thanh Ha [Sat, 6 Jun 2020 14:04:39 +0000 (10:04 -0400)]
Add .editorconfig file

EditorConfig is a universal editor configuration file supported by many
IDEs and Editors (although some might require installing a plugin) to
share consistent editor configuration with developers using different
editors.

Ref: https://editorconfig.org/
Change-Id: I59a4a57298e295dcfe7d21591a8c0fb501b0cef6
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
4 years agoPrint git-diff when pre-commit fails 24/64224/1
Thanh Ha [Sat, 6 Jun 2020 14:01:53 +0000 (10:01 -0400)]
Print git-diff when pre-commit fails

When pre-commit fails and automatically fixes issues for us in Jenkins
we are not able to see what pre-commit automatically fixed. Adding
--show-diff-on-failure makes pre-commit perform a git-diff immediately
after failing to ensure more useful output is printed to inform the
contributor what changes are needed to pass the check.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: Ic2ad49fd6222bee0341c4826e6ccc463d05fda5f

4 years agoUpdate pre-commit hooks 2020-06 23/64223/1
Thanh Ha [Sat, 6 Jun 2020 14:00:27 +0000 (10:00 -0400)]
Update pre-commit hooks 2020-06

It's been awhile since we last updated the pre-commit hooks so let's
catch up with the world.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: Ibf5e945a2bb6324c73ea5b2d7a683bc0f926384b

5 years agoFix PowerTools enablement and IUS install 42/64042/1 v0.5.1
Andrew Grimberg [Wed, 20 May 2020 20:08:50 +0000 (13:08 -0700)]
Fix PowerTools enablement and IUS install

* Use correct syntax for verifying the major version(s) we
  should be setting IUS up on
* Fix the fact we're checking to know if should be enabling
  the PowerTools repo

Change-Id: I3340d5aae147a2c5881d973442fba4e9543d8c1e
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoFix version comparison for PowerTools installation 05/63705/1
Andrew Grimberg [Fri, 17 Apr 2020 21:42:42 +0000 (14:42 -0700)]
Fix version comparison for PowerTools installation

Apparently ansible_distribution_major_version returns a string and
cannot therefore properly be tested in an integer based test. Use the
version() function instead.

https://docs.ansible.com/ansible/latest/user_guide/playbooks_tests.html#version-comparison

Change-Id: I7abb42f94b1543aed5fe3ca548184ac1082612d7
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoSupport CentOS 8 builds 31/63531/3 v0.5.0
Andrew Grimberg [Fri, 27 Mar 2020 22:14:36 +0000 (15:14 -0700)]
Support CentOS 8 builds

Changes needed to run the packer against CentOS 8 systems

Issue: RELENG-2823
Change-Id: I7748d98150630bc5327d407eab5421a2ec21e7bf
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoAdd Centos 8 base image 00/63500/2
Lott, Christopher (cl778h) [Wed, 25 Mar 2020 15:06:45 +0000 (11:06 -0400)]
Add Centos 8 base image

Issue-ID: RELENG-2823
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
Change-Id: I6c00ef45e269028f60a06f7f4aa9c38729ad952a

5 years agoUpdate ARM base images 32/63432/1
Jessica Wagantall [Tue, 17 Mar 2020 20:20:04 +0000 (13:20 -0700)]
Update ARM base images

Update the latest (tested) baseimages for ARM
Ubuntu 16.40 and Ubuntu 18.04

Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: Ide6115ef7f9d3e42935011aaf37872adc759e589

5 years agoInstall netselect depending on the Ubuntu arch 47/63147/12 v0.4.4
Bengt Thuree [Wed, 19 Feb 2020 01:01:27 +0000 (12:01 +1100)]
Install netselect depending on the Ubuntu arch

Change-Id: Ie93b7b5d510ff1e067db4fcfc1020f9ae93dadcf
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAllow the jenkins user to sudo as any user 25/63125/1
Thanh Ha [Fri, 14 Feb 2020 03:12:17 +0000 (22:12 -0500)]
Allow the jenkins user to sudo as any user

Previously the Jenkins user could only sudo and run commands as root.
This change allows the Jenkins user to `sudo -u another_user` without
providing a password. This is useful for test suites that need to
perform commands as different user accounts and not just root.

Change-Id: Ib7864e8b9370ae7f1a2a8bec82cf609e7dcccead
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
5 years agoMinor Cleanup 97/62997/1
Tim Johnson [Thu, 30 Jan 2020 16:12:45 +0000 (08:12 -0800)]
Minor Cleanup

Change-Id: Ia826901dbbc1a3fd6f5df4af22418015b9dbc78c
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoDo not fail the build when a mirror is not found 81/62981/1 v0.4.3
Anil Belur [Wed, 29 Jan 2020 10:09:38 +0000 (20:09 +1000)]
Do not fail the build when a mirror is not found

Forgot to include this check in the change 4c9db8e41b3b546a52.
netselect checks for the mirrors from the list and when one
or more mirrors are not available returns the below error
causing the build to fail.

error:
netselect: unknown host gpl.savoirfairelinux.net/pub/mirrors/ubuntu/

Issue: RELENG-2642
Change-Id: I4351a40be996092febe4efe956b18a9a575c994e
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Use netselect to choose a package mirror" v0.4.2
Anil Belur [Wed, 29 Jan 2020 01:27:01 +0000 (01:27 +0000)]
Merge "Use netselect to choose a package mirror"

5 years agoMerge "Revert "Use apt-get to install python-minimal""
Anil Belur [Wed, 29 Jan 2020 01:26:42 +0000 (01:26 +0000)]
Merge "Revert "Use apt-get to install python-minimal""

5 years agoAdd GitLint which replaces GitCommitBear 76/62976/2
Thanh Ha [Tue, 28 Jan 2020 15:39:05 +0000 (10:39 -0500)]
Add GitLint which replaces GitCommitBear

The GitCommitBear was removed via
I8d782822ca87add9108a044fd6e19b1c23686717 this replaces it with
GitLint which does similar linting for git commit messages.

Issue: RELENG-2642
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I6e47db45a6c5c519ee795adf15c4243fb0bfb6dd

5 years agoUse netselect to choose a package mirror 68/62968/1
Anil Belur [Tue, 28 Jan 2020 02:56:48 +0000 (12:56 +1000)]
Use netselect to choose a package mirror

Use netselect to choose a package mirror to install python-minimal in a
reliable manner.

apt{-get} does not refresh the package mirrors (for packer builds run
within Jenkins), therefore fails with "E: Unable to locate package
python-minimal" while installing python-minimal.

Issue: RELENG-2548
Change-Id: Iab90212657816a9b66db07b356c32ae7e90272d8
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoRevert "Use apt-get to install python-minimal" 67/62967/1
Anil Belur [Tue, 28 Jan 2020 03:00:51 +0000 (13:00 +1000)]
Revert "Use apt-get to install python-minimal"

This reverts commit 9eca5443d48cde7a5f16e16cb16d89e3cd3ab66c.

This only works intermittently, therefore reverting the commit.

Change-Id: If7a2f9850e1054e251e596d104e6c4ba9eb52338
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUse apt-get to install python-minimal 30/62930/1 v0.4.1
Anil Belur [Fri, 24 Jan 2020 06:54:13 +0000 (16:54 +1000)]
Use apt-get to install python-minimal

apt does not refresh the package mirrors for the jobs
run within the Jenkins, and fails to install python-minimal.

With apt issue is reproducable:
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/builder-packer-merge-ubuntu-16.04-docker/74

With apt-get as a workaround resolves the packages with the mirrors sussessfully:
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/builder-packer-merge-ubuntu-16.04-docker/73

Issue: RELENG-2548
Change-Id: I3436fa250bff5b90e2b4c390a97c0b989b52d7af
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd pre-commit hook for shellcheck 29/62829/3
Anil Belur [Thu, 16 Jan 2020 08:09:24 +0000 (18:09 +1000)]
Add pre-commit hook for shellcheck

Use pre-commit to run the shellcheck hook rather than coala as
we want to eventually phase out coala.

Issue: RELENG-2642
Change-Id: Ie06bbb0a06ba517bcc378480ea39320da00403c6
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpgrade pre-commit hooks version to the latest 31/62831/2
Anil Belur [Thu, 16 Jan 2020 02:18:55 +0000 (12:18 +1000)]
Upgrade pre-commit hooks version to the latest

Change-Id: I10dade2daa3b0379ff402875c1333d9457dd22c2
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoResolve shellcheck SC2196: egrep is non-standard 30/62830/1
Anil Belur [Thu, 16 Jan 2020 02:13:53 +0000 (12:13 +1000)]
Resolve shellcheck SC2196: egrep is non-standard

SC2196: egrep is non-standard and deprecated. Use grep -E instead.

Change-Id: I25f083bca244580ec01a8a1e000b13cc5c3ba2e5
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Update Ubuntu 18.04 arm64 base image to the latest" v0.4.0
Anil Belur [Tue, 7 Jan 2020 10:42:28 +0000 (10:42 +0000)]
Merge "Update Ubuntu 18.04 arm64 base image to the latest"

5 years agoMerge "Update Ubuntu 16.04 amd64 base image to latest"
Anil Belur [Tue, 7 Jan 2020 10:42:12 +0000 (10:42 +0000)]
Merge "Update Ubuntu 16.04 amd64 base image to latest"

5 years agoMerge "Update Ubuntu 16.04 arm64 base image to latest"
Anil Belur [Tue, 7 Jan 2020 10:42:08 +0000 (10:42 +0000)]
Merge "Update Ubuntu 16.04 arm64 base image to latest"

5 years agoUpdate Ubuntu 16.04 amd64 base image to latest 42/62642/3
Anil Belur [Mon, 16 Dec 2019 22:40:04 +0000 (08:40 +1000)]
Update Ubuntu 16.04 amd64 base image to latest

Change-Id: Iec964db3b3652379641e3336bfe3df18ee796300
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate Ubuntu 16.04 arm64 base image to latest 41/62641/2
Anil Belur [Mon, 16 Dec 2019 22:01:53 +0000 (08:01 +1000)]
Update Ubuntu 16.04 arm64 base image to latest

Issue: RELENG-2548
Change-Id: I88b8c6fa37dba294c7caec09d862963f77c146ef
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate Ubuntu 18.04 amd64 base image to the latest 40/62640/2
Anil Belur [Mon, 16 Dec 2019 22:14:08 +0000 (08:14 +1000)]
Update Ubuntu 18.04 amd64 base image to the latest

Issue: RELENG-2548
Change-Id: Iac6f3a9bc56172ad607ead417b4f538724e5fcc4
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate Ubuntu 18.04 arm64 base image to the latest 39/62639/2
Anil Belur [Mon, 16 Dec 2019 22:17:29 +0000 (08:17 +1000)]
Update Ubuntu 18.04 arm64 base image to the latest

Issue: RELENG-2548
Change-Id: I5d520ef30747f2e842c64057a7060a68782ebb80
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoRemove coala from common-packer 79/62679/4
Anil Belur [Thu, 19 Dec 2019 22:18:47 +0000 (08:18 +1000)]
Remove coala from common-packer

Move away from coala since that project refuses to produce any
new release and the version we're using is quite dated at this
point.

Coala does not seem to be working well with prettier pre-commit
hook in place, since prettier is auto-formating the yaml code
to make the code terse which fails the tox verify job.

Issue: RELENG-2641
Change-Id: I8d782822ca87add9108a044fd6e19b1c23686717
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoChange IUS configuration 44/62244/1
Andrew Grimberg [Wed, 13 Nov 2019 15:31:55 +0000 (07:31 -0800)]
Change IUS configuration

While trying to troubleshoot why a recent CentOS7 system was failing to
get past the installation of yum-plugin-replace from IUS it was
discovered that IUS has changed up the keys that they are signing with.

Along with this change are changes to their official distribution point
according to the standard release package.

While we don't actually install the release package, we do have reason
to configure the rest of the repository correctly, as such we need to
follow the changes that were made back in May of 2019!

Change-Id: Ibace4cdf89d0615a2e5117f9ea3ca50730d5c93d
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoNeed become true for yum_repository 93/61793/1
Aric Gardner [Thu, 19 Sep 2019 22:37:11 +0000 (18:37 -0400)]
Need become true for yum_repository

Don't set priority

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I5d0449dd983acf0fee2b69b8f2994fa709ddf3a6

5 years agoAnsible yum_repository has includepkgs option 91/61791/1
Aric Gardner [Thu, 19 Sep 2019 21:40:28 +0000 (17:40 -0400)]
Ansible yum_repository has includepkgs option

Limit IUS repo ro git2u and yum replace plugin
Something about the order of the yum package install
causes yum to try to pull down python36u

This can be reproduced in docker with
https://gist.github.com/Aricg/73a872fcaaa5eae1302d71eb2237d208

ISSUE: IT-17631
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Id968757b41fe6d99f9a22bcbaf98be74b5d05e3e

5 years agoOnly use IUS for git2u 84/61784/4
Aric Gardner [Thu, 19 Sep 2019 18:40:21 +0000 (14:40 -0400)]
Only use IUS for git2u

Limit IUS repo ro git2u and yum replace plugin
Something about the order of the yum package install
causes yum to try to pull down python36u

This can be reproduced in docker with
https://gist.github.com/Aricg/73a872fcaaa5eae1302d71eb2237d208

ISSUE: IT-17631
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I65a8b388340e28ceea45d6b3ad33bcc84f31e7ac

5 years agoConfigure basepython to python3 10/16710/1
Thanh Ha [Tue, 27 Aug 2019 19:18:34 +0000 (15:18 -0400)]
Configure basepython to python3

More and more libraries that we are pulling in require Python 3 so
to avoid having to play the pin dependencies whack-a-mole game let's
bump or base python where possible to python3.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I1cfd7eebb06d0651a1fcefefb81f900aecdef057

5 years agoMerge changes from topic "update-U1804-arm64-base-image"
Andrew Grimberg [Fri, 12 Jul 2019 14:59:20 +0000 (14:59 +0000)]
Merge changes from topic "update-U1804-arm64-base-image"

* changes:
  Fix Ubuntu 18.04 version in var file
  Update Ubuntu 18.04 arm64 base image to latest

5 years agoMerge changes from topic "prettier"
Andrew Grimberg [Fri, 12 Jul 2019 14:56:07 +0000 (14:56 +0000)]
Merge changes from topic "prettier"

* changes:
  Use prettier for auto formatting of JSON/YAML
  Reformat JSON and YAML files with prettier

5 years agoMerge "Update CentOS 7 base image to latest"
Andrew Grimberg [Fri, 12 Jul 2019 14:54:31 +0000 (14:54 +0000)]
Merge "Update CentOS 7 base image to latest"

5 years agoMerge "Update Ubuntu 18.04 amd64 base images"
Andrew Grimberg [Fri, 12 Jul 2019 14:54:20 +0000 (14:54 +0000)]
Merge "Update Ubuntu 18.04 amd64 base images"

5 years agoMerge "Update Ubuntu 16.04 base images to the latest"
Andrew Grimberg [Fri, 12 Jul 2019 14:54:09 +0000 (14:54 +0000)]
Merge "Update Ubuntu 16.04 base images to the latest"

5 years agoMerge "Update Ubuntu 16.04 arm64 base image to latest"
Andrew Grimberg [Fri, 12 Jul 2019 14:53:35 +0000 (14:53 +0000)]
Merge "Update Ubuntu 16.04 arm64 base image to latest"

5 years agoUse prettier for auto formatting of JSON/YAML 43/16143/1
Thanh Ha [Wed, 10 Jul 2019 02:24:40 +0000 (22:24 -0400)]
Use prettier for auto formatting of JSON/YAML

No more debate on which style to use, prettier is an opinionated
style formatter for JSON and YAML files.

Also standardize the pre-commit env name with other lfit projects.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I2655382ba6fb099970418475ad4cf939b11752e5

5 years agoReformat JSON and YAML files with prettier 42/16142/1
Thanh Ha [Wed, 10 Jul 2019 02:25:56 +0000 (22:25 -0400)]
Reformat JSON and YAML files with prettier

See I2655382ba6fb099970418475ad4cf939b11752e5 for implementation
details.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I88c7cd50a03f8816bb78ff54dcf574bb65276794

5 years agoUpdate Virtual HW to v2-highcpu-1 65/16065/5
Tim Johnson [Wed, 3 Jul 2019 16:40:33 +0000 (09:40 -0700)]
Update Virtual HW to v2-highcpu-1

Specify the volume size build agent to 20(GB). The defualt values for
v1-standard-1 were adequate (40GB), but the default values for
v2-highcpu-1 were not.

Issue: RELENG-2158
Change-Id: I6db62229d90573b899947171e3dba83fb00efe71
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoScript Cleanup 96/15996/3
Tim Johnson [Tue, 25 Jun 2019 20:46:48 +0000 (13:46 -0700)]
Script Cleanup

Improved error handling, more succinct output (removed -x flag). Minor
enhancements tox config (.coafile). Changed local variables to
lower-case.

Change-Id: Ice1768f01ed513b7236b7cfa42d4b32886a3c83e
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
5 years agoMerge "Make git2u swap idempotent"
Anil Belur [Thu, 27 Jun 2019 23:02:54 +0000 (23:02 +0000)]
Merge "Make git2u swap idempotent"

5 years agoMake git2u swap idempotent 82/15982/4
Andrew Grimberg [Tue, 25 Jun 2019 00:51:18 +0000 (10:51 +1000)]
Make git2u swap idempotent

Issue: RELENG-2143
Change-Id: Icba128bf8d775edd2b750f05c7662162fcdf9371
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoAdd reno note for aws template fix 99/15999/1 v0.3.1
Andrew Grimberg [Wed, 26 Jun 2019 13:46:16 +0000 (06:46 -0700)]
Add reno note for aws template fix

Change-Id: I4da1dc30a0b8d668f6b5397421bf56bbda241221
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoRemove unknown config value for aws template 98/15998/1
Andrew Grimberg [Wed, 26 Jun 2019 13:28:09 +0000 (06:28 -0700)]
Remove unknown config value for aws template

Change c0dc5f030e259b70533a4d626eae5ab176fdd560 added in an
instance_name variable to the aws template. Unfortunately the
verification which happens in a cloud not using aws as the cloud itself
apparently couldn't pick up the fact that it's not valid with an EC2
definition. Trying to upgrade the LF internal Jenkins to latest
common-packer uncovered this problem.

Change-Id: Id96671cbe674c2af0df7ad4f4babed2b1ee8a4ed
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoFix Ubuntu 18.04 version in var file 95/15995/1
Anil Belur [Tue, 25 Jun 2019 22:23:33 +0000 (08:23 +1000)]
Fix Ubuntu 18.04 version in var file

Change-Id: I5c339d72a97cccc173a30cb1c8f7366ef64d3459
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoUpdate Ubuntu 18.04 arm64 base image to latest 87/15987/2
Anil Belur [Tue, 25 Jun 2019 06:48:49 +0000 (16:48 +1000)]
Update Ubuntu 18.04 arm64 base image to latest

Issue: RELENG-2166
Change-Id: I36fc5cc5289039a6d723ff7740d64d90b2d8364e
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>