Enable YAML linter and fix lint issues

Resolve relevant lint issues and enable the YAML linter.

Change-Id: I11cb7b6a1edb72f165ba9ba051bc19794e4684bd
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Enable JSON linter and fix lint issues

Fix outstanding JSON lint issues and enable the JSON linter.

Change-Id: I858b194f3d23c6aa2f868cc2f04bbdc93a8732c6
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Make flavor configurable for templates

This is needed to support aarch64 images in vexx as they will
only run on v2 flavor types.

Change-Id: I6edb29431f14df8c3afea44b5d4f19f19c48f837
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>

Fix windows-builder to support local scripts

Original patch for this missed the setup to allow local usage
of the windows-builder provisioner.

Change-Id: I6d8b2b97d7efd844feaf8984154060f1e10b090e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Uninstall unattended-upgrades

We do not need this package in Ubuntu systems so remove it.

Issue: RELENG-1037
Change-Id: I806ab6a6f5bf8e86a9841731433968e18cdd5dde
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Add pre-commit hook for trailing-whitespace"

Add pre-commit hook for trailing-whitespace

Install using tox:

    tox -e check-hooks

ref: https://pre-commit.com

Change-Id: I9e1716a486ae027cd9eb35b415ee700ad0bc60d9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add zip and unzip to Windows builders

Change-Id: I976fcf20600a97d744df408cb48036be4d56c4c8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add centos7 aarch64 image

Change-Id: I4ea3b0690f92d0e70d5653d3918b92989901b000
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>

Install OpenSSH with the SSHAgentFeature

The SSHAgentFeature provides the Windows service necessary to
allow ssh-agent to work properly.

Issue: RELENG-999
Change-Id: I6d5ba7f599dd20e1d2052854efcae66da11cedeb
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Use Ansible provisioner for Windows builds

Issue: RELENG-939 RELENG-999
Change-Id: Ided88b444f76c5461bed313dc06ba79a51139c9c
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Build out Ubuntu 18.04 arm64 image

Change-Id: I9b449a63489577c3f1a52c26bcb765e0f7ee9309
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>

Add EC2 variables for Windows var file

Even though we don't currently have Windows systems in EC2 we need to
have the source_ami_* variables declared as empty strings to pass packer
verification.

Change-Id: Id87435a77827f1c9f35bbb8f54e363b8ddae2b96
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Add Windows Server 2016 builder

This provides a basic "hello world" of sorts and successfully
boots a Windows builder and lists the C:\ directory.

Issue: RELENG-939 RELENG-999
Change-Id: I539afdc08676e3317d823292f0692361ca9c6825
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Properly install git2u on EL systems

git2u needs special handling (yum-plugin-replace) for installation. Make
sure that all steps happen in the correct order

Change-Id: I6c6b5a49031463e02442f3ea8f50f5aea37e4a3d
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Re-order package install

yum-plugin-replace is not available in the base upstream repositories
but from IUS. It must therefore be listed after the IUS repository
installation

Change-Id: I6d3221ca05348f234f2342c3d8e072f6da379786
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

EL7 systems should use git2u

EL7 systems have a very ancient git installed. IUS provides an up to
date version that replaces the upstream version.

Issue: RELENG-987
Change-Id: I41a02037f4d5326ba64c05f081fc5f1aa8fd6e41
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Merge "Expand rendered image name to include sec and ms"

Expand rendered image name to include sec and ms

Expand rendered image name to include seconds and milliseconds.
This reduces the probability of image name clashes in the env.

The initial change Ic1853ba7566def397b8bce389e2779fe3020c7ce was
reverted, since the change had issues bringing up an instance on the
OpenStack cloud provider. This was because the default
`instance_name` value is taken as the `image_name`, when the
instance_name is unset, the instance_name does not allow FQDN format
hostname ending with a period+number.

The workaround is to use separate `instance_name` parameter in your
builder which does not end with period+number.

Tested the change to work:
"ZZCI - CentOS 7 - docker - 20180605-015801.089"

JIRA: RELENG-971
Change-Id: Ia195bd99d55f23b4f39acf590520637e54fc006a
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Add detailed instructions to setup builder

Provide minimal steps to setup an initial builder template.
Fix broken index link to the usage page.

Change-Id: Ife7c6436b2fd08c356ec0cee632fed6867c83eea
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Revert "Expand image name to include sec and ms"

Images coming online with the sec.ms patch seems to enter
error state. OpenStack the init of the nodes may not like
the "." dot in the VM name.

This reverts commit ce14f52380fd4b9dd707728d645aecf3d7a6d7fa.

Change-Id: Iae802cb4d6ccf7b1c35943fdbd69a2f65418e3b9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Expand rendered image name to include sec and ms

Expand the rendered image name to include seconds
and milli-seconds.

JIRA: RELENG-960
Change-Id: Ic1853ba7566def397b8bce389e2779fe3020c7ce
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix tox coala errors in common-packer

Change-Id: Icde085c0a1f981ff03818216defcc561b3d91c1b
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

ansible-galaxy.sh enable custom requirements.yaml

If applications provide a custom requirements.yaml
the script should use the same, instead of the default requirements.yaml

Change-Id: I4ef8b9d4754b6c11c17101cf33f672feddde4fa5
Signed-off-by: Faseela K <faseela.k@ericsson.com>

Add an Amazon template for building AMIs

The differences between this builder and the openstack one are the cloud
specific options passed, and the disabling of ANSIBLE_PIPELINING.

For some reason ansible is not able to set up the local ssh connection
(ssh <ssh_user>@127.0.0.1) on the EC2 instance after transfering the
playbook files, yet it can successfully ssh to the machine.

Disabling the pipelining support opens a new ssh connection each time
ansible runs a task. This takes more time, but works.

This patch also includes an amazon cloud example var file.

Change-Id: I2b9f273e86fd7bc1ddfde34d26eebfd6249a6a0a
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>

Activate linting for RST documents

Change-Id: Ie91caab378e320111962dcb640fe3c10ab02b4e8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Simplify README and remove markdown

Simplify our setup a little by removing markdown and turning the
README into a simple text that links to the Documentation.

Change-Id: I0fbcfadf4623bc1759984ae086c84ef249b7d567
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Migrate LF Network docs to RST

Move the LF Network portion of the README into RST. This provides
details on how to bootstrap through the LF Network for LF Staff
to initialize the first image on a new setup.

Issue: RELENG-513
Change-Id: I85bd8b298bcc330b24b6c3aa40cf71fdef527d50
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Migrate CP usage documentation

Migrate over the existing Common Packer usage documentation
from the README file.

Issue: RELENG-513
Change-Id: Ie7eb81b3a8e69e0cbd878cc2ec603a9c33ac5a23
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Migrate doc on how to install CP to RST

Migrate the Common Packer install documentation into RST.

Issue: RELENG-513
Change-Id: Ibe61a55e9e4857c33173d5c67bc88ed07d449420
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bootstrap common-packer RTD documentation

Issue: RELENG-513
Change-Id: Ic199def610eea4daa3d07713e54e373258bec815
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Remove obsolete section on lf-ansible

Roles come from Ansible Galaxy now.

Change-Id: I39d063e96553bd8b00f2bd2fcaaa14c8098efcbb
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Set ansible galaxy roles path for builder

Change-Id: I2f7700dbb9d7e05b77e8dbb393837dbc29d5a2b1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Update common-packer unstructions

lf-ansible and common-packer instructions with github references need to
call releng-lf-ansible and releng-common-packer to clone the repos.

Change-Id: I432d1896dd06ce5ce5297be211058ea6564c3655
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>

Install roles via Ansible Galaxy

Support using Ansible Galaxy to install lf-ansible roles. This
patch adds a new script ansible-galaxy.sh which will install
roles from common-packer's requirements.yaml and if provided
local project requirements.yaml as well. The project local
requirements.yaml needs to exist in their local packer directory.

Change-Id: I06bb5d433bac37b3c08e761c25dfdf942b4f3229
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Packer bootstrap through LF network instructions

Add instructions for how to get through the LF network when running
packer on a local machine.

Change-Id: I38d90abe6d1e5d01828e0968ac42dad9cce97af6
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Set docker default mtu to 1458 for LF Network

LF Network requires the MTU be set to 1458 so that container
routing will work correctly.

Change-Id: I637ef8f2a0f310af3c1e2a3108d7da61c2570dba
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Remove .example extension from templates"

Merge "Update templates to using local-*.yaml"

Add EC2 filter defintions for future support

EC2 templates have different requirements for the image base definition
that uses filters. As we can have unused variables defined in our vars
files we can just add these extra parameters without causing problems
with OpenStack builds and still support EC2 builds.

Change-Id: I28c773d16ae4102c53416e957509e9f87e7d802b
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Remove .example extension from templates

The templates are no longer examples but actually fully usable
templates so let us stop calling them examples.

Change-Id: I54d54e42359528e5c186a3c082257ce98b1594a9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Update templates to using local-*.yaml

This change should allow us to keep the template files relatively
the same while allowing consumers of builder and docker to have
some local modifications in their own ciman repos.

Change-Id: I43351ea259af5bb55be8895f8cac97d5e90b923b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Revert "Bootstrap Ansible if not available"

This is not actually necessary. We need Ansible in the VM that's
running the test not the VM being built. Which means the VM attached
in to Jenkins needs to have Ansible installed in the first place. We
just need to ensure that Jenkins is initially configured with one
(which we currently have to do anyway even for the regular shell
script provisioner).

This reverts commit 99f875674576f3643c9713750a7017f581ff758b.

Change-Id: I607526f4f6575e2a29c7184e0196e9c92040afd9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add lf-dev-libs role to baseline

Change-Id: Ice5662aa955f7b9b2b166e73e3bfa4ceb6cfdaae
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Bootstrap Ansible if not available"

Fix typo in profile_tasks

Change-Id: I93eb9eda5994501363370c4022eb67264b079ffc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bootstrap Ansible if not available

Check if Ansible is already installed on the system and install
it if necessary.

Change-Id: I8d89cf656642a333b3ba6214d12af9d7187acd59
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Do not remove all of /tmp

Legacy scripts in ODL depend on things being available in /tmp so
let us only clear out specific items.

Change-Id: I92c1174b81e561b67a56874111efe62ec0d584b5
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Remove unnecessary SELinux script"

System reseal scripts need to run as root

Change-Id: I85d1c059a9147f8d9ff6aa55eada912572d75d57
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Enable ansible pipelining

Pipelining speeds up tasks considerably as long as !requiretty is
set in sudoers.

Change-Id: Iaf968efce0b028608b3777b0ff64d21bbaa0bed4
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Enable profiler for time statistics

Enables a profile which providers improved time statistics of
individual job runs.

Change-Id: Ia116089ac3438eedf1ece11cc71b3958afd083bf
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Switch system-reseal script to shell

While it would be nice to do things the Ansible way the "file"
method unnecessarily takes too long to remove files in a loop.
In this case we should just handle the cleanup with a quick script
which will run significantly more quickly.

Change-Id: I09145b1ece235ea523cf30fd9c8e13efdcbdb8fb
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add example cloud-env.json

Change-Id: I49eb8cf29fe6af89d28aa044fbe1d50746ce7273
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Make it possible to locally test common-packer

By symlinking common-packer to itself this makes it possible to test
common-packer from it's own project repo.

Change-Id: I022b63aeab561815da3e8856f195201b668cfed0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Be more specific about files to scan with coala

Because of the common-packer symlink we need to be much more careful
about where packer is scanning things to avoid an infinite scan
cycle.

Change-Id: I7ae75004ee69db2d407ff69cbc9df187bd69d504
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Pin nodeenv 1.1.2"

Update common-packer instructions

Instructions are a little different now that we are using the
Ansible provisioner.

Change-Id: I7296ed75af8f4962c427199a8ad755ec0fccc062
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Pin nodeenv 1.1.2

Last known version that works with multiple tox runs.

Change-Id: Icc157039bf4ee42bf133a7ed8448c5de9c72a66a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Remove unnecessary SELinux script

It turns out CentOS 7 by defaults sets SELinux up the way we want
so remove this unnecessary step.

Change-Id: I9d276f2b4a117c07113d7eeb996b7d6e6a92c3e2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add docker template

Change-Id: I493d653e5d515250580cc44a2c65f07299b800c8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Update relative paths

Packer paths are relative to where it is called from so update the
paths to match where we expect users of common-packer to load the
submodule from.

Change-Id: I03cd26c4fdb7cd2611cd6ae968ddd4470ba33fad
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Initial baseline image (Ansible)

Create a baseline image using Ansible Playbooks. Considering using
Ansible for our Job builds so why not also use it here for packer
builds.

Ansible seems to have support for the things our helper scripts do
which allows us to get rid of them if we use Ansible. Each Ansible
task also validates itself to make sure the commands run through,
something we don't currently do and can better let us know when
something goes wrong.

Issue: RELENG-288
Depends-On: Ibf4e1ff13d93ad59eaea4a0582a4b77446870dc0
Change-Id: I96d1ec83cec82130d30a245c39b6ac8018f92428
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Create README for usage and design of repo

We need a README for how to use common-packer. As we are just getting
this up and running, writing out how the templates and general thought
process of how it should be used is also handy so that others can start
adding to the repository without having to wait for specific examples.

Change-Id: Ifc14d2ede0251a35af85a37e3733f640ed0c84bc
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Add .gitreview and .gitignore

Change-Id: Ia30cd1ddba586e56a277773a91259573733ab7b6
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add Coala Linting

Change-Id: Ie729bb6d22c5ff3e20c938e157914239ae398665
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Initial empty repository