Add detailed instructions to setup builder

Provide minimal steps to setup an initial builder template.
Fix broken index link to the usage page.

Change-Id: Ife7c6436b2fd08c356ec0cee632fed6867c83eea
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Revert "Expand image name to include sec and ms"

Images coming online with the sec.ms patch seems to enter
error state. OpenStack the init of the nodes may not like
the "." dot in the VM name.

This reverts commit ce14f52380fd4b9dd707728d645aecf3d7a6d7fa.

Change-Id: Iae802cb4d6ccf7b1c35943fdbd69a2f65418e3b9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Expand rendered image name to include sec and ms

Expand the rendered image name to include seconds
and milli-seconds.

JIRA: RELENG-960
Change-Id: Ic1853ba7566def397b8bce389e2779fe3020c7ce
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix tox coala errors in common-packer

Change-Id: Icde085c0a1f981ff03818216defcc561b3d91c1b
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

ansible-galaxy.sh enable custom requirements.yaml

If applications provide a custom requirements.yaml
the script should use the same, instead of the default requirements.yaml

Change-Id: I4ef8b9d4754b6c11c17101cf33f672feddde4fa5
Signed-off-by: Faseela K <faseela.k@ericsson.com>

Add an Amazon template for building AMIs

The differences between this builder and the openstack one are the cloud
specific options passed, and the disabling of ANSIBLE_PIPELINING.

For some reason ansible is not able to set up the local ssh connection
(ssh <ssh_user>@127.0.0.1) on the EC2 instance after transfering the
playbook files, yet it can successfully ssh to the machine.

Disabling the pipelining support opens a new ssh connection each time
ansible runs a task. This takes more time, but works.

This patch also includes an amazon cloud example var file.

Change-Id: I2b9f273e86fd7bc1ddfde34d26eebfd6249a6a0a
Signed-off-by: Trevor Bramwell <tbramwell@linuxfoundation.org>

Activate linting for RST documents

Change-Id: Ie91caab378e320111962dcb640fe3c10ab02b4e8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Simplify README and remove markdown

Simplify our setup a little by removing markdown and turning the
README into a simple text that links to the Documentation.

Change-Id: I0fbcfadf4623bc1759984ae086c84ef249b7d567
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Migrate LF Network docs to RST

Move the LF Network portion of the README into RST. This provides
details on how to bootstrap through the LF Network for LF Staff
to initialize the first image on a new setup.

Issue: RELENG-513
Change-Id: I85bd8b298bcc330b24b6c3aa40cf71fdef527d50
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Migrate CP usage documentation

Migrate over the existing Common Packer usage documentation
from the README file.

Issue: RELENG-513
Change-Id: Ie7eb81b3a8e69e0cbd878cc2ec603a9c33ac5a23
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Migrate doc on how to install CP to RST

Migrate the Common Packer install documentation into RST.

Issue: RELENG-513
Change-Id: Ibe61a55e9e4857c33173d5c67bc88ed07d449420
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bootstrap common-packer RTD documentation

Issue: RELENG-513
Change-Id: Ic199def610eea4daa3d07713e54e373258bec815
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Remove obsolete section on lf-ansible

Roles come from Ansible Galaxy now.

Change-Id: I39d063e96553bd8b00f2bd2fcaaa14c8098efcbb
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Set ansible galaxy roles path for builder

Change-Id: I2f7700dbb9d7e05b77e8dbb393837dbc29d5a2b1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Update common-packer unstructions

lf-ansible and common-packer instructions with github references need to
call releng-lf-ansible and releng-common-packer to clone the repos.

Change-Id: I432d1896dd06ce5ce5297be211058ea6564c3655
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>

Install roles via Ansible Galaxy

Support using Ansible Galaxy to install lf-ansible roles. This
patch adds a new script ansible-galaxy.sh which will install
roles from common-packer's requirements.yaml and if provided
local project requirements.yaml as well. The project local
requirements.yaml needs to exist in their local packer directory.

Change-Id: I06bb5d433bac37b3c08e761c25dfdf942b4f3229
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Packer bootstrap through LF network instructions

Add instructions for how to get through the LF network when running
packer on a local machine.

Change-Id: I38d90abe6d1e5d01828e0968ac42dad9cce97af6
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Set docker default mtu to 1458 for LF Network

LF Network requires the MTU be set to 1458 so that container
routing will work correctly.

Change-Id: I637ef8f2a0f310af3c1e2a3108d7da61c2570dba
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Remove .example extension from templates"

Merge "Update templates to using local-*.yaml"

Add EC2 filter defintions for future support

EC2 templates have different requirements for the image base definition
that uses filters. As we can have unused variables defined in our vars
files we can just add these extra parameters without causing problems
with OpenStack builds and still support EC2 builds.

Change-Id: I28c773d16ae4102c53416e957509e9f87e7d802b
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Remove .example extension from templates

The templates are no longer examples but actually fully usable
templates so let us stop calling them examples.

Change-Id: I54d54e42359528e5c186a3c082257ce98b1594a9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Update templates to using local-*.yaml

This change should allow us to keep the template files relatively
the same while allowing consumers of builder and docker to have
some local modifications in their own ciman repos.

Change-Id: I43351ea259af5bb55be8895f8cac97d5e90b923b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Revert "Bootstrap Ansible if not available"

This is not actually necessary. We need Ansible in the VM that's
running the test not the VM being built. Which means the VM attached
in to Jenkins needs to have Ansible installed in the first place. We
just need to ensure that Jenkins is initially configured with one
(which we currently have to do anyway even for the regular shell
script provisioner).

This reverts commit 99f875674576f3643c9713750a7017f581ff758b.

Change-Id: I607526f4f6575e2a29c7184e0196e9c92040afd9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add lf-dev-libs role to baseline

Change-Id: Ice5662aa955f7b9b2b166e73e3bfa4ceb6cfdaae
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Bootstrap Ansible if not available"

Fix typo in profile_tasks

Change-Id: I93eb9eda5994501363370c4022eb67264b079ffc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bootstrap Ansible if not available

Check if Ansible is already installed on the system and install
it if necessary.

Change-Id: I8d89cf656642a333b3ba6214d12af9d7187acd59
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Do not remove all of /tmp

Legacy scripts in ODL depend on things being available in /tmp so
let us only clear out specific items.

Change-Id: I92c1174b81e561b67a56874111efe62ec0d584b5
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Remove unnecessary SELinux script"

System reseal scripts need to run as root

Change-Id: I85d1c059a9147f8d9ff6aa55eada912572d75d57
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Enable ansible pipelining

Pipelining speeds up tasks considerably as long as !requiretty is
set in sudoers.

Change-Id: Iaf968efce0b028608b3777b0ff64d21bbaa0bed4
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Enable profiler for time statistics

Enables a profile which providers improved time statistics of
individual job runs.

Change-Id: Ia116089ac3438eedf1ece11cc71b3958afd083bf
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Switch system-reseal script to shell

While it would be nice to do things the Ansible way the "file"
method unnecessarily takes too long to remove files in a loop.
In this case we should just handle the cleanup with a quick script
which will run significantly more quickly.

Change-Id: I09145b1ece235ea523cf30fd9c8e13efdcbdb8fb
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add example cloud-env.json

Change-Id: I49eb8cf29fe6af89d28aa044fbe1d50746ce7273
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Make it possible to locally test common-packer

By symlinking common-packer to itself this makes it possible to test
common-packer from it's own project repo.

Change-Id: I022b63aeab561815da3e8856f195201b668cfed0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Be more specific about files to scan with coala

Because of the common-packer symlink we need to be much more careful
about where packer is scanning things to avoid an infinite scan
cycle.

Change-Id: I7ae75004ee69db2d407ff69cbc9df187bd69d504
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Pin nodeenv 1.1.2"

Update common-packer instructions

Instructions are a little different now that we are using the
Ansible provisioner.

Change-Id: I7296ed75af8f4962c427199a8ad755ec0fccc062
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Pin nodeenv 1.1.2

Last known version that works with multiple tox runs.

Change-Id: Icc157039bf4ee42bf133a7ed8448c5de9c72a66a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Remove unnecessary SELinux script

It turns out CentOS 7 by defaults sets SELinux up the way we want
so remove this unnecessary step.

Change-Id: I9d276f2b4a117c07113d7eeb996b7d6e6a92c3e2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add docker template

Change-Id: I493d653e5d515250580cc44a2c65f07299b800c8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Update relative paths

Packer paths are relative to where it is called from so update the
paths to match where we expect users of common-packer to load the
submodule from.

Change-Id: I03cd26c4fdb7cd2611cd6ae968ddd4470ba33fad
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Initial baseline image (Ansible)

Create a baseline image using Ansible Playbooks. Considering using
Ansible for our Job builds so why not also use it here for packer
builds.

Ansible seems to have support for the things our helper scripts do
which allows us to get rid of them if we use Ansible. Each Ansible
task also validates itself to make sure the commands run through,
something we don't currently do and can better let us know when
something goes wrong.

Issue: RELENG-288
Depends-On: Ibf4e1ff13d93ad59eaea4a0582a4b77446870dc0
Change-Id: I96d1ec83cec82130d30a245c39b6ac8018f92428
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Create README for usage and design of repo

We need a README for how to use common-packer. As we are just getting
this up and running, writing out how the templates and general thought
process of how it should be used is also handy so that others can start
adding to the repository without having to wait for specific examples.

Change-Id: Ifc14d2ede0251a35af85a37e3733f640ed0c84bc
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Add .gitreview and .gitignore

Change-Id: Ia30cd1ddba586e56a277773a91259573733ab7b6
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add Coala Linting

Change-Id: Ie729bb6d22c5ff3e20c938e157914239ae398665
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Initial empty repository