Release lftools v0.13.1

Change-Id: Ib62dedc9f4d3c23a88a7bc86733497505cc79151
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Pin python-jenkins to 0.4.16

It seems python-jenkins 1.0.0 broke the run_scripts() function which
we use to run the jenkins-cfg commands. Pin to the older version
until we figure out why.

Change-Id: I9e643e3aafbc9eb680fd3b8181a0d8b4806060ac
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Set default branch to v0.13.x

Change-Id: Idca6e313059d9e303db084509b880c12ea674ca7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Release lftools v0.13.0

Change-Id: If195384c5e9c624d61aa7f1ecccf9268d134c1b3
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Simplify Nexus 2 repo target regex

Add unit test to check real world artifact paths against the regex
pattern.

Issue: RELENG-758
Change-Id: I580fef69ef03d9cf85c59da250fdfee66fcb7ad5
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Make OpenStack requirements optional

Users can now `pip install lftools[openstack]` as an optional
dependency so that we don't have to download the entire openstack
shade depchain.

Change-Id: Icf0065eeb1bf97f92560434edec3576397223af7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge changes I84773de5,I91de27b2

* changes:
  Add test for latest upstream of dependencies
  Move tests requirements to separate file

Merge "Fail Sphinx build even on warnings"

Add test for latest upstream of dependencies

Useful for identifying potential issues if we were to bump versions
of upstream projects that we have pinned.

Change-Id: I84773de551f2bb882c14e5531657c0a37fce70c7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Move tests requirements to separate file

Will be used later so we can add additional tests that depend on
this file.

Change-Id: I91de27b2024050894ba5ddeef5ec849074811f9b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Remove trailing slashes in nexus_url

lftools should not fail if the user passes extra trailing slashes that
the command was not expecting.

Change-Id: I9a32c48ddd5ee2898816db7faaeb0de805a38b3c
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Pin requests and python-jenkins"

Add pytest cache to .gitignore

Change-Id: Ie3ff833149896f6beab8eb137f049a288538ff90
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Pin requests and python-jenkins

Change-Id: Ic830bad231d98efe721a299f18f634d87aad5e9f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Fail Sphinx build even on warnings

Change-Id: I935e57b4536f58eb71b49d6558fefc170d50a8da
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bump version to 0.13.0-dev

Change-Id: I8f643e0a9b6fefce7c723d4b46b955df4334dbb8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Release lftools v0.12.4

Change-Id: If4fba3ca0b18bf785ed421ff42c20f20f608f9ca
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Fix remove volume time format

Created_at also uses a different time format that needs to be
addressed.

Change-Id: I5618a5f1ce3fdf0d49ce01cbad47683cf8e02762
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bump version to 0.13.0-dev

Change-Id: Ie73dfc81edc257dea06450bd236d98746cd35290
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Release lftools v0.12.3

Change-Id: Ie38022c49c78a5a1ebe12ba596b339b3a9e110b7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Fix volume remove command polling wrong metadata

Volumes use created_at instead of created.

Change-Id: I53a11dc791ee2d4035f9b641f97c019f2f19610f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bump version to 0.13.0-dev

Change-Id: Ice5411654c110e4630c1e8d95796c5bfa02f40b8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Release lftools v0.12.2

Change-Id: I76af823e2c391e3ecf4ab7f5738a642cd554401f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add support for volume management

Issue: RELENG-853
Change-Id: Id412315238abfdea9f655dfeb2957964b37d5f19
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bump version to 0.13.0-dev

Change-Id: Iac54f1c832e159fc960a18de4191052a67ba82dd
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Release lftools v0.12.1

Change-Id: I9aed03e79c38ef2823722e3a76be4bb45f47b549
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Pin netifaces to 0.10.5

netifaces decided to release a repackaging of 0.10.6 this morning
that broke us. Let us add them to the pinned deps list to resolve
the issue.

Change-Id: Id67c3bdc9e7da252892c1a1e7514dc25e7c2ae05
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bump version to 0.13.0-dev

Change-Id: I90cd117ed3d436cbebb38e359b7e32b23df157f1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Release lftools v0.12.0

Change-Id: Ifa0a9344badb1ca942707b833cd71a77dc3f4a9e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add lftools command to remove specific server

This command also supports an optional minutes parameter which
can be used to ignore the server if it is not older than a certain
number of minutes.

This is useful for cases like Jenkins orphan server cleanup scripts
so that they do not try to remove servers that have just been
spawned and have not yet connected to Jenkins causing the server to
be mistakenly marked as orphaned.

Issue: RELENG-780
Change-Id: I7f12218e0ac730ff43def4ec9b6133b4cc8ee0c8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bump version to 0.12.0-dev

Change-Id: Ie320efc1e9917751505b071df0cc59e09791d1d1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Release lftools v0.11.2

Change-Id: I0361a50e1aa239e97185809124cd8df8be9b8b56
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

upload_to_nexus to return a non-zero exit status

As it turns out Nexus does not produce a http status code on upload
failure and returns back a blank message. This means if we do not
detect a successful status from Nexus we need to exit 1 explicity
to make sure we return to parallel a proper error status code.

JIRA: RELENG-765
Change-Id: Ic221c668bb6cd35befc614159ee8d7220d3f4cfe
Co-authored-by: Thanh Ha <thanh.ha@linuxfoundation.org>
Signed-off-by: Robert Varga <nite@hq.sk>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Switch to using lfdocs-conf 0.1.3

Change-Id: If0ad2ba50ffa6f5ecdb0fb3a06563706b56c4015
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add a groovy command to lftools

Using python-jenkins' run_groovy API to allow us to execute
groovy scripts from lftools.

Change-Id: If5afb69b738a0cee6ee653b5d70ba9b5dac67fd0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Switch lftools to lfdocs-conf

Consolidate conf.py into lfdocs-conf project.

Change-Id: I51c0931d5bc03cc71d10edca73a46379cf02ac2e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Improve debugging output for deploy nexus-zip cmd

Print out the HTTP Status code as well as the http response from
the server (if any, Nexus doesn't always return any) and the
contents of the zip file on failure.

Issue: RT-50048
Change-Id: I54014819d18c0090c45fd8378954dc9aa5e540f8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Fix ShellCheck v0.4.7 lint issues

v0.4.7 introduces some new rules which now need to be handled.

Change-Id: I9c769bd819b787d817ef7708c312e9c637ee7dfc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bump version to v0.12.0-dev

Change-Id: Id5606835c44a5771025154b732aa49343627a922
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Release lftools v0.11.1

Change-Id: I4eb27f0dbb3ccb2632b93be0f09830755ff67125
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Fix wget command for Nexus sign script"

Fix wget command for Nexus sign script

Add --level=15 to wget command that fetches the
autoreleases from Nexus. Looks like the previous
command wasn't fetching all the artifacts in the
autorelease tree.

Change-Id: Ieb49331d7b192187ba2e6783b334d90c6477503e
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>

Print CPU, Memory, Disk and Network stats

Print sar CPU, Memory, Disk, Network statistics at end of console log

JIRA: RELENG-672
Change-Id: I42aae6a07a91e7a666b28fcd027a5c0a91a8e64c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Bump version to v0.12.0-dev

Change-Id: I37dfe4d790490aa3d40216c7034c77f3d830a5a0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Release lftools v0.11.0

Change-Id: If40b6319e21f6d42eb770a22de2b93677a6eb87f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Automatically hide protected images

We are not able to delete protected images so automatically filter
them out of our results.

Issue: RELENG-705
Change-Id: Id7fa5ab73ec912aac9b04dfb30c29492320402f2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Update openstack cmd to support ci_managed filter

Add support for filtering ci_managed metadata when querying for
images.

Issue: RELENG-705
Change-Id: I9570a0683e1d87547bda8fc6741369549450045e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Suppress GNU parallel notice on every run

By default the notice is output on every run, therefore
use the `--no-notice` option to supress the output.

Change-Id: Id1dbe7b8daee390c886486eeb3c4c3c436fc26e4
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Add a snapshot mode to the deploy-nexus command

This is necessary for RELENG-61 in order to allow pushing
maven-metadata.xml files which in the case of release jobs should not be
allowed. So add an option to the deploy-nexus command to allow an
exception for snapshot artifacts.

Issue: RELENG-173
Change-Id: I6f086bd0987502c5a967ab0b6f313ac405ceabc5
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Fail gracefully when no files to sign

Fixes issue where if there are no files to sign detected the script just
hangs indefinitely. Instead if the files_to_sign array is empty then
exit 1 and inform user.

Change-Id: I82a5957faedfe739800305a8224fed67d2c74801
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Use parallel to sign when available

Speed up the signing process by using parallel when available.

Change-Id: I8a4bebde397b59491039ef6591796f3bfb1db2e8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bump version to v0.11.0-dev

Change-Id: Ia83890a4ee3c2119d18d6e50845e07b487068002
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Release lftools v0.10.0

Change-Id: I0e203548adad9057d58d7893ba86eb594cdddfea
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Rename docs requirements for better sorting

Change-Id: Ie1fb5e86aa22d49c7d144f85a6f1fbf0340360f8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add LF Logo

Change-Id: Ia82ca35196630689f67f8b1b6fb5f2091a6e5c6d
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add custom navbar ala OpenDaylight docs

Change-Id: I2af32b6847a6dd1242efd6b9c320a22eff0d8d76
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bump Sphinx minimal dependencies to latest

Also splits out docs requirements to docs-requirements.txt to avoid
coala failing to run because of version conflict with Sphinx version we
want to use for docs.

Change-Id: I64d0b5953be03f91bf67e75e54072d1da9df5d1a
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Remove best practices guide from lftools

The guide sections were divided out and moved to either releng/docs or
releng/global-jjb depending on context.

Issue: RELENG-507
Change-Id: I4eb4b0fe2820415a3e4bda8a98b2a2d5fed3f78f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add cross link

Issue: RELENG-507
Change-Id: I925889331d7904f2b85e5a16a1c581b32893779b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Update setuptools version

Hyperledger builds fail on the Z platform with versions
of setuptools < 30.

FAB-6595

Change-Id: I6889507870f66e6b44720579f8be7b0b35b7c7a9
Signed-off-by: Ry Jones <rjones@linuxfoundation.org>

Refactor descriptions to README.md and __summary__

This change makes it so that the __summary__ is a pydoc variable and we
can pull in the long description directly from README.md. Also add a
website url.

Change-Id: Ibab9e436d4e2c8188a456976e1ab5f19dbf88060
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Re-order log ENV stripping and add new key

* Make the environment stripping be asciibetical so it's easier to know
where to insert new elements

* Add KEY as an environment variable base to be stripped from the
shipped logs

Change-Id: I46367ec36c44b5b29534fff7f86356e8b9abffcb
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Merge "Update JJB best practices"

Update JJB best practices

Using single quotes around variables.

Update parameter passing section to avoid using `!include-raw`.

Also remove single quotes around examples where its not required.

Change-Id: I176c861a90fcea29c3fd77308287f241e9f26a6c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Spell check fixes in best practices

Change-Id: Ibcc86f61181edcfad09cc1cdc056b1b4c8a09954
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Bump version to 0.10.0-dev for next dev cycle

Change-Id: I784869e966f1bb12842663b9386c275dacc59fb6
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Release lftools v0.9.0

Change-Id: I34c8b9db49b7942329f911e5671fd7dff84402dd
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Exit script with error if artifacts fail download

Improve error checking to handle a potential error case.

Change-Id: I8ac39a0c5c84f6b23ea60a5c8e6601be76bf6afc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Use gpg2 if available

Prefer gpg2 over gpg if it is available. In some systems such as Arch
Linux gpg is gpg2. We really only support gpg2 for signing.
Minor fix indentation to 4 spaces like the rest of the script.

Change-Id: I6733521838c78da94b67ab81d9397f5b0622e283
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Add a best practice for JJB templates"

DO NOT archive if filename is longer than 255

Most Unix filesystems do not allow files being created that are longer
than 255 characters. The LF Infra VMs are built with XFS and seem to
have the extended attribute allowing longer filenames. This
unfortuantely creates zip that when we copy to Nexus which does not have
this to fail to unpack. Limit archive files to 255 characters or less to
avoid this.

Issue: RELENG-322
Change-Id: I9c70a67d69559b834ec1c1bce0c2b909e61127dd
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add a best practice for JJB templates

Having maintained global-jjb for awhile we have realised some good
design patterns that makes maintaining the global-jjb project easier.
This is an attempt to document these as best practices.

Issue: RELENG-414
Change-Id: I773e227d47a532499bad643f771c88edd45af236
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Add suport for .tgz and .gz file extensions

Handle .tgz and .gz files which are usually shortened name fir .tar.gz.

Jira: releng-431
Change-Id: I06fbb701d28eb1e5958698afbcffc0c504968672
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Set a default logger if file methods fail

In some environment such as ReadTheDocs it appears that pyprefix does
not appear to work as a backup. Let's fall back to a hardcoded logger in
the worst case situation.

Change-Id: Id3389fb05b7386f6d79026cc111982dac6e20efa
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bump version to 0.9.0-dev

Change-Id: I3600d0b5aaa73a3e32da9437305979efe5e06483
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Set version to 0.8.0 for release

Change-Id: I5503409a5a069fc6675442e33664ec14003f5272
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Release lftools v0.8.0

Change-Id: I7348e33a27434ab6355aec09a7cb5a7beec35914
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Add all release notes up to date"

Add all release notes up to date

Keeping a history for our release notes would be useful for folks to
understand what changed since the version the are using.

Change-Id: I7eebbf8fc49be87facd0f87291f70d544accfe02
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Re-factor deploy maven file to remove dup code

Change-Id: I2a4ddd37eff6da11eb04b91ceb96810e39d218aa
Jira: releng-360
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Convert license print statements to logger

Change-Id: I0597053944aef49b9537fd8d492781fa76c05ad0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>

Merge "Fix best practice doc example code indent"

Fix best practice doc example code indent

Change-Id: I61362fbba37c85713468b7af1eadd132f768b16f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Verify lftools scripts have license headers

Add a test to check for license headers in Python and Shell code.

Change-Id: I61b82d463105fca9bb2a0385ca18482ec0fea7c0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Bump version to 0.8.0-dev

Change-Id: Iadfdcdf72d4f6bfb51e336fb3013c3488eb43560
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Release lftools v0.7.0

Noteable Changes:

f3b7292 Jessica Wagantall       Implement lftools logger
f16b45d Anil Belur      Make maven deploy-file more generic
c4084d7 Anil Belur      Pass maven parameters to deploy maven-file
2f7fc1a Jessica Wagantall       Add script to fix yamllint errors
8f0d84c Thanh Ha        Add a license header scanner

Change-Id: I3b25d66720120d79180d0241e625305edf095e45
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Document installation of lftools

Change-Id: I762b8fc6cd82b786c81eb9cc3a0711030a0d1ee5
JIRA: RELENG-325
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>

Use logger for Nexus create repo script

Nexus create repo needs to output what it's
doing into a log file.
Use the generic logger to obtain that output.

JIRA: RELENG-253

Change-Id: I1705b52761f79a1331032175c49484c9a75f9810
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>

Implement lftools logger

Add generic logger configuration file to be used by lftools scripts.

This logger checks 3 locations for a logger.ini file:

1) ~/.config/lftools/logging.ini
2) /etc/lftools/logging.ini
3) sys.prefix/etc/logging.ini

Dev only...
4) etc/logging.ini (Default fallback in dev environments)

JIRA: RELENG-321
Change-Id: I8fef663dd6b02947f7722424ecce6e2cc78debec
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>

Ignore shebang when evaluating license headers

The shebang lines in some scripts should be ignored when evaluating
license headers.

Change-Id: I7d895cf6b00a6edf8303c229278a262129ec2661
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Use a regex pattern instead for license searching

This allows us to search for files for example in shell/ where there are
no file extensions.

Change-Id: I5634b233d9feb0d2dc3421b0c1a9c5280faf82f1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Merge "Make maven deploy-file more generic"

Trim trailing slashes from URL inputs

Trailing slashes in URLs causes the staging code to upload to
//org.opendaylight.project instead of /org.opendaylight.project which
fails our URL pattern matching target defined in Nexus since Nexus
thinks // as a blank directory in the path. We need to make sure
extraneous / does not get pushed in by the user so trim it off if they
do.

Issue: RELENG-351
Change-Id: I264fec5c41e89d588e03a9c447917ee8a3623c62
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>

Make maven deploy-file more generic

Add support for war, jar and tar.gz files
Add support for passing pom files with -f

JIRA: releng-201
Change-Id: Ia5da1e5df23eddd803877f5b42bc8a594cbe58c1
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Merge "Add a license header scanner"

Merge changes from topic '5663'

* changes:
  Add CONTRIBUTING file
  Enable MarkDown linting

Merge "Pass maven parameters to deploy maven-file"

Pass maven parameters to deploy maven-file

Change-Id: I361a735779a3441551bc619cc57ff5db762cc5d5
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Add script to fix yamllint errors

Run this script as "./fix_yamllint.sh" inside the desired repo.
This script will ask the option of running for all *.yaml files or
specific ones for which the user will need to provide the complete
path per file separated by spaces.

Change-Id: I63f200fd9289f12715d354c95af18b619e4911a2
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>

Add CONTRIBUTING file

CONTRIBUTING files are useful to notify potential developers of how to
contribute to our project. Especially if they found it via the GitHub
mirror.

Change-Id: I707332e1b6d5135fb100481645136bbbbe8c37db
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>