releng/lftools.git
6 years agoUse logger instead of print in version module 98/12398/3
Thanh Ha [Mon, 27 Aug 2018 01:25:24 +0000 (21:25 -0400)]
Use logger instead of print in version module

Change-Id: I31329886088d8d7a469e19bc0937912546400109
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoUpdate config subsystem to use logger 97/12397/4
Thanh Ha [Mon, 27 Aug 2018 00:58:29 +0000 (20:58 -0400)]
Update config subsystem to use logger

Use our logger service in the config subsystem.

Change-Id: I47235cf9accd362ba1355ff77e15757543d30da1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoOnly print extra log details in DEBUG mode 90/12390/8
Thanh Ha [Sat, 25 Aug 2018 01:04:13 +0000 (21:04 -0400)]
Only print extra log details in DEBUG mode

This allow us to change all our print statements to log.info() and
have it still print sanely like a CLI application. Then when DEBUG
mode is enabled it will print the extra details to let us
troubleshoot. ERROR and CRITICAL logs will prefix with "ERROR: "
for example.

Change-Id: Ic35222ee654306e686052c209b641f9e49d38144
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add support to jenkins cmd for jenkins_jobs.ini"
Thanh Ha (zxiiro) [Mon, 27 Aug 2018 22:23:43 +0000 (22:23 +0000)]
Merge "Add support to jenkins cmd for jenkins_jobs.ini"

6 years agoMerge "Add ruamel.yaml and yaml as known 3rdparty"
Thanh Ha (zxiiro) [Mon, 27 Aug 2018 22:23:34 +0000 (22:23 +0000)]
Merge "Add ruamel.yaml and yaml as known 3rdparty"

6 years agoMerge "Enable or disable Jenkins jobs matching regex"
Thanh Ha (zxiiro) [Mon, 27 Aug 2018 17:03:09 +0000 (17:03 +0000)]
Merge "Enable or disable Jenkins jobs matching regex"

6 years agoEnable or disable Jenkins jobs matching regex 82/12382/8
Anil Belur [Fri, 24 Aug 2018 04:22:11 +0000 (09:52 +0530)]
Enable or disable Jenkins jobs matching regex

ex:
To enable jobs

To disable jobs

JIRA: RELENG-1117
Change-Id: Iff800472cecfe056285f07e1c2461775457cb0ab
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoAdd support to jenkins cmd for jenkins_jobs.ini 91/12391/2
Thanh Ha [Sat, 25 Aug 2018 05:23:08 +0000 (01:23 -0400)]
Add support to jenkins cmd for jenkins_jobs.ini

Allow the jenkins command to use existing jenkins_jobs.ini for
configuration. By default assume jenkins_jobs.ini's default
configuration 'jenkins'.

This change makes 'server', 'user', and 'password' parameters no
longer required as defaults replace the need for these options.

'server' now defaults to 'jenkins' if jenkins_jobs.ini exists,
otherwise defaults to 'http://localhost:8080' which is the default
Jenkins deploy port.

'user' now defaults to 'admin' which is the Jenkins default admin
user.

'password' has no default and must be set by the user as this
is randomly generated by Jenkins at boot.

Change-Id: Ic7af1eb53e8e9079b845203f44914310616c4bab
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd ruamel.yaml and yaml as known 3rdparty 92/12392/1
Thanh Ha [Sat, 25 Aug 2018 05:38:40 +0000 (01:38 -0400)]
Add ruamel.yaml and yaml as known 3rdparty

Change-Id: I954835b3e1a1700119d2e2328444b7794e7365fc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMake ldap module optional 84/12384/3 v0.16.1
Thanh Ha [Fri, 24 Aug 2018 15:38:27 +0000 (11:38 -0400)]
Make ldap module optional

Install via 'pip install lftools[ldap]'.

Change-Id: Ie2206b43b094e1576936ba934268dc0b68fc5a40
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRemove summary/version which comes from PBR now 83/12383/2 v0.16.0
Thanh Ha [Fri, 24 Aug 2018 14:38:04 +0000 (10:38 -0400)]
Remove summary/version which comes from PBR now

Change-Id: I19b0358a85beb281a07df1ef8ea84c8c7593927f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Extend lftools with scripts for ldap lookups"
Thanh Ha (zxiiro) [Fri, 24 Aug 2018 14:26:48 +0000 (14:26 +0000)]
Merge "Extend lftools with scripts for ldap lookups"

6 years agoAdd option to enable debug logs 79/12379/4
Thanh Ha [Fri, 24 Aug 2018 01:36:46 +0000 (21:36 -0400)]
Add option to enable debug logs

Default the logger to INFO level and add an option to turn on DEBUG
level logs. Update the unit_tests to pass in a context since we need
it now in order to support the top-level configuration.

Change-Id: I253a127f7dc0cee5475d8403da8f82e018fdf36e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSimplify logging configuration 78/12378/3
Thanh Ha [Fri, 24 Aug 2018 01:23:18 +0000 (21:23 -0400)]
Simplify logging configuration

lftools should be a simple tool and folks should not ever need to
configure the logger so let's simplify things and just pre-define
an acceptable logger configuration. The important component is
having (DEBUG) messages print via logger service. A subsequent
patch will be provided to enable --debug level logs via CLI.

Change-Id: I74158d94400112161ad4fc206426420d908deb0c
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd extras declaration in docs-linkcheck 81/12381/1
Thanh Ha [Fri, 24 Aug 2018 02:05:54 +0000 (22:05 -0400)]
Add extras declaration in docs-linkcheck

This should have been added to
I1f33c65a44cf6eaa0e29e5734f3a8edcd8865868

Also include missing reno notes.

Change-Id: I132bebe298164419fae43e94299f43c45940bc4e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoFix broken help from openstack and sign cmds 80/12380/1
Thanh Ha [Fri, 24 Aug 2018 02:05:54 +0000 (22:05 -0400)]
Fix broken help from openstack and sign cmds

These were never caught since we did not enable fail on warnings
for Sphinx until recently.

Change-Id: I1f33c65a44cf6eaa0e29e5734f3a8edcd8865868
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoExtend lftools with scripts for ldap lookups 58/11558/46
Aric Gardner [Tue, 26 Jun 2018 19:16:08 +0000 (15:16 -0400)]
Extend lftools with scripts for ldap lookups

Add 2 new commands to lftools:

    - infofile
    - ldap

prereqs:
For ldap lookups to work you must be on the VPN and have the cert
to get the cert: log in to any collab system and grab /etc/ipa/ca.crt
in /etc/openldap/ldap.conf

TLS_REQCERT always
TLS_CACERTDIR /etc/openldap/certs
TLS_CACERT /etc/openldap/certs/ca.crt

To test:

1. Clone this patchset
2. start venv

   pip uninstall lftools && pip install -e .

Usage:

$ lftools ldap
Usage: lftools ldap [OPTIONS] COMMAND [ARGS]...

  LDAP TOOLS.

Commands:
  autocorrectinfofile  Verify INFO.yaml against LDAP group.
  csv                  Query an Ldap server.
  inactivecommitters   Check committer participation.
  yaml4info            Build yaml of commiters for your INFO.yaml.

$ lftools infofile
Usage: lftools infofile [OPTIONS] COMMAND [ARGS]...

  INFO.yaml TOOLS.

Commands:
  get-committers    Extract Committer info from INFO.yaml or LDAP...
  sync-committers   Script to insert missing values from ldap...

Issue: RELENG-407
Change-Id: I4f4055441042d790008754bb085447f52e1c1a78
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
6 years agoUse reno for release notes 30/12330/2
Thanh Ha [Mon, 20 Aug 2018 16:55:54 +0000 (12:55 -0400)]
Use reno for release notes

Change-Id: Iadb0ca93a6f1f6a56c948b42bd257040547fffcc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSetup PBR for lftools 07/12307/3
Thanh Ha [Fri, 17 Aug 2018 18:46:33 +0000 (14:46 -0400)]
Setup PBR for lftools

Change-Id: I028540301560b76af1cb31f2d9c40612b3b3480f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRemove unnecessary echo 17/12017/2
Thanh Ha [Fri, 27 Jul 2018 15:07:44 +0000 (11:07 -0400)]
Remove unnecessary echo

Since Sphinx 1.7.x, Sphinx itself now prints the output location
at the end of a build so there's no need to do this unnecessary echo
anymore.

Change-Id: I517dea4686b68140be4ab672d6e909e446919163
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump to 0.16.0-dev 31/12031/1
Thanh Ha [Mon, 30 Jul 2018 15:33:03 +0000 (11:33 -0400)]
Bump to 0.16.0-dev

Change-Id: Ia255aefd6e1a05d2fb8440584406b6d8fbe5f2f3
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease lftools v0.15.0 19/12019/3 v0.15.0
Thanh Ha [Fri, 27 Jul 2018 15:47:18 +0000 (11:47 -0400)]
Release lftools v0.15.0

Change-Id: I3a3196f64cefabb390315a85e277d4652c9b30cb
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Handle snapshot artifacts in get_file_info()"
Thanh Ha [Mon, 30 Jul 2018 15:16:07 +0000 (15:16 +0000)]
Merge "Handle snapshot artifacts in get_file_info()"

6 years agoHandle snapshot artifacts in get_file_info() 29/12029/2
Anil Belur [Mon, 30 Jul 2018 02:29:22 +0000 (12:29 +1000)]
Handle snapshot artifacts in get_file_info()

This fixes lftools deploy maven-file to upload snapshot
artifacts to Nexus.

Issue: RELENG-1098
Change-Id: I5ed1c030b2b9c8f62e7f7e4ee0a63858782a747c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoAdd utility to check repos for commits missing DCO 27/11927/25
Jeremy Phelps [Mon, 23 Jul 2018 19:04:30 +0000 (14:04 -0500)]
Add utility to check repos for commits missing DCO

This only works on git repositories.

Change-Id: I45a0a66dfaaf517cdc5d85fc15f60e403bccfeb9
Co-Authored-By: Eric Ball <eball@linuxfoundation.org>
Co-Authored-By: Thanh Ha <thanh.ha@linuxfoundation.org>
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
6 years agoAdd optional classifier def to deploy cmd 38/11938/1
Thanh Ha [Mon, 23 Jul 2018 22:45:09 +0000 (18:45 -0400)]
Add optional classifier def to deploy cmd

The artifact/maven/content API supports an optional classifier
parameter so add option for it.

Change-Id: Ibf89ac2fb15e7fe289cb76aafad41936b24f3a9f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd cURL command to push an artifact to Nexus 44/11844/5
Thanh Ha [Wed, 18 Jul 2018 18:30:36 +0000 (14:30 -0400)]
Add cURL command to push an artifact to Nexus

Issue: RELENG-1081
Change-Id: If36810924d2d0f91e04bae469067495a25c0b2dc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSplit Nexus staged repo create & close functions 43/11843/8
Thanh Ha [Wed, 18 Jul 2018 18:05:44 +0000 (14:05 -0400)]
Split Nexus staged repo create & close functions

Split out these functions so that they can be reused.

Issue: RELENG-1081
Change-Id: I5a5c2ee10ed98f7c0c25134298e5a10bb7b23dd1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd pre-commit hook for trailing-whitespace 32/11832/1
Thanh Ha [Wed, 18 Jul 2018 01:31:17 +0000 (21:31 -0400)]
Add pre-commit hook for trailing-whitespace

Install using tox:

    tox -e check-hooks

ref: https://pre-commit.com

Change-Id: I2423e9619870b1b97c0082cd54aad06479dcf172
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Bump version to 0.15.0-dev for next dev cycle"
Andrew Grimberg [Mon, 16 Jul 2018 22:26:49 +0000 (22:26 +0000)]
Merge "Bump version to 0.15.0-dev for next dev cycle"

6 years agoMerge "Add options to continue interrupted downloads"
Andrew Grimberg [Mon, 16 Jul 2018 22:26:37 +0000 (22:26 +0000)]
Merge "Add options to continue interrupted downloads"

6 years agoBump version to 0.15.0-dev for next dev cycle 00/11800/1
Thanh Ha [Mon, 16 Jul 2018 18:29:37 +0000 (14:29 -0400)]
Bump version to 0.15.0-dev for next dev cycle

Change-Id: Ied5c6052b3fcede73129e8d5cb82f8bccea0afec
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease lftools v0.14.0 76/11776/3 v0.14.0
Thanh Ha [Fri, 13 Jul 2018 15:38:53 +0000 (11:38 -0400)]
Release lftools v0.14.0

Change-Id: I36f0364dfad341e00d9d381a5369622ced34bd23
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add sigul signing capabilities to lftools"
Thanh Ha [Mon, 16 Jul 2018 00:58:50 +0000 (00:58 +0000)]
Merge "Add sigul signing capabilities to lftools"

6 years agoAdd sigul signing capabilities to lftools 73/11773/9
Andrew Grimberg [Fri, 13 Jul 2018 01:01:45 +0000 (18:01 -0700)]
Add sigul signing capabilities to lftools

Add in the ability to do sigul signing using lftools. This is an
extension of the current sign subcommand

Issue: RELENG-1070
Change-Id: Ie0cc0b2de0375f3c6306858a8ecb873e44888d42
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
6 years agoAdd configuration subsystem 73/10273/7
Thanh Ha [Mon, 30 Apr 2018 18:16:03 +0000 (14:16 -0400)]
Add configuration subsystem

Create a basic ini based configuration subsystem
to be used in the future by lftools subsystems
as a single point of pulling configuration from.

Usage:

  lftools config get section [OPTION]
  lftools config set section option value

Example:

  lftools config set global username zxiiro
  lftools config get global

Issue: RELENG-915
Change-Id: I346662ae2c623538e8a6881574dad357a8ae4dee
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoUpdate to python-jenkins 1.1.0 77/11777/1
Thanh Ha [Fri, 13 Jul 2018 15:42:53 +0000 (11:42 -0400)]
Update to python-jenkins 1.1.0

Pull in feature to detect groovy script failures allowing better
error detection and to actually return a proper error code when
a script fails.

Change-Id: I1e8735519060a024ab725373b3d76b11c89818d4
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoFix ShellCheck warnings for signing script 68/11768/4
Andrew Grimberg [Thu, 12 Jul 2018 22:44:25 +0000 (15:44 -0700)]
Fix ShellCheck warnings for signing script

In prepration for adding sigul signing, the shellcheck warnings need to
be cleaned up

Change-Id: I27479a28f2a7898e489c8a2198ffbf8cdea0cbef
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
6 years agoAdd options to continue interrupted downloads 24/11624/4
Anil Belur [Sat, 30 Jun 2018 08:47:42 +0000 (18:47 +1000)]
Add options to continue interrupted downloads

Add support to continue interrupted downloads with
`lftools sign deploy-nexus`. This allows to continue partially
downloaded files and when used with user specified path (-d).

RELENG-929
Change-Id: I58b576cdcea370f4ae1386529b90130140d6840e
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoUpdate Nexus string substition formating 42/11742/1
Andrew Grimberg [Wed, 11 Jul 2018 21:44:36 +0000 (14:44 -0700)]
Update Nexus string substition formating

Bring the string substitution formatting on Nexus code to the latest
standards for Python 3

Change-Id: I2c38cbbecc8b918aee968d46a12e251976cc8259
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
6 years agoCleanup nexus repo command for Python 3 17/11717/5
Andrew Grimberg [Tue, 10 Jul 2018 21:59:24 +0000 (14:59 -0700)]
Cleanup nexus repo command for Python 3

The original Nexus repo sub-command was written for Python 2 and needs
to be updated to work properly under Python 3.

Change-Id: I5558f833f4fc213e8eea64fc5a53ebcf1714d7be
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
6 years agoMerge "Add user-specified download path for lftools sign"
Andrew Grimberg [Tue, 10 Jul 2018 15:18:14 +0000 (15:18 +0000)]
Merge "Add user-specified download path for lftools sign"

6 years agoAdd user-specified download path for lftools sign 04/11604/6
Anil Belur [Fri, 29 Jun 2018 12:16:57 +0000 (22:16 +1000)]
Add user-specified download path for lftools sign

Add a user-specified download location for lftools sign
deploy-nexus command. The default download location
is "/tmp/gpg-signatures.*" where '*' is random generated
suffix added to a directory name returned by mkdtemp()

Projects like ODL require more than 5 GiB free space
which might not be available in the returned path
therefore we should allow the user to use an alternative
download path.

ex:

lftools sign deploy-nexus -d <path> <URL> <repo> <profile-id>

JIRA: RELENG-928
Change-Id: I960e6c2dd5b5311ec0551834c6b96a297a2eb8bd
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoAdd remove-offline-nodes command 54/11554/2
Thanh Ha [Tue, 26 Jun 2018 00:58:59 +0000 (20:58 -0400)]
Add remove-offline-nodes command

Migrates the 2 scripts for removing offline nodes from global-jjb
jenkins-admin into lftools.

https://github.com/lfit/releng-global-jjb/blob/v0.21.4/jenkins-admin/del_computer.groovy
https://github.com/lfit/releng-global-jjb/blob/v0.21.4/jenkins-admin/slay_computer.groovy

Change-Id: Ifb96689274d91d2655fe3fe1122856c82a3edaf2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add get-credentials groovy script to lftools"
Anil Belur [Wed, 4 Jul 2018 11:44:14 +0000 (11:44 +0000)]
Merge "Add get-credentials groovy script to lftools"

6 years agoAdd Nexus tags 73/11573/2
Jessica Wagantall [Wed, 27 Jun 2018 01:20:24 +0000 (18:20 -0700)]
Add Nexus tags

Fix first Nexus commands tag to remove namespace.
Add other needed tags for repo, create and reorder.

Change-Id: I187cca7d77227cf7fe7a42a7c46389cdd337f09c
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoAdd tag to Nexus docs 72/11572/1
Jessica Wagantall [Wed, 27 Jun 2018 00:49:57 +0000 (17:49 -0700)]
Add tag to Nexus docs

Add tag to Nexus commands documentation

Change-Id: Iba783cd83398d3ece0d73b94c8d2e716c35d5392
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoUpdate Nexus settings example 57/11557/1
Jessica Wagantall [Tue, 26 Jun 2018 18:31:17 +0000 (11:31 -0700)]
Update Nexus settings example

Update Nexus settings.example.yaml file to match
the convention used in the public docs when referring to
a project's domain.
Also, update http to https.

Change-Id: I7cc2e717ba507ea7d7eee577de9744ee57b2e0fd
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoAdd get-credentials groovy script to lftools 53/11553/1
Thanh Ha [Tue, 26 Jun 2018 00:50:27 +0000 (20:50 -0400)]
Add get-credentials groovy script to lftools

This migrates the script in global-jjb into lftools as a command.

https://github.com/lfit/releng-global-jjb/blob/v0.21.4/jenkins-admin/get_all_creds.groovy

Change-Id: Ib60627e031b185b484ac0c3eeb99bf9d82470282
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoPrint result of groovy script to console 57/11157/1
Thanh Ha [Sun, 10 Jun 2018 23:16:42 +0000 (19:16 -0400)]
Print result of groovy script to console

Change-Id: I84824c34c8d13df55a40b8409a9174055ab8e5a9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoIgnore protected images that are protected 52/11152/1
Thanh Ha [Sat, 9 Jun 2018 18:18:03 +0000 (14:18 -0400)]
Ignore protected images that are protected

Print a warning to inform the caller that we cannot delete
images that are protected.

Change-Id: I236f6f55964f483016500b58cbeb48fd8665b62b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoPin python-jenkins to 0.4.16 06/11106/2
Thanh Ha [Thu, 7 Jun 2018 20:07:16 +0000 (16:07 -0400)]
Pin python-jenkins to 0.4.16

It seems python-jenkins 1.0.0 broke the run_scripts() function which
we use to run the jenkins-cfg commands. Pin to the older version
until we figure out why.

Change-Id: I9e643e3aafbc9eb680fd3b8181a0d8b4806060ac
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump version to 0.14.0-dev 41/10941/2
Thanh Ha [Fri, 1 Jun 2018 17:52:24 +0000 (13:52 -0400)]
Bump version to 0.14.0-dev

Change-Id: Ib4313b61d77593aa9c1cf4256aef5ceec1bc1941
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease lftools v0.13.0 31/10931/1 v0.13.0
Thanh Ha [Fri, 1 Jun 2018 16:14:41 +0000 (12:14 -0400)]
Release lftools v0.13.0

Change-Id: If195384c5e9c624d61aa7f1ecccf9268d134c1b3
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSimplify Nexus 2 repo target regex 55/9855/4
Thanh Ha [Mon, 9 Apr 2018 18:14:47 +0000 (14:14 -0400)]
Simplify Nexus 2 repo target regex

Add unit test to check real world artifact paths against the regex
pattern.

Issue: RELENG-758
Change-Id: I580fef69ef03d9cf85c59da250fdfee66fcb7ad5
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMake OpenStack requirements optional 79/6879/9
Thanh Ha [Sat, 14 Oct 2017 15:56:22 +0000 (11:56 -0400)]
Make OpenStack requirements optional

Users can now `pip install lftools[openstack]` as an optional
dependency so that we don't have to download the entire openstack
shade depchain.

Change-Id: Icf0065eeb1bf97f92560434edec3576397223af7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge changes I84773de5,I91de27b2
Jamo Luhrsen [Fri, 27 Apr 2018 17:37:54 +0000 (17:37 +0000)]
Merge changes I84773de5,I91de27b2

* changes:
  Add test for latest upstream of dependencies
  Move tests requirements to separate file

6 years agoMerge "Fail Sphinx build even on warnings"
Thanh Ha [Fri, 27 Apr 2018 15:07:48 +0000 (15:07 +0000)]
Merge "Fail Sphinx build even on warnings"

6 years agoAdd test for latest upstream of dependencies 26/10226/1
Thanh Ha [Mon, 23 Apr 2018 19:49:03 +0000 (15:49 -0400)]
Add test for latest upstream of dependencies

Useful for identifying potential issues if we were to bump versions
of upstream projects that we have pinned.

Change-Id: I84773de551f2bb882c14e5531657c0a37fce70c7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMove tests requirements to separate file 25/10225/1
Thanh Ha [Mon, 23 Apr 2018 19:53:54 +0000 (15:53 -0400)]
Move tests requirements to separate file

Will be used later so we can add additional tests that depend on
this file.

Change-Id: I91de27b2024050894ba5ddeef5ec849074811f9b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRemove trailing slashes in nexus_url 05/10205/1
Thanh Ha [Tue, 24 Apr 2018 16:53:33 +0000 (12:53 -0400)]
Remove trailing slashes in nexus_url

lftools should not fail if the user passes extra trailing slashes that
the command was not expecting.

Change-Id: I9a32c48ddd5ee2898816db7faaeb0de805a38b3c
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Pin requests and python-jenkins"
Jamo Luhrsen [Mon, 23 Apr 2018 20:30:47 +0000 (20:30 +0000)]
Merge "Pin requests and python-jenkins"

6 years agoAdd pytest cache to .gitignore 97/10197/1
Thanh Ha [Mon, 23 Apr 2018 19:57:16 +0000 (15:57 -0400)]
Add pytest cache to .gitignore

Change-Id: Ie3ff833149896f6beab8eb137f049a288538ff90
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoPin requests and python-jenkins 96/10196/1
Thanh Ha [Mon, 23 Apr 2018 19:38:23 +0000 (15:38 -0400)]
Pin requests and python-jenkins

Change-Id: Ic830bad231d98efe721a299f18f634d87aad5e9f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoFail Sphinx build even on warnings 82/10182/1
Thanh Ha [Fri, 20 Apr 2018 18:45:46 +0000 (14:45 -0400)]
Fail Sphinx build even on warnings

Change-Id: I935e57b4536f58eb71b49d6558fefc170d50a8da
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump version to 0.13.0-dev 74/10074/1
Thanh Ha [Tue, 17 Apr 2018 17:20:43 +0000 (13:20 -0400)]
Bump version to 0.13.0-dev

Change-Id: I8f643e0a9b6fefce7c723d4b46b955df4334dbb8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease lftools v0.12.4 73/10073/1 v0.12.4
Thanh Ha [Tue, 17 Apr 2018 18:56:54 +0000 (14:56 -0400)]
Release lftools v0.12.4

Change-Id: If4fba3ca0b18bf785ed421ff42c20f20f608f9ca
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoFix remove volume time format 72/10072/1
Thanh Ha [Tue, 17 Apr 2018 18:56:11 +0000 (14:56 -0400)]
Fix remove volume time format

Created_at also uses a different time format that needs to be
addressed.

Change-Id: I5618a5f1ce3fdf0d49ce01cbad47683cf8e02762
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump version to 0.13.0-dev 70/10070/1
Thanh Ha [Tue, 17 Apr 2018 17:20:43 +0000 (13:20 -0400)]
Bump version to 0.13.0-dev

Change-Id: Ie73dfc81edc257dea06450bd236d98746cd35290
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease lftools v0.12.3 68/10068/1 v0.12.3
Thanh Ha [Tue, 17 Apr 2018 18:19:16 +0000 (14:19 -0400)]
Release lftools v0.12.3

Change-Id: Ie38022c49c78a5a1ebe12ba596b339b3a9e110b7
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoFix volume remove command polling wrong metadata 67/10067/1
Thanh Ha [Tue, 17 Apr 2018 18:18:18 +0000 (14:18 -0400)]
Fix volume remove command polling wrong metadata

Volumes use created_at instead of created.

Change-Id: I53a11dc791ee2d4035f9b641f97c019f2f19610f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump version to 0.13.0-dev 63/10063/1
Thanh Ha [Tue, 17 Apr 2018 17:20:43 +0000 (13:20 -0400)]
Bump version to 0.13.0-dev

Change-Id: Ice5411654c110e4630c1e8d95796c5bfa02f40b8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease lftools v0.12.2 62/10062/1 v0.12.2
Thanh Ha [Tue, 17 Apr 2018 17:03:04 +0000 (13:03 -0400)]
Release lftools v0.12.2

Change-Id: I76af823e2c391e3ecf4ab7f5738a642cd554401f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd support for volume management 54/10054/3
Thanh Ha [Tue, 17 Apr 2018 15:39:20 +0000 (11:39 -0400)]
Add support for volume management

Issue: RELENG-853
Change-Id: Id412315238abfdea9f655dfeb2957964b37d5f19
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump version to 0.13.0-dev 35/8835/1
Thanh Ha [Tue, 13 Feb 2018 17:06:00 +0000 (12:06 -0500)]
Bump version to 0.13.0-dev

Change-Id: Iac54f1c832e159fc960a18de4191052a67ba82dd
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease lftools v0.12.1 34/8834/1 v0.12.1
Thanh Ha [Tue, 13 Feb 2018 16:26:58 +0000 (11:26 -0500)]
Release lftools v0.12.1

Change-Id: I9aed03e79c38ef2823722e3a76be4bb45f47b549
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoPin netifaces to 0.10.5 33/8833/1
Thanh Ha [Tue, 13 Feb 2018 16:25:28 +0000 (11:25 -0500)]
Pin netifaces to 0.10.5

netifaces decided to release a repackaging of 0.10.6 this morning
that broke us. Let us add them to the pinned deps list to resolve
the issue.

Change-Id: Id67c3bdc9e7da252892c1a1e7514dc25e7c2ae05
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump version to 0.13.0-dev 28/8828/1
Thanh Ha [Mon, 12 Feb 2018 22:51:17 +0000 (17:51 -0500)]
Bump version to 0.13.0-dev

Change-Id: I90cd117ed3d436cbebb38e359b7e32b23df157f1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease lftools v0.12.0 85/8785/1 v0.12.0
Thanh Ha [Fri, 9 Feb 2018 19:11:03 +0000 (14:11 -0500)]
Release lftools v0.12.0

Change-Id: Ifa0a9344badb1ca942707b833cd71a77dc3f4a9e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd lftools command to remove specific server 25/8725/3
Thanh Ha [Wed, 7 Feb 2018 16:09:57 +0000 (11:09 -0500)]
Add lftools command to remove specific server

This command also supports an optional minutes parameter which
can be used to ignore the server if it is not older than a certain
number of minutes.

This is useful for cases like Jenkins orphan server cleanup scripts
so that they do not try to remove servers that have just been
spawned and have not yet connected to Jenkins causing the server to
be mistakenly marked as orphaned.

Issue: RELENG-780
Change-Id: I7f12218e0ac730ff43def4ec9b6133b4cc8ee0c8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump version to 0.12.0-dev 28/8628/1
Thanh Ha [Thu, 1 Feb 2018 00:42:50 +0000 (19:42 -0500)]
Bump version to 0.12.0-dev

Change-Id: Ie320efc1e9917751505b071df0cc59e09791d1d1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease lftools v0.11.2 27/8627/1 v0.11.2
Thanh Ha [Thu, 1 Feb 2018 00:41:32 +0000 (19:41 -0500)]
Release lftools v0.11.2

Change-Id: I0361a50e1aa239e97185809124cd8df8be9b8b56
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoupload_to_nexus to return a non-zero exit status 22/8622/2
Robert Varga [Wed, 31 Jan 2018 11:03:28 +0000 (12:03 +0100)]
upload_to_nexus to return a non-zero exit status

As it turns out Nexus does not produce a http status code on upload
failure and returns back a blank message. This means if we do not
detect a successful status from Nexus we need to exit 1 explicity
to make sure we return to parallel a proper error status code.

JIRA: RELENG-765
Change-Id: Ic221c668bb6cd35befc614159ee8d7220d3f4cfe
Co-authored-by: Thanh Ha <thanh.ha@linuxfoundation.org>
Signed-off-by: Robert Varga <nite@hq.sk>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSwitch to using lfdocs-conf 0.1.3 24/8524/1
Thanh Ha [Thu, 25 Jan 2018 02:41:44 +0000 (21:41 -0500)]
Switch to using lfdocs-conf 0.1.3

Change-Id: If0ad2ba50ffa6f5ecdb0fb3a06563706b56c4015
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd a groovy command to lftools 33/8433/1
Thanh Ha [Thu, 18 Jan 2018 22:03:01 +0000 (17:03 -0500)]
Add a groovy command to lftools

Using python-jenkins' run_groovy API to allow us to execute
groovy scripts from lftools.

Change-Id: If5afb69b738a0cee6ee653b5d70ba9b5dac67fd0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSwitch lftools to lfdocs-conf 13/8413/1
Thanh Ha [Wed, 17 Jan 2018 03:16:52 +0000 (22:16 -0500)]
Switch lftools to lfdocs-conf

Consolidate conf.py into lfdocs-conf project.

Change-Id: I51c0931d5bc03cc71d10edca73a46379cf02ac2e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoImprove debugging output for deploy nexus-zip cmd 17/8317/5
Thanh Ha [Tue, 9 Jan 2018 21:38:32 +0000 (16:38 -0500)]
Improve debugging output for deploy nexus-zip cmd

Print out the HTTP Status code as well as the http response from
the server (if any, Nexus doesn't always return any) and the
contents of the zip file on failure.

Issue: RT-50048
Change-Id: I54014819d18c0090c45fd8378954dc9aa5e540f8
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoFix ShellCheck v0.4.7 lint issues 20/8320/1
Thanh Ha [Tue, 9 Jan 2018 23:21:41 +0000 (18:21 -0500)]
Fix ShellCheck v0.4.7 lint issues

v0.4.7 introduces some new rules which now need to be handled.

Change-Id: I9c769bd819b787d817ef7708c312e9c637ee7dfc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump version to v0.12.0-dev 93/8293/1
Thanh Ha [Mon, 8 Jan 2018 16:22:26 +0000 (11:22 -0500)]
Bump version to v0.12.0-dev

Change-Id: Id5606835c44a5771025154b732aa49343627a922
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease lftools v0.11.1 92/8292/1 v0.11.1
Thanh Ha [Mon, 8 Jan 2018 16:09:21 +0000 (11:09 -0500)]
Release lftools v0.11.1

Change-Id: I4eb27f0dbb3ccb2632b93be0f09830755ff67125
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Fix wget command for Nexus sign script"
Thanh Ha [Wed, 3 Jan 2018 19:40:13 +0000 (19:40 +0000)]
Merge "Fix wget command for Nexus sign script"

6 years agoFix wget command for Nexus sign script 73/7573/3
Jessica Wagantall [Thu, 16 Nov 2017 03:57:54 +0000 (19:57 -0800)]
Fix wget command for Nexus sign script

Add --level=15 to wget command that fetches the
autoreleases from Nexus. Looks like the previous
command wasn't fetching all the artifacts in the
autorelease tree.

Change-Id: Ieb49331d7b192187ba2e6783b334d90c6477503e
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoPrint CPU, Memory, Disk and Network stats 46/8046/1
Anil Belur [Mon, 18 Dec 2017 21:58:33 +0000 (07:58 +1000)]
Print CPU, Memory, Disk and Network stats

Print sar CPU, Memory, Disk, Network statistics at end of console log

JIRA: RELENG-672
Change-Id: I42aae6a07a91e7a666b28fcd027a5c0a91a8e64c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoBump version to v0.12.0-dev 18/7918/1
Thanh Ha [Mon, 11 Dec 2017 17:59:57 +0000 (12:59 -0500)]
Bump version to v0.12.0-dev

Change-Id: I37dfe4d790490aa3d40216c7034c77f3d830a5a0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease lftools v0.11.0 17/7917/1 v0.11.0
Thanh Ha [Mon, 11 Dec 2017 17:46:11 +0000 (12:46 -0500)]
Release lftools v0.11.0

Change-Id: If40b6319e21f6d42eb770a22de2b93677a6eb87f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAutomatically hide protected images 87/7887/2
Thanh Ha [Sat, 9 Dec 2017 01:45:25 +0000 (20:45 -0500)]
Automatically hide protected images

We are not able to delete protected images so automatically filter
them out of our results.

Issue: RELENG-705
Change-Id: Id7fa5ab73ec912aac9b04dfb30c29492320402f2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoUpdate openstack cmd to support ci_managed filter 86/7886/2
Thanh Ha [Sat, 9 Dec 2017 01:40:13 +0000 (20:40 -0500)]
Update openstack cmd to support ci_managed filter

Add support for filtering ci_managed metadata when querying for
images.

Issue: RELENG-705
Change-Id: I9570a0683e1d87547bda8fc6741369549450045e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSuppress GNU parallel notice on every run 41/7541/3
Anil Belur [Wed, 15 Nov 2017 05:23:16 +0000 (15:23 +1000)]
Suppress GNU parallel notice on every run

By default the notice is output on every run, therefore
use the `--no-notice` option to supress the output.

Change-Id: Id1dbe7b8daee390c886486eeb3c4c3c436fc26e4
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoAdd a snapshot mode to the deploy-nexus command 21/7221/1
Thanh Ha [Wed, 1 Nov 2017 18:28:03 +0000 (14:28 -0400)]
Add a snapshot mode to the deploy-nexus command

This is necessary for RELENG-61 in order to allow pushing
maven-metadata.xml files which in the case of release jobs should not be
allowed. So add an option to the deploy-nexus command to allow an
exception for snapshot artifacts.

Issue: RELENG-173
Change-Id: I6f086bd0987502c5a967ab0b6f313ac405ceabc5
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoFail gracefully when no files to sign 31/6931/2
Thanh Ha [Tue, 17 Oct 2017 02:14:01 +0000 (22:14 -0400)]
Fail gracefully when no files to sign

Fixes issue where if there are no files to sign detected the script just
hangs indefinitely. Instead if the files_to_sign array is empty then
exit 1 and inform user.

Change-Id: I82a5957faedfe739800305a8224fed67d2c74801
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>