releng/lftools.git
5 years agoRefactor Deploy Nexus Zip command to Python 86/13286/7
Bengt Thuree [Thu, 1 Nov 2018 23:20:12 +0000 (19:20 -0400)]
Refactor Deploy Nexus Zip command to Python

Refactor the deploy nexus zip command for better portability with
Windows systems.

Issue: RELENG-1372
Change-Id: I58ea1d7703b626f791dcd74e63251c4f3261ca7d
Co-Authored-By: Thanh Ha <thanh.ha@linuxfoundation.org>
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
5 years agoRefactor deploy-logs cmd to Python 13/13413/7
Thanh Ha [Thu, 8 Nov 2018 02:31:30 +0000 (10:31 +0800)]
Refactor deploy-logs cmd to Python

Issue: RELENG-1377
Change-Id: Ibf58a92fe8800ad77d8be32a123da603c6a36c56
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRefactor deploy-archives to Python 72/13272/10
Thanh Ha [Thu, 1 Nov 2018 15:52:48 +0000 (11:52 -0400)]
Refactor deploy-archives to Python

CLI wise the function was reworked a bit to allow multiple
(-p pattern) calls for better usability.

Issue: RELENG-1376
Change-Id: I232a284c785b413129f8386fc7e799019da1f754
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Add jenkins and xdg to known 3rdparty"
Thanh Ha (zxiiro) [Fri, 9 Nov 2018 17:24:37 +0000 (17:24 +0000)]
Merge "Add jenkins and xdg to known 3rdparty"

5 years agoInternal func _log_error_and_exit & _request_post 88/13288/11
Bengt Thuree [Fri, 2 Nov 2018 00:51:38 +0000 (11:51 +1100)]
Internal func _log_error_and_exit & _request_post

Adds two internal functions
_log_error_and_exist : prints out 0, 1 or 2 lines of text
                       and then exit with sys.exit(1)

_request_post        : This is a wrapper around request.post
                       Handles three exceptions
                         ConnectionError, MissingSchema, InvalidURL

Change-Id: If8a4d20054d958cbfb3ab651eae037a124420bd3
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
5 years agoAdd jenkins and xdg to known 3rdparty 73/13373/1
Thanh Ha [Wed, 7 Nov 2018 03:49:45 +0000 (11:49 +0800)]
Add jenkins and xdg to known 3rdparty

Change-Id: Ica655586a045d6aaca226e667e1f3eb1edf1dad3
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Add docker list/delete subcommands for nexus"
Eric Ball [Tue, 6 Nov 2018 21:05:44 +0000 (21:05 +0000)]
Merge "Add docker list/delete subcommands for nexus"

5 years agoMerge changes from topic '13163'
Thanh Ha (zxiiro) [Mon, 5 Nov 2018 03:36:17 +0000 (03:36 +0000)]
Merge changes from topic '13163'

* changes:
  Allow additional credential input methods
  Refactor config sys error handler

5 years agoInternal func _format_url() to verify url fmt 87/13287/3
Bengt Thuree [Thu, 1 Nov 2018 23:48:58 +0000 (19:48 -0400)]
Internal func _format_url() to verify url fmt

Adds a internal function _format_url() to verify url formatting for
domain root.

Change-Id: I0020cf5fb6a848c0fdb2fbc3cd79a48f4c79df4c
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
5 years agoMerge "Fix copy_archives crash when pattern is None"
Jessica Wagantall [Thu, 1 Nov 2018 22:36:33 +0000 (22:36 +0000)]
Merge "Fix copy_archives crash when pattern is None"

5 years agoFix copy_archives crash when pattern is None 85/13285/1
Thanh Ha [Thu, 1 Nov 2018 21:53:47 +0000 (17:53 -0400)]
Fix copy_archives crash when pattern is None

We should just exist out of the function if no pattern is passed.

Change-Id: I7299b2606713685720ff0cf76b5fddb77296a954
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Test against explicit major versions of Python"
Thanh Ha (zxiiro) [Thu, 1 Nov 2018 20:50:48 +0000 (20:50 +0000)]
Merge "Test against explicit major versions of Python"

5 years agoFix cli creating /tmp/gpg-sig dir unexpectedly 70/13270/1
Thanh Ha [Thu, 1 Nov 2018 14:42:04 +0000 (10:42 -0400)]
Fix cli creating /tmp/gpg-sig dir unexpectedly

Simply calling the cli always creates 2 directories named
/tmp/gpg-signatures.* and flooding /tmp with unnecessary directories.

This patch resolves these unexpected directory creations by declaring
these tmpdirs in the code rather than the function definition to avoid
this issue.

Change-Id: I3762854fea187082bb8f459ca062e02c1f4eb43f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoTest against explicit major versions of Python 67/13267/1
Thanh Ha [Wed, 31 Oct 2018 22:16:31 +0000 (18:16 -0400)]
Test against explicit major versions of Python

We need to support py27 and py34 for Cent OS but list later
interpreters for local builds.

Bump minimum tox version to 2.0 to ensure we have support for the
skip_missing_interpreters configuration.

Change-Id: If8eb6c988c4fe558ac411e1b1001b265cae9ae99
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRefactor copy_archives deploy function to Python 51/13251/6
Thanh Ha [Wed, 31 Oct 2018 14:38:27 +0000 (10:38 -0400)]
Refactor copy_archives deploy function to Python

Rewrites the copy_archives() bash script to Python for
better portability on Windows systems.

Issue: RELENG-1375
Change-Id: I264c29f9a3b86415fd879ef9dce38714105309f0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoIgnore .pytest_cache directory in Coala 50/13250/1
Thanh Ha [Wed, 31 Oct 2018 12:19:41 +0000 (08:19 -0400)]
Ignore .pytest_cache directory in Coala

Change-Id: I49476c5bf63fc83217cbf8e5957286a2dc9e82f4
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAllow additional credential input methods 63/13163/3
Thanh Ha [Wed, 24 Oct 2018 01:08:14 +0000 (21:08 -0400)]
Allow additional credential input methods

Provide additional methods for credential passing.

1. Via explicit --password parameter
2. Via environment variable LFTOOLS_PASSWORD
3. At runtime if --interactive mode is set

Issue: RELENG-1207
Change-Id: I9f083cc7b7381ab8df0578664ae29fa18ef279cf
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoRefactor config sys error handler 37/13237/3
Thanh Ha [Mon, 29 Oct 2018 23:42:59 +0000 (19:42 -0400)]
Refactor config sys error handler

Use exception system to allow downstream users to raise their
own exceptions.

Change-Id: I184eadf93f101cbe96ea5a1e10406dac6c906767
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd docker list/delete subcommands for nexus 81/12581/19
Eric Ball [Thu, 13 Sep 2018 00:36:16 +0000 (17:36 -0700)]
Add docker list/delete subcommands for nexus

This commit adds a "docker" subcommand to "nexus", with options to
list or delete images. This is intended for docker repos in Nexus 3.

Also includes logic in the Nexus class to find the version and
correct API base path from the provided URL.

Issue-ID: RELENG-1164
Change-Id: I759297ed9f49566df6062b2a158e0954673e29eb
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoLock pre-commit to version 1.11.2 62/13162/1
Eric Ball [Wed, 24 Oct 2018 01:02:18 +0000 (18:02 -0700)]
Lock pre-commit to version 1.11.2

The latest version, 1.12.0, is causing problems with the
lftools-tox-verify Jenkins job.

Change-Id: Ifa7cb9d90bb6691a3892778e3b186256bc4e4501
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoAdd --conf parameter to jenkins sub-cmd 45/13045/2
Thanh Ha [Thu, 18 Oct 2018 00:09:58 +0000 (20:09 -0400)]
Add --conf parameter to jenkins sub-cmd

Per request from Ieb54acc2d12de9bb21620112380f20be14fe9113 this
patch adds a --conf parameter to the jenkins subcommand to allow
overriding the JJB_INI default locations.

Issue: RELENG-1206
Change-Id: Ia1af96ccb64451199a1dc3994ae8b3004272afdb
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoMerge "Fix get-credentials subcommand" v0.18.0
Jeremy Phelps [Fri, 19 Oct 2018 15:05:21 +0000 (15:05 +0000)]
Merge "Fix get-credentials subcommand"

5 years agoUpdate reno to 2.11.2 67/13067/2
Thanh Ha [Thu, 18 Oct 2018 18:19:19 +0000 (14:19 -0400)]
Update reno to 2.11.2

Issue: RELENG-1355
Change-Id: I2a194aa557a7a29ed4e70f084dd4255266ee29ab
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoFix get-credentials subcommand 62/13062/1
Thanh Ha [Thu, 18 Oct 2018 17:39:59 +0000 (13:39 -0400)]
Fix get-credentials subcommand

When we refactored the Jenkins module in
I2168adf9bc992b719da6c0350a446830015e6df6 we forgot to refactor
the get-credentials command so it broke.

Change-Id: I1ee04d2753327557e4d7e5e017a3d6ca7741fef0
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoAdd cmd to clean up stale openstack heat stacks 74/12674/5
Thanh Ha [Thu, 20 Sep 2018 01:57:41 +0000 (21:57 -0400)]
Add cmd to clean up stale openstack heat stacks

Issue: RELENG-237
Change-Id: I8e2fcef93798395cc61f686a143ca3add5d081f2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
5 years agoFix typo in reorder-staged-repos help cmd 19/13019/1
Thanh Ha [Tue, 16 Oct 2018 23:28:03 +0000 (19:28 -0400)]
Fix typo in reorder-staged-repos help cmd

Strangely the docs builder is only catching this now.

Change-Id: I73b40d600183a8075f085a504c974a21e9daa31e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRefactor the Jenkins object into a class 75/12675/4
Thanh Ha [Thu, 20 Sep 2018 02:24:42 +0000 (22:24 -0400)]
Refactor the Jenkins object into a class

Make it easier for us to reuse the Jenkins instance in commands
outside of the jenkins command group.

Issue: RELENG-237
Change-Id: I2168adf9bc992b719da6c0350a446830015e6df6
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd token init cmd to add new server cfg 11/12511/11
Thanh Ha [Sat, 8 Sep 2018 19:52:01 +0000 (15:52 -0400)]
Add token init cmd to add new server cfg

New token init command to automatically fetch API token and add new
server configuration section into jenkins_jobs.ini. This file should
already exist. This cmd uses the globally configured user credentials
in lftools.ini to initialize the new Jenkins server configuration.

Change-Id: I3d4bb0fd5696fd1bbeb26eb7d609bab5e064ba92
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd cmd to reset API tokens in jenkins_jobs.ini 10/12510/11
Thanh Ha [Sat, 8 Sep 2018 19:31:35 +0000 (15:31 -0400)]
Add cmd to reset API tokens in jenkins_jobs.ini

Convenience command to automatically reset API tokens in
jenkins-jobs.ini.

Change-Id: I6d84760368b6a12e9e9622b5ce9496a278321dc4
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd jenkins token cmd to retrieve the API token 08/12508/8
Thanh Ha [Sat, 8 Sep 2018 17:53:14 +0000 (13:53 -0400)]
Add jenkins token cmd to retrieve the API token

Change-Id: I43397d25978aaf752db33003044e7cb8d4884705
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Move JJB_INI to static variable"
Andrew Grimberg [Wed, 12 Sep 2018 18:48:53 +0000 (18:48 +0000)]
Merge "Move JJB_INI to static variable"

6 years agoMerge changes from topic 'share-cloud-images'
Andrew Grimberg [Wed, 12 Sep 2018 18:34:21 +0000 (18:34 +0000)]
Merge changes from topic 'share-cloud-images'

* changes:
  Add cmd to upload openstack images
  Add cmd to share openstack images

6 years agoMerge "Enable universal wheel"
Andrew Grimberg [Wed, 12 Sep 2018 18:31:10 +0000 (18:31 +0000)]
Merge "Enable universal wheel"

6 years agoMove JJB_INI to static variable 09/12509/7
Thanh Ha [Sat, 8 Sep 2018 18:56:00 +0000 (14:56 -0400)]
Move JJB_INI to static variable

We don't expect this location to change so set a static variable
that we can reuse more easily across modules.

Change-Id: Ieb54acc2d12de9bb21620112380f20be14fe9113
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoUse logger instead of print in Jenkins CLI 07/12507/3
Thanh Ha [Sat, 8 Sep 2018 18:27:34 +0000 (14:27 -0400)]
Use logger instead of print in Jenkins CLI

Change-Id: I6e99c69f75a0be5c0e54da5c21a2930652ad3f8e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoDisable fast fail on parallel sigul 23/12523/1 v0.17.1
Andrew Grimberg [Mon, 10 Sep 2018 19:29:04 +0000 (12:29 -0700)]
Disable fast fail on parallel sigul

Sigul bridges seem to possibly have an issue with a lot of in parallel
signing requests. This is either an issue with the bridge, the server,
or a disconnected across the network VPN backhaul between the server and
bridge. Instead of having the signing just outright fail on the first
problem, we should try a few times.

Change-Id: I98a5c8adeb25e892899a221dc2078a9fc779f74e
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
6 years agoEnable universal wheel 05/12505/1
Thanh Ha [Fri, 7 Sep 2018 23:24:30 +0000 (19:24 -0400)]
Enable universal wheel

lftools is Python 2 and 3 compatible so enable universal wheels.

Change-Id: I7a75f944030e35d4b021709700084abf11b6ae00
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd cmd to upload openstack images 98/12498/4
Thanh Ha [Fri, 7 Sep 2018 17:42:09 +0000 (13:42 -0400)]
Add cmd to upload openstack images

Issue: RELENG-1201
Change-Id: I1f6072fbfff003a4602d3765a6a5ce7c94fe4c58
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd cmd to share openstack images 83/12483/5
Thanh Ha [Fri, 7 Sep 2018 03:06:18 +0000 (23:06 -0400)]
Add cmd to share openstack images

Allows us to more easily share images to multiple openstack tenants.

Issue: RELENG-1201
Change-Id: I58dc200f7b29429ee1215bacdc533d234ad153fb
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Use reno 2.10.0 and remove reno.yaml"
Jessica Wagantall [Fri, 7 Sep 2018 18:41:59 +0000 (18:41 +0000)]
Merge "Use reno 2.10.0 and remove reno.yaml"

6 years agoRemove prelude 97/12497/1
Thanh Ha [Fri, 7 Sep 2018 17:24:38 +0000 (13:24 -0400)]
Remove prelude

This prelude did not make much sense for the patch the introduced it.

Change-Id: I8ccd3ba8ff02a43db043cfca842645dd1dccd89c
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoUse reno 2.10.0 and remove reno.yaml 96/12496/1
Thanh Ha [Fri, 7 Sep 2018 17:12:26 +0000 (13:12 -0400)]
Use reno 2.10.0 and remove reno.yaml

Reno 2.10.0 supports our regex pattern so we no longer need to
configure it.

Also add reno to tox so that people can run it via:

    tox -e reno new slug

Change-Id: Ic5f4de1af139e0cf99e955a30e3d957772563f57
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd npm metadata to .gitignore 93/12393/2 v0.17.0
Thanh Ha [Sat, 25 Aug 2018 05:41:57 +0000 (01:41 -0400)]
Add npm metadata to .gitignore

Change-Id: Ifa5ebace74db851e00fb2431ef06f38952ec3e24
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add stack create & delete commands"
Sam Hague [Thu, 6 Sep 2018 14:54:57 +0000 (14:54 +0000)]
Merge "Add stack create & delete commands"

6 years agoMerge "Deduplicate test requirements"
Sam Hague [Thu, 6 Sep 2018 14:54:48 +0000 (14:54 +0000)]
Merge "Deduplicate test requirements"

6 years agoUse logger instead of print in version module 98/12398/3
Thanh Ha [Mon, 27 Aug 2018 01:25:24 +0000 (21:25 -0400)]
Use logger instead of print in version module

Change-Id: I31329886088d8d7a469e19bc0937912546400109
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoUpdate config subsystem to use logger 97/12397/4
Thanh Ha [Mon, 27 Aug 2018 00:58:29 +0000 (20:58 -0400)]
Update config subsystem to use logger

Use our logger service in the config subsystem.

Change-Id: I47235cf9accd362ba1355ff77e15757543d30da1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoOnly print extra log details in DEBUG mode 90/12390/8
Thanh Ha [Sat, 25 Aug 2018 01:04:13 +0000 (21:04 -0400)]
Only print extra log details in DEBUG mode

This allow us to change all our print statements to log.info() and
have it still print sanely like a CLI application. Then when DEBUG
mode is enabled it will print the extra details to let us
troubleshoot. ERROR and CRITICAL logs will prefix with "ERROR: "
for example.

Change-Id: Ic35222ee654306e686052c209b641f9e49d38144
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add support to jenkins cmd for jenkins_jobs.ini"
Thanh Ha (zxiiro) [Mon, 27 Aug 2018 22:23:43 +0000 (22:23 +0000)]
Merge "Add support to jenkins cmd for jenkins_jobs.ini"

6 years agoMerge "Add ruamel.yaml and yaml as known 3rdparty"
Thanh Ha (zxiiro) [Mon, 27 Aug 2018 22:23:34 +0000 (22:23 +0000)]
Merge "Add ruamel.yaml and yaml as known 3rdparty"

6 years agoMerge "Enable or disable Jenkins jobs matching regex"
Thanh Ha (zxiiro) [Mon, 27 Aug 2018 17:03:09 +0000 (17:03 +0000)]
Merge "Enable or disable Jenkins jobs matching regex"

6 years agoEnable or disable Jenkins jobs matching regex 82/12382/8
Anil Belur [Fri, 24 Aug 2018 04:22:11 +0000 (09:52 +0530)]
Enable or disable Jenkins jobs matching regex

ex:
To enable jobs

To disable jobs

JIRA: RELENG-1117
Change-Id: Iff800472cecfe056285f07e1c2461775457cb0ab
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoDeduplicate test requirements 00/12400/1
Thanh Ha [Mon, 27 Aug 2018 02:43:02 +0000 (22:43 -0400)]
Deduplicate test requirements

Some of these tests being pulled in are duplicates. Let's consolidate
into requirements-test.txt as the source of truth.

Change-Id: I2481a364b9ddf2aa0ba9e2168d4ee408ca477c43
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd support to jenkins cmd for jenkins_jobs.ini 91/12391/2
Thanh Ha [Sat, 25 Aug 2018 05:23:08 +0000 (01:23 -0400)]
Add support to jenkins cmd for jenkins_jobs.ini

Allow the jenkins command to use existing jenkins_jobs.ini for
configuration. By default assume jenkins_jobs.ini's default
configuration 'jenkins'.

This change makes 'server', 'user', and 'password' parameters no
longer required as defaults replace the need for these options.

'server' now defaults to 'jenkins' if jenkins_jobs.ini exists,
otherwise defaults to 'http://localhost:8080' which is the default
Jenkins deploy port.

'user' now defaults to 'admin' which is the Jenkins default admin
user.

'password' has no default and must be set by the user as this
is randomly generated by Jenkins at boot.

Change-Id: Ic7af1eb53e8e9079b845203f44914310616c4bab
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd ruamel.yaml and yaml as known 3rdparty 92/12392/1
Thanh Ha [Sat, 25 Aug 2018 05:38:40 +0000 (01:38 -0400)]
Add ruamel.yaml and yaml as known 3rdparty

Change-Id: I954835b3e1a1700119d2e2328444b7794e7365fc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMake ldap module optional 84/12384/3 v0.16.1
Thanh Ha [Fri, 24 Aug 2018 15:38:27 +0000 (11:38 -0400)]
Make ldap module optional

Install via 'pip install lftools[ldap]'.

Change-Id: Ie2206b43b094e1576936ba934268dc0b68fc5a40
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRemove summary/version which comes from PBR now 83/12383/2 v0.16.0
Thanh Ha [Fri, 24 Aug 2018 14:38:04 +0000 (10:38 -0400)]
Remove summary/version which comes from PBR now

Change-Id: I19b0358a85beb281a07df1ef8ea84c8c7593927f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Extend lftools with scripts for ldap lookups"
Thanh Ha (zxiiro) [Fri, 24 Aug 2018 14:26:48 +0000 (14:26 +0000)]
Merge "Extend lftools with scripts for ldap lookups"

6 years agoAdd option to enable debug logs 79/12379/4
Thanh Ha [Fri, 24 Aug 2018 01:36:46 +0000 (21:36 -0400)]
Add option to enable debug logs

Default the logger to INFO level and add an option to turn on DEBUG
level logs. Update the unit_tests to pass in a context since we need
it now in order to support the top-level configuration.

Change-Id: I253a127f7dc0cee5475d8403da8f82e018fdf36e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSimplify logging configuration 78/12378/3
Thanh Ha [Fri, 24 Aug 2018 01:23:18 +0000 (21:23 -0400)]
Simplify logging configuration

lftools should be a simple tool and folks should not ever need to
configure the logger so let's simplify things and just pre-define
an acceptable logger configuration. The important component is
having (DEBUG) messages print via logger service. A subsequent
patch will be provided to enable --debug level logs via CLI.

Change-Id: I74158d94400112161ad4fc206426420d908deb0c
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd extras declaration in docs-linkcheck 81/12381/1
Thanh Ha [Fri, 24 Aug 2018 02:05:54 +0000 (22:05 -0400)]
Add extras declaration in docs-linkcheck

This should have been added to
I1f33c65a44cf6eaa0e29e5734f3a8edcd8865868

Also include missing reno notes.

Change-Id: I132bebe298164419fae43e94299f43c45940bc4e
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoFix broken help from openstack and sign cmds 80/12380/1
Thanh Ha [Fri, 24 Aug 2018 02:05:54 +0000 (22:05 -0400)]
Fix broken help from openstack and sign cmds

These were never caught since we did not enable fail on warnings
for Sphinx until recently.

Change-Id: I1f33c65a44cf6eaa0e29e5734f3a8edcd8865868
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd stack create & delete commands 53/12353/4
Thanh Ha [Tue, 21 Aug 2018 23:05:31 +0000 (19:05 -0400)]
Add stack create & delete commands

Add 2 new commands:

    - stack create
    - stack delete

These commands allow lftools the ability to create and delete stacks.

Issue: RELENG-235
Change-Id: Ied7ff51a292199675dc944e36f1d821a5a2d045d
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoExtend lftools with scripts for ldap lookups 58/11558/46
Aric Gardner [Tue, 26 Jun 2018 19:16:08 +0000 (15:16 -0400)]
Extend lftools with scripts for ldap lookups

Add 2 new commands to lftools:

    - infofile
    - ldap

prereqs:
For ldap lookups to work you must be on the VPN and have the cert
to get the cert: log in to any collab system and grab /etc/ipa/ca.crt
in /etc/openldap/ldap.conf

TLS_REQCERT always
TLS_CACERTDIR /etc/openldap/certs
TLS_CACERT /etc/openldap/certs/ca.crt

To test:

1. Clone this patchset
2. start venv

   pip uninstall lftools && pip install -e .

Usage:

$ lftools ldap
Usage: lftools ldap [OPTIONS] COMMAND [ARGS]...

  LDAP TOOLS.

Commands:
  autocorrectinfofile  Verify INFO.yaml against LDAP group.
  csv                  Query an Ldap server.
  inactivecommitters   Check committer participation.
  yaml4info            Build yaml of commiters for your INFO.yaml.

$ lftools infofile
Usage: lftools infofile [OPTIONS] COMMAND [ARGS]...

  INFO.yaml TOOLS.

Commands:
  get-committers    Extract Committer info from INFO.yaml or LDAP...
  sync-committers   Script to insert missing values from ldap...

Issue: RELENG-407
Change-Id: I4f4055441042d790008754bb085447f52e1c1a78
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
6 years agoUse reno for release notes 30/12330/2
Thanh Ha [Mon, 20 Aug 2018 16:55:54 +0000 (12:55 -0400)]
Use reno for release notes

Change-Id: Iadb0ca93a6f1f6a56c948b42bd257040547fffcc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSetup PBR for lftools 07/12307/3
Thanh Ha [Fri, 17 Aug 2018 18:46:33 +0000 (14:46 -0400)]
Setup PBR for lftools

Change-Id: I028540301560b76af1cb31f2d9c40612b3b3480f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRemove unnecessary echo 17/12017/2
Thanh Ha [Fri, 27 Jul 2018 15:07:44 +0000 (11:07 -0400)]
Remove unnecessary echo

Since Sphinx 1.7.x, Sphinx itself now prints the output location
at the end of a build so there's no need to do this unnecessary echo
anymore.

Change-Id: I517dea4686b68140be4ab672d6e909e446919163
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump to 0.16.0-dev 31/12031/1
Thanh Ha [Mon, 30 Jul 2018 15:33:03 +0000 (11:33 -0400)]
Bump to 0.16.0-dev

Change-Id: Ia255aefd6e1a05d2fb8440584406b6d8fbe5f2f3
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease lftools v0.15.0 19/12019/3 v0.15.0
Thanh Ha [Fri, 27 Jul 2018 15:47:18 +0000 (11:47 -0400)]
Release lftools v0.15.0

Change-Id: I3a3196f64cefabb390315a85e277d4652c9b30cb
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Handle snapshot artifacts in get_file_info()"
Thanh Ha [Mon, 30 Jul 2018 15:16:07 +0000 (15:16 +0000)]
Merge "Handle snapshot artifacts in get_file_info()"

6 years agoHandle snapshot artifacts in get_file_info() 29/12029/2
Anil Belur [Mon, 30 Jul 2018 02:29:22 +0000 (12:29 +1000)]
Handle snapshot artifacts in get_file_info()

This fixes lftools deploy maven-file to upload snapshot
artifacts to Nexus.

Issue: RELENG-1098
Change-Id: I5ed1c030b2b9c8f62e7f7e4ee0a63858782a747c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoAdd utility to check repos for commits missing DCO 27/11927/25
Jeremy Phelps [Mon, 23 Jul 2018 19:04:30 +0000 (14:04 -0500)]
Add utility to check repos for commits missing DCO

This only works on git repositories.

Change-Id: I45a0a66dfaaf517cdc5d85fc15f60e403bccfeb9
Co-Authored-By: Eric Ball <eball@linuxfoundation.org>
Co-Authored-By: Thanh Ha <thanh.ha@linuxfoundation.org>
Signed-off-by: Jeremy Phelps <jphelps@linuxfoundation.org>
6 years agoAdd optional classifier def to deploy cmd 38/11938/1
Thanh Ha [Mon, 23 Jul 2018 22:45:09 +0000 (18:45 -0400)]
Add optional classifier def to deploy cmd

The artifact/maven/content API supports an optional classifier
parameter so add option for it.

Change-Id: Ibf89ac2fb15e7fe289cb76aafad41936b24f3a9f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd cURL command to push an artifact to Nexus 44/11844/5
Thanh Ha [Wed, 18 Jul 2018 18:30:36 +0000 (14:30 -0400)]
Add cURL command to push an artifact to Nexus

Issue: RELENG-1081
Change-Id: If36810924d2d0f91e04bae469067495a25c0b2dc
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoSplit Nexus staged repo create & close functions 43/11843/8
Thanh Ha [Wed, 18 Jul 2018 18:05:44 +0000 (14:05 -0400)]
Split Nexus staged repo create & close functions

Split out these functions so that they can be reused.

Issue: RELENG-1081
Change-Id: I5a5c2ee10ed98f7c0c25134298e5a10bb7b23dd1
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoAdd pre-commit hook for trailing-whitespace 32/11832/1
Thanh Ha [Wed, 18 Jul 2018 01:31:17 +0000 (21:31 -0400)]
Add pre-commit hook for trailing-whitespace

Install using tox:

    tox -e check-hooks

ref: https://pre-commit.com

Change-Id: I2423e9619870b1b97c0082cd54aad06479dcf172
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Bump version to 0.15.0-dev for next dev cycle"
Andrew Grimberg [Mon, 16 Jul 2018 22:26:49 +0000 (22:26 +0000)]
Merge "Bump version to 0.15.0-dev for next dev cycle"

6 years agoMerge "Add options to continue interrupted downloads"
Andrew Grimberg [Mon, 16 Jul 2018 22:26:37 +0000 (22:26 +0000)]
Merge "Add options to continue interrupted downloads"

6 years agoBump version to 0.15.0-dev for next dev cycle 00/11800/1
Thanh Ha [Mon, 16 Jul 2018 18:29:37 +0000 (14:29 -0400)]
Bump version to 0.15.0-dev for next dev cycle

Change-Id: Ied5c6052b3fcede73129e8d5cb82f8bccea0afec
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoRelease lftools v0.14.0 76/11776/3 v0.14.0
Thanh Ha [Fri, 13 Jul 2018 15:38:53 +0000 (11:38 -0400)]
Release lftools v0.14.0

Change-Id: I36f0364dfad341e00d9d381a5369622ced34bd23
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add sigul signing capabilities to lftools"
Thanh Ha [Mon, 16 Jul 2018 00:58:50 +0000 (00:58 +0000)]
Merge "Add sigul signing capabilities to lftools"

6 years agoAdd sigul signing capabilities to lftools 73/11773/9
Andrew Grimberg [Fri, 13 Jul 2018 01:01:45 +0000 (18:01 -0700)]
Add sigul signing capabilities to lftools

Add in the ability to do sigul signing using lftools. This is an
extension of the current sign subcommand

Issue: RELENG-1070
Change-Id: Ie0cc0b2de0375f3c6306858a8ecb873e44888d42
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
6 years agoAdd configuration subsystem 73/10273/7
Thanh Ha [Mon, 30 Apr 2018 18:16:03 +0000 (14:16 -0400)]
Add configuration subsystem

Create a basic ini based configuration subsystem
to be used in the future by lftools subsystems
as a single point of pulling configuration from.

Usage:

  lftools config get section [OPTION]
  lftools config set section option value

Example:

  lftools config set global username zxiiro
  lftools config get global

Issue: RELENG-915
Change-Id: I346662ae2c623538e8a6881574dad357a8ae4dee
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoUpdate to python-jenkins 1.1.0 77/11777/1
Thanh Ha [Fri, 13 Jul 2018 15:42:53 +0000 (11:42 -0400)]
Update to python-jenkins 1.1.0

Pull in feature to detect groovy script failures allowing better
error detection and to actually return a proper error code when
a script fails.

Change-Id: I1e8735519060a024ab725373b3d76b11c89818d4
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoFix ShellCheck warnings for signing script 68/11768/4
Andrew Grimberg [Thu, 12 Jul 2018 22:44:25 +0000 (15:44 -0700)]
Fix ShellCheck warnings for signing script

In prepration for adding sigul signing, the shellcheck warnings need to
be cleaned up

Change-Id: I27479a28f2a7898e489c8a2198ffbf8cdea0cbef
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
6 years agoAdd options to continue interrupted downloads 24/11624/4
Anil Belur [Sat, 30 Jun 2018 08:47:42 +0000 (18:47 +1000)]
Add options to continue interrupted downloads

Add support to continue interrupted downloads with
`lftools sign deploy-nexus`. This allows to continue partially
downloaded files and when used with user specified path (-d).

RELENG-929
Change-Id: I58b576cdcea370f4ae1386529b90130140d6840e
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoUpdate Nexus string substition formating 42/11742/1
Andrew Grimberg [Wed, 11 Jul 2018 21:44:36 +0000 (14:44 -0700)]
Update Nexus string substition formating

Bring the string substitution formatting on Nexus code to the latest
standards for Python 3

Change-Id: I2c38cbbecc8b918aee968d46a12e251976cc8259
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
6 years agoCleanup nexus repo command for Python 3 17/11717/5
Andrew Grimberg [Tue, 10 Jul 2018 21:59:24 +0000 (14:59 -0700)]
Cleanup nexus repo command for Python 3

The original Nexus repo sub-command was written for Python 2 and needs
to be updated to work properly under Python 3.

Change-Id: I5558f833f4fc213e8eea64fc5a53ebcf1714d7be
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
6 years agoMerge "Add user-specified download path for lftools sign"
Andrew Grimberg [Tue, 10 Jul 2018 15:18:14 +0000 (15:18 +0000)]
Merge "Add user-specified download path for lftools sign"

6 years agoAdd user-specified download path for lftools sign 04/11604/6
Anil Belur [Fri, 29 Jun 2018 12:16:57 +0000 (22:16 +1000)]
Add user-specified download path for lftools sign

Add a user-specified download location for lftools sign
deploy-nexus command. The default download location
is "/tmp/gpg-signatures.*" where '*' is random generated
suffix added to a directory name returned by mkdtemp()

Projects like ODL require more than 5 GiB free space
which might not be available in the returned path
therefore we should allow the user to use an alternative
download path.

ex:

lftools sign deploy-nexus -d <path> <URL> <repo> <profile-id>

JIRA: RELENG-928
Change-Id: I960e6c2dd5b5311ec0551834c6b96a297a2eb8bd
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
6 years agoAdd remove-offline-nodes command 54/11554/2
Thanh Ha [Tue, 26 Jun 2018 00:58:59 +0000 (20:58 -0400)]
Add remove-offline-nodes command

Migrates the 2 scripts for removing offline nodes from global-jjb
jenkins-admin into lftools.

https://github.com/lfit/releng-global-jjb/blob/v0.21.4/jenkins-admin/del_computer.groovy
https://github.com/lfit/releng-global-jjb/blob/v0.21.4/jenkins-admin/slay_computer.groovy

Change-Id: Ifb96689274d91d2655fe3fe1122856c82a3edaf2
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoMerge "Add get-credentials groovy script to lftools"
Anil Belur [Wed, 4 Jul 2018 11:44:14 +0000 (11:44 +0000)]
Merge "Add get-credentials groovy script to lftools"

6 years agoAdd Nexus tags 73/11573/2
Jessica Wagantall [Wed, 27 Jun 2018 01:20:24 +0000 (18:20 -0700)]
Add Nexus tags

Fix first Nexus commands tag to remove namespace.
Add other needed tags for repo, create and reorder.

Change-Id: I187cca7d77227cf7fe7a42a7c46389cdd337f09c
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoAdd tag to Nexus docs 72/11572/1
Jessica Wagantall [Wed, 27 Jun 2018 00:49:57 +0000 (17:49 -0700)]
Add tag to Nexus docs

Add tag to Nexus commands documentation

Change-Id: Iba783cd83398d3ece0d73b94c8d2e716c35d5392
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoUpdate Nexus settings example 57/11557/1
Jessica Wagantall [Tue, 26 Jun 2018 18:31:17 +0000 (11:31 -0700)]
Update Nexus settings example

Update Nexus settings.example.yaml file to match
the convention used in the public docs when referring to
a project's domain.
Also, update http to https.

Change-Id: I7cc2e717ba507ea7d7eee577de9744ee57b2e0fd
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
6 years agoAdd get-credentials groovy script to lftools 53/11553/1
Thanh Ha [Tue, 26 Jun 2018 00:50:27 +0000 (20:50 -0400)]
Add get-credentials groovy script to lftools

This migrates the script in global-jjb into lftools as a command.

https://github.com/lfit/releng-global-jjb/blob/v0.21.4/jenkins-admin/get_all_creds.groovy

Change-Id: Ib60627e031b185b484ac0c3eeb99bf9d82470282
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoPrint result of groovy script to console 57/11157/1
Thanh Ha [Sun, 10 Jun 2018 23:16:42 +0000 (19:16 -0400)]
Print result of groovy script to console

Change-Id: I84824c34c8d13df55a40b8409a9174055ab8e5a9
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoIgnore protected images that are protected 52/11152/1
Thanh Ha [Sat, 9 Jun 2018 18:18:03 +0000 (14:18 -0400)]
Ignore protected images that are protected

Print a warning to inform the caller that we cannot delete
images that are protected.

Change-Id: I236f6f55964f483016500b58cbeb48fd8665b62b
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoPin python-jenkins to 0.4.16 06/11106/2
Thanh Ha [Thu, 7 Jun 2018 20:07:16 +0000 (16:07 -0400)]
Pin python-jenkins to 0.4.16

It seems python-jenkins 1.0.0 broke the run_scripts() function which
we use to run the jenkins-cfg commands. Pin to the older version
until we figure out why.

Change-Id: I9e643e3aafbc9eb680fd3b8181a0d8b4806060ac
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
6 years agoBump version to 0.14.0-dev 41/10941/2
Thanh Ha [Fri, 1 Jun 2018 17:52:24 +0000 (13:52 -0400)]
Bump version to 0.14.0-dev

Change-Id: Ib4313b61d77593aa9c1cf4256aef5ceec1bc1941
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>