releng/lftools.git
4 years agoRemove garbage var 96/63196/1 v0.31.0
Aric Gardner [Wed, 26 Feb 2020 17:29:22 +0000 (12:29 -0500)]
Remove garbage var

No memory of how this snuck in here, should be removed

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I8fa62db1ff6b481f865229643ac319413c8660f4

4 years agoMerge "Change the copy_archives method to use copy2"
Andrew Grimberg [Wed, 26 Feb 2020 17:16:13 +0000 (17:16 +0000)]
Merge "Change the copy_archives method to use copy2"

4 years agoChange the copy_archives method to use copy2 73/63173/3
Eric Ball [Sat, 22 Feb 2020 03:15:32 +0000 (19:15 -0800)]
Change the copy_archives method to use copy2

Uses copy2 with follow_symlinks set to False, which will copy
symlinks as symlinks, rather than following their path. Since glob
does not have an option to not follow symlinks, we instead just check
for file existence to ignore issues with globbing symlinked paths
that "open" fails to find.

Issue: RELENG-2576
Change-Id: Ie5a6ffba5c1cb79cff7dd9c691d9b9bd2eaeeba3
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
4 years agoTest released staging repo for closure 19/63119/4
DW Talton [Wed, 12 Feb 2020 20:59:53 +0000 (13:59 -0700)]
Test released staging repo for closure

Add long-polling to ensure that a released staging repo is fully
closed prior to exiting.

Issue-ID: RELENG-2695
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I032cccabe28ba7f5783b2ec82d73725d228ab797

4 years agoAdd nexus2 API endpoints 80/62780/13
DW Talton [Fri, 10 Jan 2020 21:03:59 +0000 (14:03 -0700)]
Add nexus2 API endpoints

Change-Id: I8ba68e819fbbcda8208e88d83007880cf543a194
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
4 years agoMerge "Add GitLint which replaces GitCommitBear"
Andrew Grimberg [Tue, 4 Feb 2020 21:48:14 +0000 (21:48 +0000)]
Merge "Add GitLint which replaces GitCommitBear"

4 years agoRemove extra request error checking 90/62990/1 v0.30.1
DW Talton [Wed, 29 Jan 2020 21:59:59 +0000 (14:59 -0700)]
Remove extra request error checking

Remove additional requests error checking. Current lftools needs
the ability to continue despite a http session error.

Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: Icad82ce16f7df7466e43be7e9035abac3fe63033

4 years agoAdd GitLint which replaces GitCommitBear 78/62978/1
Thanh Ha [Tue, 28 Jan 2020 15:58:50 +0000 (10:58 -0500)]
Add GitLint which replaces GitCommitBear

This replaces GitCommitBear with GitLint which does similar
linting for git commit messages but using pre-commit hooks.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I3480ad81702fa6b5015523a7274e219ba2f240d0

4 years agoMake output parsable 21/62921/3 v0.30.0
Aric Gardner [Thu, 23 Jan 2020 20:11:11 +0000 (15:11 -0500)]
Make output parsable

Need valid JSON in the output for automation work

-thanks DW

ISSUE-ID: RELENG-2683

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ie85adcec15577f432299846075068ed2a72783db

4 years agoEnable project_version_update 58/62858/11
Aric Gardner [Tue, 21 Jan 2020 18:37:26 +0000 (13:37 -0500)]
Enable project_version_update

Typo in the rtd docs api lead this to not be implemented.
corrected endpoint and implemented in CLI

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I09aee9e06e1b54c8f6c143762c501286627098d5

4 years agoImplement the Nexus 3 REST API in lftools 07/62607/16
DW Talton [Thu, 12 Dec 2019 00:26:02 +0000 (17:26 -0700)]
Implement the Nexus 3 REST API in lftools

Issue: RELENG-308
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I3dc1b1b7280fe6a2d69a75e2f8c7f5a9d64c5126

4 years agolftools github create-team no longer requires repo 70/62770/2
Aric Gardner [Thu, 9 Jan 2020 20:37:32 +0000 (15:37 -0500)]
lftools github create-team no longer requires repo

repo is now optional

ISSUE-ID: RELENG-2646
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I3cb31d17bb2fbe9e0ec7c419c99ad5cc2397d0f6

4 years agoFix reviewid logic 36/62636/2 v0.29.1
Aric Gardner [Mon, 16 Dec 2019 19:04:42 +0000 (14:04 -0500)]
Fix reviewid logic

When ammending a review we need to skip
create change not publish

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I3e2832ee3d228af12f337d63b49b66cb91357915

4 years agoMerge "Fix jenkins info job creation"
Aric Gardner [Thu, 12 Dec 2019 21:30:41 +0000 (21:30 +0000)]
Merge "Fix jenkins info job creation"

4 years agoMerge "Fix for empty INFO file creation"
Aric Gardner [Thu, 12 Dec 2019 01:50:10 +0000 (01:50 +0000)]
Merge "Fix for empty INFO file creation"

4 years agoFix jenkins info job creation 88/62588/1
Aric Gardner [Tue, 10 Dec 2019 17:36:55 +0000 (12:36 -0500)]
Fix jenkins info job creation

Fix jenkins info job creation.
Auto created info jobs need to go to the
"ci-managment" repo.

Also fix description on project creation.

ISSUE-ID: RELENG-2574
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Iab6860152c4623989997588b9a46804fc5693c2f

4 years agoGithub cli project specific tokens 86/62586/1
Aric Gardner [Tue, 10 Dec 2019 17:16:10 +0000 (12:16 -0500)]
Github cli project specific tokens

Github cli commands should first search for a
[github.organization] section.

Then fall back to a [github] section

ISSUE-ID: RELENG-2574
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I23957b43c50b4f6fda3483eec956f04afe9509e8

4 years agoFix for empty INFO file creation 17/62517/2
Aric Gardner [Thu, 5 Dec 2019 21:15:20 +0000 (16:15 -0500)]
Fix for empty INFO file creation

Do not require and lftools
[gerrit]
section.

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I118b4a945d6dd03231cdd7a2cbcf8cb82d35acee

4 years agoMerge "Create gerrit project via gerrit api" v0.29.0
Aric Gardner [Wed, 4 Dec 2019 16:58:36 +0000 (16:58 +0000)]
Merge "Create gerrit project via gerrit api"

4 years agoCreate gerrit project via gerrit api 17/62117/53
Aric Gardner [Wed, 30 Oct 2019 20:58:41 +0000 (16:58 -0400)]
Create gerrit project via gerrit api

internal jenkins implementation:
gerrit.linuxfoundation.org/infra/c/ci-management/+/62158

Extend lfidapi to exit with good message and code if
group does not exist. (check if group exists functionality)

Remove pythonsix compatability for config parser

Usage: lftools gerrit [OPTIONS] COMMAND [ARGS]...

  GERRIT TOOLS.

Options:
  --help  Show this message and exit.

Commands:
  abandonchanges              Abandon all OPEN changes for a gerrit
project.
  addfile                     Add an file for review to a Project.
  addgithubrights             Grant Github read for a project.
  addgitreview                Add git review to a project.
  addinfojob                  Add an INFO job for a new Project.
  createproject               Create a project via the gerrit API.
  list-project-inherits-from  List who a project inherits from.
  list-project-permissions    List Owners of a Project.

ISSUE: RELENG-2370
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I3a24751eeb2e739dee183074bbef6742df58a793

4 years agoSupport python 3.5.x 46/62446/1
Aric Gardner [Tue, 3 Dec 2019 21:57:21 +0000 (16:57 -0500)]
Support python 3.5.x

lftools must pin xdg to 3.x to support python 3.5
needed for onap ubuntu 16.04 builders

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: If97846bbb5a52410cfe23a66fb0f25c87b6bc5b4

4 years agoMerge "Add missing releasenote"
Bengt Thuree [Wed, 27 Nov 2019 02:07:01 +0000 (02:07 +0000)]
Merge "Add missing releasenote"

4 years agoAdd openstack cost command 82/62382/5
Tim Johnson [Fri, 22 Nov 2019 18:32:51 +0000 (10:32 -0800)]
Add openstack cost command

Issue: RELENG-2550
Change-Id: I9bee88cd5cb3d89f3f554470e86925d8f24dbb5a
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
4 years agoAdd .venv to .gitignore 02/62402/1
DW Talton [Tue, 26 Nov 2019 03:32:04 +0000 (20:32 -0700)]
Add .venv to .gitignore

Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I953896c94868ad1fbb39288be5a221c5541f9d26

4 years agoAdd HTTPBasicAuth support to REST client 68/62168/2
DW Talton [Tue, 5 Nov 2019 20:53:51 +0000 (13:53 -0700)]
Add HTTPBasicAuth support to REST client

Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I6467c8aeb2b2f24800ab1912ec6b485693a3378f

4 years agoAdd missing releasenote 88/62388/2
Bengt Thuree [Sun, 24 Nov 2019 22:39:07 +0000 (09:39 +1100)]
Add missing releasenote

Patch Id4efd6eaf870c9ce60d670094b6916b3c7b407cd did not have releasenote

Print rule failures for unclosed repos

Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Change-Id: Ibc624e6e7c52716178e8e16995d7065982539271

4 years agoMerge "Fix help text for github create team"
Aric Gardner [Fri, 22 Nov 2019 19:47:30 +0000 (19:47 +0000)]
Merge "Fix help text for github create team"

4 years agoFix help text for github create team 84/62384/1
Aric Gardner [Fri, 22 Nov 2019 19:03:04 +0000 (14:03 -0500)]
Fix help text for github create team

let user know to set privacy to closed

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I9f2b583eb266cb6678a56ce6c4678f3f23e1af76

4 years agoAdd --exact to releasedockerhub 74/62374/6
Bengt Thuree [Fri, 22 Nov 2019 00:13:06 +0000 (11:13 +1100)]
Add --exact to releasedockerhub

Enables working only on one repo, compared to wildcards or all.

  -e, --exact  Only exact match repo name will be used.
         If used, --repo parameter can not be empty

ISSUE: RELENG-2551

Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Change-Id: I774d2b206305229af76d1e78570d09849b6c836f

4 years agoMerge "Add "create role" subcommand for nexus"
Andrew Grimberg [Thu, 21 Nov 2019 22:28:59 +0000 (22:28 +0000)]
Merge "Add "create role" subcommand for nexus"

4 years agoMerge "Catch errors thrown by check_response_code"
Eric Ball [Thu, 21 Nov 2019 19:39:58 +0000 (19:39 +0000)]
Merge "Catch errors thrown by check_response_code"

4 years agoAdd extra output for lftools nexus release 28/62228/7
Bengt Thuree [Wed, 13 Nov 2019 02:05:26 +0000 (13:05 +1100)]
Add extra output for lftools nexus release

Add "Missing" value to the failure condition, as well as log it.
Refactored the code for finding errors as well.

ISSUE: RELENG-2453

Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Change-Id: Id4efd6eaf870c9ce60d670094b6916b3c7b407cd

4 years agoCatch errors thrown by check_response_code 50/62350/1
Eric Ball [Wed, 20 Nov 2019 02:02:24 +0000 (18:02 -0800)]
Catch errors thrown by check_response_code

For the current methods in lfidapi, we don't need to see the stack
trace or raise the error, so we should just catch and print it, and
then exit 1.
This also includes some minor cleanup.

Issue: RELENG-2024
Change-Id: Ife7f9b82be148a208c66134ef912bc091afac041
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
4 years agoAdd RTD project details update support 26/62326/2 v0.28.0
DW Talton [Mon, 18 Nov 2019 22:39:03 +0000 (15:39 -0700)]
Add RTD project details update support

Add the ability to update RTD project details from the CLI.

Issue: RELENG-2543
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: Ic165376cfcd7a81acb4ac602882d9d7925827c6a

4 years agoDrop support for python2.7 and 3.4(EOL) 14/62114/3
Aric Gardner [Wed, 30 Oct 2019 16:20:00 +0000 (12:20 -0400)]
Drop support for python2.7 and 3.4(EOL)

Drop support for python2.7 and python3.4(EOL)
lftools now requires python >= 3.6 This allows us to remove
remaining pins, and to move from glob2 to builtin glob

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I48ffc27376e3a7b56f0627e8eaeb9ac3dff908dc

4 years agoMerge "Rewrite DCO check in python"
DW Talton [Thu, 7 Nov 2019 17:52:21 +0000 (17:52 +0000)]
Merge "Rewrite DCO check in python"

4 years agoRefactor API client to support more auth types 67/62167/1
DW Talton [Tue, 5 Nov 2019 16:55:01 +0000 (09:55 -0700)]
Refactor API client to support more auth types

Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: Ic74499afa718f276db0007813cf77bf3c61d1ad8

4 years agoAdd "create role" subcommand for nexus 15/61815/8
Eric Ball [Wed, 22 May 2019 21:03:10 +0000 (14:03 -0700)]
Add "create role" subcommand for nexus

This enables users to create Nexus roles outside of project creation.

Add docs/commands/nexus.rst to WriteGoodLintBear ignore list, due to
issues with it failing the check based on example yaml code.

Issue: RELENG-290
Change-Id: Ie51e2bcd4d03221661b3cec79f8f62ea6d3e8c39
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
4 years agoRewrite DCO check in python 40/61840/18
DW Talton [Wed, 25 Sep 2019 17:38:57 +0000 (10:38 -0700)]
Rewrite DCO check in python

Move to python for more control and readability

Issue: RELENG-2374
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: If3c60718579fcb8f966c36fa212d23360c637864

4 years agolftools info file generator 80/62080/15
Aric Gardner [Thu, 24 Oct 2019 19:59:43 +0000 (15:59 -0400)]
lftools info file generator

Creates a preliminary info file to streamline workflow.
Missing fields are to be corrected in review.

Includes needed fixes for
lftools ldap for python3

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I3af2d768c356c8cd6950c527b67623e15032080e

4 years agoMerge "Update lftools docs to install pyhton3"
Aric Gardner [Tue, 22 Oct 2019 14:43:24 +0000 (14:43 +0000)]
Merge "Update lftools docs to install pyhton3"

4 years agoUpdate lftools docs to install pyhton3 81/61981/2
Anil Belur [Fri, 18 Oct 2019 15:45:27 +0000 (21:15 +0530)]
Update lftools docs to install pyhton3

Python2 is EOL soon, and its receommended to install pyhton3.

Issue:IT-17903
Change-Id: I8f81539846777f50ee219e656300dfa6c81109ec
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoUpdate pytest to python3 19/62019/1
Anil Belur [Thu, 17 Oct 2019 08:50:39 +0000 (14:20 +0530)]
Update pytest to python3

Change-Id: I46c6d4ad90c0e9e6874481a93a08d6c961dc81ae
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
4 years agoAdd support for RTD subprojects 54/61954/1 v0.27.1
DW Talton [Wed, 9 Oct 2019 18:26:31 +0000 (11:26 -0700)]
Add support for RTD subprojects

Add new rtd subprojects commands to allow for subproject CRUD operations.

Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I9004a1e9a38dd939d3752dfb9b2efb3ba366d113

4 years agoRefactored multithreaded nexus upload 25/16625/6
DW Talton [Tue, 20 Aug 2019 19:24:48 +0000 (12:24 -0700)]
Refactored multithreaded nexus upload

Refactored multithreaded nexus upload to use concurrent.futures
which allows for easy information gathering. Added additional
error code trapping to various requests methods, and overall
improved message output of the entire upload process from end
to end.

ISSUE: RELENG-2264

Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I0707342b2017cdfe0139738ddff5123b110ddec2

4 years agoSwitch volume delete to use the ID 10/61910/3 v0.27.0
Mohammed Naser [Thu, 3 Oct 2019 14:45:35 +0000 (10:45 -0400)]
Switch volume delete to use the ID

At the moment, the current code that's used when deleting volumes
uses a function that attempts to pull by ID, and then falls back
to name.

This is potentially dangerous because it might match more than one
volume and therefore potentially delete the wrong volume.  This
patch revises the code to enforce retrieving the volume based on
the ID instead which is globally unique.

The tools that leverage this inside releng-global-jjb already
provide an ID at the moment[1].

[1]: https://github.com/lfit/releng-global-jjb/blob/d22d069117b64cee73be1165861013de9e79ab19/shell/openstack-cleanup-orphaned-volumes.sh#L18

Change-Id: I078c8c4447ac308c66678e86ae31e8947f40340e
Signed-off-by: Mohammed Naser <mnaser@vexxhost.com>
4 years agoRemove python27 from tox env. 61/61861/8
Aric Gardner [Fri, 27 Sep 2019 17:00:09 +0000 (13:00 -0400)]
Remove python27 from tox env.

Remove py27 pins
fix LFTOOLS_CONFIG_FILE to use os.path

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Id237c116cecf80c7d827413ac3d63c58960f7b45

4 years agoAdd support for Read the Docs API v3 23/61523/14
DW Talton [Thu, 29 Aug 2019 01:29:37 +0000 (18:29 -0700)]
Add support for Read the Docs API v3

Add support for RTD APIv3, which allows for PUT and POST
operations (v2 was read-only). Support added via modular
API client with RTD module

Issue: RELENG-2239
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: Iaad90ed360eaac0de0e7a6c5a4c6fa77ab349f70

5 years agoRework DCO with check and match commands 42/16642/9
DW Talton [Thu, 22 Aug 2019 01:59:06 +0000 (18:59 -0700)]
Rework DCO with check and match commands

The DCO shell script now has two commands, 'check' and 'match'.
Check will scan a git repo for missing DCO signatures and output
the results to the console. Match checks commits that do have
DCO signatures and confirms that the signature matches the commit
author's email address

Issue: RELENG-1980
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I36f57ccdc993df9923dc2d9102b7bac4c92e2739

5 years agoAdd lxml to requirements.txt to lftools 17/16717/2 v0.26.2
Anil Belur [Wed, 28 Aug 2019 06:01:58 +0000 (11:31 +0530)]
Add lxml to requirements.txt to lftools

'lftools nexus release -v --server https://nexus.opendaylight.org
odlparent-xxxx' returns the following error.

Error:
bs4.FeatureNotFound: Couldn't find a tree builder with the features
you requested: xml. Do you need to install a parser library?

Change-Id: Icf92537c338870cfa2508702ecbaaa5d4cf67edb
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoConfigure basepython to python3 07/16707/1
Thanh Ha [Tue, 27 Aug 2019 19:11:02 +0000 (15:11 -0400)]
Configure basepython to python3

More and more libraries that we are pulling in require Python 3 so
to avoid having to play the pin dependencies whack-a-mole game let's
bump or base python where possible to python3.

This also renames the pre-commit env to the standard "pre-commit" name
that we use in other projects.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I66d1e72390ba682d0e7bd6fdbbb1a7cc3c8f929d

5 years agoDo not copy sub-directory 51/16651/5 v0.26.1
Bengt Thuree [Fri, 23 Aug 2019 14:19:55 +0000 (19:49 +0530)]
Do not copy sub-directory

Create the missing paths as needed when copying files.

Fixes
lftools.deploy:265: Missing path, will create it <dir>
ERROR: [Errno 17] File exists: '<dir>'

Issue: IT-17315
Change-Id: Ic1dd29f7cb51ef1ca7a8740974e5b877a1348522
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Changes needed in lftools for github orgs" v0.26.0
Andrew Grimberg [Wed, 14 Aug 2019 16:20:34 +0000 (16:20 +0000)]
Merge "Changes needed in lftools for github orgs"

5 years agoexit if errors are found in a nexus repo 13/16513/12
Aric Gardner [Fri, 9 Aug 2019 21:04:45 +0000 (17:04 -0400)]
exit if errors are found in a nexus repo

nexus release now checks "{}/staging/repository/{}/activity"

Ensures that Repository is in closed state
Checks if Repository is already released (exit 0)
Check for failures, if found (exit 1)

Added
click.option('-v', '--verify-only', is_flag=True, required=False)
if -v is passed, only checks for errors, skips release.

This could be better, but I have officially spent too much time
fiddling with it.
Its a bit of a brute force, but as

curl -v -u <lfid:pass>
'https://nexus.opendaylight.org/\
service/local/staging/repository/honeycomb-vbd-1629/activity'

Actually returns two xml pages.
not being too familliar with soup I used
soup.find_all("value")
and then used a re.search
to check for needed values

Removing the pytests, they don't make sense imo.

ISSUE: RELENG-2291
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I7ae6c487b2aaf0e1df01effa7da15a1b28b624ad

5 years agoChanges needed in lftools for github orgs 62/16262/15
Aric Gardner [Thu, 18 Jul 2019 20:42:51 +0000 (16:42 -0400)]
Changes needed in lftools for github orgs

Part of the effort for self serve user management

check-votes now works against a a githubpr
Note: You can't vote on your own pr so I add the pr owner as
a voter automatically.

infofile match-ldap-to-info now works against github groups

github submit-pr will submit a pr if mergable

github votes will count votes on a pr

Fixes a bug in github team invitation.
Teams needs to be array of :class:`github.Team.Team`

adds: --add_team and --remove_team to  github update-repo
so that the replication user can be added to each new repo
in an automated way

ISSUE: RELENG-1864
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I65d30dab1d1638123fdb46bc705ec6190cb3182a

5 years agoSupport multiple nexus sections in lftools.ini 56/16456/10 v0.25.5
Aric Gardner [Tue, 6 Aug 2019 17:32:42 +0000 (13:32 -0400)]
Support multiple nexus sections in lftools.ini

[nexus] section is taken from -s "server" passed to release job.
The "https://" part of passed url is stripped before match.

example:
lftools nexus release -s https://nexus.example.com

Would require auth section in lftools.ini of
[nexus.example.org]
username=
password=

Issue: RELENG-2129
Issue: RELENG-1666
Change-Id: I64ab0a18a4d0e9060f1273f5acc1971fcffe497c
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoRemove drop of staging repos on release 48/16448/4 v0.25.4
Aric Gardner [Mon, 5 Aug 2019 22:35:34 +0000 (18:35 -0400)]
Remove drop of staging repos on release

The api returns that the relese is completed.
in the background java threads are still running.

Then we call drop and nexus has threads promoting and dropping
at the same time.

In this way we lose data.

Something else needs to drop, the api does not correctly
handle this.

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I3ad52502d6ac44960f697561eb82e2b6896ac781

5 years agoInclude auth credentials with release requests 23/16423/1 v0.25.3
Eric Ball [Fri, 2 Aug 2019 21:08:06 +0000 (14:08 -0700)]
Include auth credentials with release requests

Also includes a small logging improvement. During troubleshooting, I
found that it wasn't clear whether the credentials were being picked
up successfully from lftools.ini. Having some feedback during a
handled error will be much more helpful.

Change-Id: I2fa49f11ccb242296db9875e3018a8adf8b70f18
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoFix commit message length 86/16386/1
Aric Gardner [Wed, 31 Jul 2019 19:04:08 +0000 (15:04 -0400)]
Fix commit message length

Also provide better error message on failure
to push .gitreview to a new repository

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I758d35ba9ca6d95828d2b1a341cc44e39476d742

5 years agoRe-add display of group members 29/16229/1
Andrew Grimberg [Tue, 16 Jul 2019 20:59:46 +0000 (13:59 -0700)]
Re-add display of group members

Change I76a5db63d584ab84a1ad12bfde1daffde0b619b0 removed the print out
of the members of groups from the search-members due to having moved to
the logging module and the output having been debug only output.

Since the point of search-members is to list out the members we need to
output the list.

Change-Id: Ib7ae9107a2db8da96067eb904982fafb8c7f10d7
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoMerge "Fix ldap check in gerrit create"
Andrew Grimberg [Fri, 12 Jul 2019 15:02:15 +0000 (15:02 +0000)]
Merge "Fix ldap check in gerrit create"

5 years agoFix ldap check in gerrit create 51/16151/1
Aric Gardner [Wed, 10 Jul 2019 18:10:26 +0000 (14:10 -0400)]
Fix ldap check in gerrit create

ldap csv is not needed as we have lfidapi

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ie3c29138ebd5562155afd43958da94bf95714652

5 years agoChange out lfidapi module print statements 59/16059/6
Andrew Grimberg [Tue, 2 Jul 2019 22:36:42 +0000 (15:36 -0700)]
Change out lfidapi module print statements

Switch to using the logging facility so that we can more easily switch
our output streams and also not through debugging output when we don't
need it.

Also add in some useful log.info where it is appropriate.

Change-Id: I76a5db63d584ab84a1ad12bfde1daffde0b619b0
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoFix retyping issue of function 58/16058/5
Andrew Grimberg [Tue, 2 Jul 2019 22:23:01 +0000 (15:23 -0700)]
Fix retyping issue of function

There was a subtle bug where a function call was being overwritten by a
local variable of the same name and then a call to the function was
being attempted.

Change-Id: I7316df0c773db347ccb1e7eb79e2f40fee3a9db2
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoPin pytest to 4.6.x 60/16060/1
Andrew Grimberg [Tue, 2 Jul 2019 22:55:51 +0000 (15:55 -0700)]
Pin pytest to 4.6.x

PyTest 5 was released on June 28th that now requires Python >= 3.5
so pin to the 4.6.x stream until we are ready to drop 2.7 support.

Change-Id: If25020214f778390c4e2705fbf454ce4db59791c
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoMerge "Use requests.put for Nexus 2+3 compatibility" v0.25.2
Eric Ball [Mon, 1 Jul 2019 17:12:49 +0000 (17:12 +0000)]
Merge "Use requests.put for Nexus 2+3 compatibility"

5 years agoUse requests.put for Nexus 2+3 compatibility 60/15960/2
Eric Ball [Thu, 20 Jun 2019 17:39:24 +0000 (10:39 -0700)]
Use requests.put for Nexus 2+3 compatibility

While both post and put methods work for deploying to Nexus 2, Nexus 3
is not compatible with post. Change to put for all deploy_nexus
uploads.

Change-Id: Ib07ba6b72b0c05afe226f7ce4c36f021535d4bba
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoRevert plugin listing print change 06/16006/7 v0.25.1
Andrew Grimberg [Wed, 26 Jun 2019 20:23:59 +0000 (13:23 -0700)]
Revert plugin listing print change

Change 5df955f24bc0154f7069ecfc188311052e67febc had a minor change that
slipped through the code review changing the Jenkins plugin listing from
long names to using the short names. This however, is not utilized at
all by the security listing function that was the primary portion of
this change.

This modification causes the jenkins configuration management merge job
to fail on updates to the OpenStack cloud configuration.

Change-Id: I04cfbfe5a49904875a734e223bda13c6164df664
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoAdd --force option to stack delete 30/15930/1
Anil Belur [Wed, 19 Jun 2019 00:42:08 +0000 (10:42 +1000)]
Add --force option to stack delete

Add --force option to stack delete to improve error
handling in the builder-openstack-cron job.

Issue: RELENG-2058
Change-Id: I3f9918000ce615d765c27d9334c161a1cdf2443c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoFix deploy_logs unicode handling for Python 2 & 3 86/15886/1 v0.25.0
Eric Ball [Thu, 13 Jun 2019 18:25:39 +0000 (11:25 -0700)]
Fix deploy_logs unicode handling for Python 2 & 3

six.text_type will always return a utf-8 encoded string, while
str.encode will return a bytes object in Python 3. io.open is
used to ensure that the file is being opened for utf-8 writing in
Python 2. decode('utf-8') is used on the requests response due to
how requests encodes unicode responses.

Change-Id: If58f8a3b9617f0be4effc58414d22f0cf09946cb
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agolftools deploy archives handle dup patterns/file 49/15849/10
Bengt Thuree [Thu, 13 Jun 2019 00:51:35 +0000 (10:51 +1000)]
lftools deploy archives handle dup patterns/file

If the pattern is not properly done, the resulting file list might
contain duplicated files.

This fix will remove the duplicated patterns, as well as
duplicated matched files.

This fix should fix the OSError exception in mkdir

Change-Id: Ib7ac21ecdec2fdefe84b710a90666b70d966be12
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoHandle archives is a file instead of dir. 30/15830/6
Bengt Thuree [Thu, 13 Jun 2019 00:51:56 +0000 (10:51 +1000)]
Handle archives is a file instead of dir.

This fix raises an Exception, and exists lftools with (1),
if there is any issues with the Archive directory (missing,
a file instead of directory, or something else)

Change-Id: I3102bd2bd06d14a2e420aada64d76d280221db49
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoRevert "Use six.text_type rather than str.encode" 74/15874/2
Anil Belur [Wed, 12 Jun 2019 23:54:24 +0000 (09:54 +1000)]
Revert "Use six.text_type rather than str.encode"

This reverts commit e99dca16eef12f2e09c7911cc145ed637f9584bc.

The change was tested with ODL merge job and does not work.
Reverting this until we have this properly tested with the
jobs.

Change-Id: If1382188753ff590c767ce4ab5770f2c271fda69
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoFix release notes typos 68/15868/2
Jessica Wagantall [Wed, 12 Jun 2019 00:16:26 +0000 (17:16 -0700)]
Fix release notes typos

Fix release notes typos for commit:
9d64c0644c4df403243eb89014d6257827d9d281

"Get jenkins config only with the key exists"

Change-Id: I78a567ffae6c5333021dd86e5b2aa8db8c3fc6be
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
5 years agoMerge "Get jenkins config only with the key exists"
Andrew Grimberg [Tue, 11 Jun 2019 21:43:00 +0000 (21:43 +0000)]
Merge "Get jenkins config only with the key exists"

5 years agoGet jenkins config only with the key exists 51/15851/5
Anil Belur [Tue, 11 Jun 2019 02:53:10 +0000 (12:53 +1000)]
Get jenkins config only with the key exists

Handle config parser correctly which defaults to "[jenkins]" section
when no server is pased. This fixes the issue with checking if the key
exists in the configuration read before reading the key-value.

The issue is reproduceable by running `lftools jenkins plugins --help`
or `tox -e docs`, with the ``jenkins.ini`` file having no "[jenkins]"
section.

Use SafeConfigParser which is a saner version for parsing config files.

Issue: RELENG-2120
Change-Id: I274ac46cf7a1d03cf7c3c6fdaec6094b5e02aa42
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Use six.text_type rather than str.encode('utf-8')"
Thanh Ha (zxiiro) [Tue, 11 Jun 2019 01:47:12 +0000 (01:47 +0000)]
Merge "Use six.text_type rather than str.encode('utf-8')"

5 years agoUse six.text_type rather than str.encode('utf-8') 40/15840/3
Eric Ball [Fri, 7 Jun 2019 13:59:20 +0000 (06:59 -0700)]
Use six.text_type rather than str.encode('utf-8')

Encoding the string in Python 3 results in a bytes object rather
than str. To avoid this, use six.text_type method, which is designed
to safely perform this function in Python 2 and 3.

Change-Id: Ice7f02f4c02fd71d115af627d5b15622cd04d35d
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoMerge "Github api tools"
Andrew Grimberg [Mon, 10 Jun 2019 20:07:33 +0000 (20:07 +0000)]
Merge "Github api tools"

5 years agoGithub api tools 64/15264/30
Aric Gardner [Thu, 11 Apr 2019 20:35:20 +0000 (16:35 -0400)]
Github api tools

create-repo [OPTIONS] ORGANIZATION REPOSITORY DESCRIPTION
Create a Github repo within an Organization.
By default has_issues has_wiki and has_projects are set to false. See
--help to create a repo with these enabled.

modify-repo [OPTIONS] ORGANIZATION REPOSITORY
Modify a Github repo within an Organization.
By default has_issues has_wiki and has_projects are set to false. See
--help to use this command to enable these options

create-team [OPTIONS] ORGANIZATION NAME REPO PRIVACY
Create a Github team within an Organization.

list         List an Organization's GitHub repos.
  --audit          list members without 2fa
  --repos          list all repos
  --full           All members and their respective teams
  --teams          List avaliable teams
  --repofeatures   list enabled features for repos in an org
  --help           Show this message and exit.

user [OPTIONS] ORGANIZATION USER TEAM
Add and remove users
  --delete  remove user from a team within an org
  --admin   user is admin for org
  --help    Show this message and exit.

Github token is read from
~/.config/lftools/lftools.ini
In the form of:

[github]
token =

ISSUE: RELENG-1526
Change-Id: I558eb11897b555d26fac5abacb660d4bdb630a12
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
5 years agoMerge "lfidapi create group checks if group exists"
Andrew Grimberg [Fri, 7 Jun 2019 16:45:41 +0000 (16:45 +0000)]
Merge "lfidapi create group checks if group exists"

5 years agoMerge "Move numpy dependency into coala linter" v0.24.0
Anil Belur [Thu, 6 Jun 2019 21:56:56 +0000 (21:56 +0000)]
Merge "Move numpy dependency into coala linter"

5 years agoMove numpy dependency into coala linter 28/15828/1
Thanh Ha [Thu, 6 Jun 2019 21:25:37 +0000 (17:25 -0400)]
Move numpy dependency into coala linter

The NLTK upstream library started depending on numpy so pull it
in when calling Coala.

Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I50c4ee38eb84e570551ebc3dd8bd9d9d5cb4e473

5 years agoInclude *.{html,xml} file extensions to compress 93/15793/7
Anil Belur [Thu, 6 Jun 2019 14:05:51 +0000 (00:05 +1000)]
Include *.{html,xml} file extensions to compress

Add file extensions `*.{html,xml}` to ensure they are compressed.
`*.xml` files can be quite large on log server and this ensures
that log files are compressed on Nexus have a smaller foot print.

Issue: RELENG-2072
Change-Id: I64bcf7f75f45ba22d0fca50d5df55fa156193580
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoadd numpy to requirements 27/15827/1
Aric Gardner [Thu, 6 Jun 2019 17:48:27 +0000 (13:48 -0400)]
add numpy to requirements

I dont understand why this error suddenly appeared.
Or what is pulling in NumPy

But this fixed it for a local run

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I5b9b69a91919135268e41a7ea6a7cc775a83e580

5 years agolfidapi create group checks if group exists 26/15826/2
Aric Gardner [Thu, 6 Jun 2019 17:25:14 +0000 (13:25 -0400)]
lfidapi create group checks if group exists

There is a bug in lfidapi where if a group exists
the reponse code does the right thing but some
shadow group is created.

Add a check so that we only create a group
if it does not yet exist.

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ia80400cf74431a8ffdea1ae06459969649d94501

5 years agoFix release notes format 22/15822/1
Anil Belur [Thu, 6 Jun 2019 00:22:54 +0000 (10:22 +1000)]
Fix release notes format

The change 6624bcb3191fdbe45bc7ae1c570a1b7873a5d193
breaks the verify jobs

Change-Id: I70778a7ed26c9c197b431a1f399eaeaae70a2baf
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
5 years agoMerge "Fix token generation."
Andrew Grimberg [Sun, 2 Jun 2019 19:08:42 +0000 (19:08 +0000)]
Merge "Fix token generation."

5 years agoMerge "List insecure plugins"
Andrew Grimberg [Sun, 2 Jun 2019 19:07:48 +0000 (19:07 +0000)]
Merge "List insecure plugins"

5 years agoPin detox 00/15800/1
Andrew Grimberg [Sun, 2 Jun 2019 00:24:27 +0000 (17:24 -0700)]
Pin detox

detox 0.19 is not compatible with the version of tox we are getting from
upstream

Change-Id: Icc0fd40057c08e5241cd4d06c6f4ee57f7c0e7ec
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
5 years agoList insecure plugins 12/15712/12
Aric Gardner [Fri, 17 May 2019 21:34:30 +0000 (17:34 -0400)]
List insecure plugins

List active plugins that have a known
vulnerability

Example output:

$ lftools jenkins -s 'build.opnfv.org/ci' plugins sec
pam-auth:1.5    pam-auth:1.5
https://jenkins.io/security/advisory/2019-05-21/#SECURITY-1316
credentials:2.1.18      credentials:2.1.18
https://jenkins.io/security/advisory/2019-05-21/#SECURITY-1322

jenkins_jobs.ini config
$ cat ~/.config/jenkins_jobs/jenkins_jobs.ini | grep build.opnfv.org/ci
[build.opnfv.org/ci]
user=
password=
url=https://build.opnfv.org/ci

ISSUE: RELENG-2046
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I807a1e33d10c42c0d48f7fd179858eebc368c1e8

5 years agoFix token generation. 50/15750/4
Aric Gardner [Thu, 23 May 2019 16:36:25 +0000 (12:36 -0400)]
Fix token generation.

Remove legacy method of token generation.

username and password come from global section of lftools.ini.
[global]
username = lfid
password =

url comes from -s section of jenkins_jobs.ini
eg: [jenkins.acumos.org/sandbox]
url=jenkins.acumos.org/sandbox

example command
lftools jenkins -s 'jenkins.acumos.org/sandbox' token change

option --name set token name default is token-created-by-lftools

changeApiToken no longer exists, so we must allways generate a
new token.

NOTE: token print and token change now do the same thing.
That is generate a new token and return it to console.
I could remove one of them but Im wary of breaking any
legacy scripts that may exist.

NOTE: This will not work with SAML Plugin jenkins.
But I need it for automation before we switch all
jenkins to SAML

ISSUE: RELENG-2046
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I075c530f6e9fad7b477383867d52b83bba745662

5 years agoAdd documenation on lfidapi configuration 44/15344/6
Aric Gardner [Fri, 19 Apr 2019 18:20:43 +0000 (14:20 -0400)]
Add documenation on lfidapi configuration

Documentation on [lfid] for lftools was missing.

Change-Id: Ia8e4e41fc4412edc7b5153f6e5e51a822875d889
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
5 years agoNote in git-tag docs that tag must be annotated 42/15742/1
Eric Ball [Wed, 22 May 2019 21:12:27 +0000 (14:12 -0700)]
Note in git-tag docs that tag must be annotated

This caused a significant holdup in testing this function. Though
this is a very small change, I believe it can potentially save
future users a lot of frustration.

Change-Id: I1994391ab5cabdbec4aa7e751a5663f6de726f4d
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoTitles wrong for gerrit infofile and lfid commands 42/15642/1
Aric Gardner [Mon, 13 May 2019 16:32:21 +0000 (12:32 -0400)]
Titles wrong for gerrit infofile and lfid commands

I noticed that I had done the titles incorrectly via some
copy paste lazyness in the docs.

Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I3da9d6cfa7513b3aedcb68e8b6bed02823b31271

5 years agoRemove python 3.4 support 02/15302/1
Aric Gardner [Tue, 16 Apr 2019 20:38:08 +0000 (16:38 -0400)]
Remove python 3.4 support

Python 3.4 is EOL
And no longer receives security updates

Change-Id: Iefe7efae8614062b1fcf54b2555ffe40652502cd
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
5 years agoMerge "Update docs for git-tag and container signing"
Andrew Grimberg [Fri, 12 Apr 2019 20:57:10 +0000 (20:57 +0000)]
Merge "Update docs for git-tag and container signing"

5 years agoUpdate docs for git-tag and container signing 68/15268/1
Eric Ball [Fri, 12 Apr 2019 20:00:54 +0000 (13:00 -0700)]
Update docs for git-tag and container signing

This also reorganizes subcommands into alphabetical order in order
to match the "lftools sign --help" output.

Issue: RELENG-1851
Change-Id: I67a3a875d688ce47ab952ad82f39e39fb7093b1f
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
5 years agoMerge "Add git tag and Docker container sigul signing" v0.23.0 v0.23.1
Anil Belur [Sun, 7 Apr 2019 23:47:48 +0000 (23:47 +0000)]
Merge "Add git tag and Docker container sigul signing"

5 years agoAdd git tag and Docker container sigul signing 94/15194/4
Eric Ball [Thu, 4 Apr 2019 13:56:25 +0000 (06:56 -0700)]
Add git tag and Docker container sigul signing

Issue: RELENG-1851
Change-Id: I02ecde037ff603dafd5820f15b454f2a0f2571f1
Signed-off-by: Eric Ball <eball@linuxfoundation.org>