Aric Gardner [Tue, 28 Jul 2020 18:01:49 +0000 (14:01 -0400)]
Fix: openstack image upload defaults to qcow2
Openstack cmd and upload should default to raw not qcow.
builds were timing out. Internally ceph is converting from
qcow2 to raw. Which can cause the time outs.
Issue-ID: RELENG-3102
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I78b2fe7bcc6e6f631013cf1d58455cf17cbea291
Anil Belur [Mon, 27 Jul 2020 01:37:17 +0000 (11:37 +1000)]
Pin osc-lib to 2.2.0 to allow sharing images
Using lftools openstack image share returns an error
"You are not authorized to find project with the name".
The issue is with osc_lib bug [1] resolved in osc_lib==2.2.0
[1] https://bugs.launchpad.net/ubuntu/+source/python-openstackclient/+bug/
1864203
[2] https://opendev.org/openstack/osc-lib/commit/
1ff3720daefd98a77557e5692fd7052b5930ae6c
Issue: RELENG-3099
Change-Id: I5e70443007d1ed6074802e502edb149008fcdced
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Eric Ball [Thu, 16 Jul 2020 00:47:52 +0000 (17:47 -0700)]
Remove pinned distlib requirement
Distlib is a common requirement for other libraries, and having it
pinned is causing failures in builds. It is not explicitly used in
lftools, so it does not need to be pinned.
Issue: RELENG-3071
Change-Id: I6a6a33585b1666b93656a43f16fb1db7e7b4f584
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Thanh Ha [Sun, 14 Jun 2020 19:12:24 +0000 (15:12 -0400)]
Remove dependency on pre-commit
lftools has 0 dependencies on pre-commit and thus should not be
listed as one of its requirements. lftools simply uses pre-commit
as a linter which the dependency is already covered by the tox.ini
deps configuration.
Commit
54d1936f introduces what seems like an exceptionally large
list of dependencies which should be re-evaluated if lftools really
depends on any of those packages directly as some of those deps
are likely not really deps of lftools. This patch simply takes care
of pre-commit as I don't currently have time for a full audit.
Change-Id: I18acffc492ba6905e639cdb452dee0c2f5f2fcbb
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Thanh Ha [Sat, 6 Jun 2020 13:52:04 +0000 (09:52 -0400)]
Update pre-commit hooks 2020-06
It's been awhile since we updated pre-commit hook versions so let's
catch up with the world.
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I19d99ad86e8f60478af8daae2acfe19911b254ca
Thanh Ha [Sat, 6 Jun 2020 13:50:47 +0000 (09:50 -0400)]
Setup gitlint
Run gitlint to lint commit messages in the lftools repo.
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I3daf64a3a234e191d035c583d6e752fa060accae
Thanh Ha [Sat, 6 Jun 2020 13:48:59 +0000 (09:48 -0400)]
Print git-diff when pre-commit fails
When pre-commit fails and automatically fixes issues for us in Jenkins
we are not able to see what pre-commit automatically fixed. Adding
--show-diff-on-failure makes pre-commit perform a git-diff immediately
after failing to ensure more useful output is printed to inform the
contributor what changes are needed to pass the check.
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: Icd6484c8c4157d38885b1de460517d9b656a06d7
Thanh Ha [Sat, 6 Jun 2020 14:13:38 +0000 (10:13 -0400)]
Use black version 19.10b0
The stable branch for some reason is failing to fetch in Jenkins build
now so let's switch to a tagged release.
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I57c829e1cd831ae62cb5be1ca4414f144309005b
Eric Ball [Thu, 28 May 2020 15:22:42 +0000 (08:22 -0700)]
Fix string limit issue in _request_put_file
Requests can't handle a put call for very large data objects. However, it
can accept the data as a file-like object instead, and the size issue will
not show up. Documented here: https://github.com/psf/requests/issues/2717.
Issue: IT-19717
Change-Id: I826d3fa2eebbd3ba0389a0d7042701b4389e406d
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Aric Gardner [Mon, 25 May 2020 19:00:58 +0000 (15:00 -0400)]
Fix voting math
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I450270710e67d08873f9bd40440417879f5f21c7
Anil Belur [Tue, 28 Apr 2020 20:51:57 +0000 (20:51 +0000)]
Merge "Add ODL exception"
Aric Gardner [Tue, 28 Apr 2020 20:31:01 +0000 (16:31 -0400)]
Add ODL exception
umbrella name does not match ldap name
add exception
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I7edada167774a3078f86e1818bba70230bcc9c59
Houa Yang [Tue, 24 Sep 2019 20:51:50 +0000 (15:51 -0500)]
Add S3 log shipping subcommand
Add new click command for logs-s3.
Issue-Id: RELENG-2379, RELENG-2380
Signed-off-by: Houa Yang <hyang@contractor.linuxfoundation.org>
Change-Id: I4295272cd903c9cc33962f2ee54667c28c3132e0
Anil Belur [Tue, 7 Apr 2020 01:25:44 +0000 (11:25 +1000)]
Add note to check for DCO exit status
Issue: RELENG-2764
Change-Id: I36f8cb328d4ecd42338a26e33a53b3a3ea7d3598
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
DW Talton [Mon, 6 Apr 2020 17:04:00 +0000 (17:04 +0000)]
Merge "Add utils command"
DW Talton [Sun, 29 Mar 2020 21:15:01 +0000 (14:15 -0700)]
Add utils command
Add utils command to hold utilities like password generators
and whatnot.
Issue-ID: RELENG-2754
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I968c73f52a87c4436c355a1b9b66a49128a3bf56
DW Talton [Fri, 3 Apr 2020 20:40:18 +0000 (13:40 -0700)]
Add password option to Nexus2 add user
Add the ability to optionally pass in password on user account
creation.
Issue-ID: RELENG-2755
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: Iacda5eadb3f04503bf26ca551c990b6173d1ec0f
Aric Gardner [Tue, 31 Mar 2020 18:56:58 +0000 (14:56 -0400)]
Changes for nexus2 automation
Allow more permissive regex match
project names like
integration-simulators-pnf-simulator
should allow . or - in their pom.xml
Enabled by strict_url_regex=False in config file
Now allows passing a url in the format https://
and sourcing nexus user and pass from lftools.ini
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I85d40f5964a2588093febd29ba9a1d25bc3e1046
DW Talton [Tue, 31 Mar 2020 19:31:55 +0000 (12:31 -0700)]
Add Nexus3 user capabilities
Add Nexus3 CRUD user capabilities. Also includes password generator
helper.
Issue-ID: RELENG-2701
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I47fbf7939edb883a07b8aa421e776062317d6103
Thanh Ha [Tue, 6 Nov 2018 07:10:05 +0000 (15:10 +0800)]
Use development mode for tox testenv
Speeds up tests slighly as it skips the sdist install.
Change-Id: Ic4d4b860b373b1d1cd18ca2375971422f0034c4f
Signed-off-by: Thanh Ha <thanh.ha@linuxfoundation.org>
Andrew Grimberg [Tue, 24 Mar 2020 20:44:45 +0000 (20:44 +0000)]
Merge "Fix potential symlink loop in _compress_text()"
Eric Ball [Fri, 20 Mar 2020 23:26:47 +0000 (16:26 -0700)]
Fix potential symlink loop in _compress_text()
By checking if os.path.exists can find the path, we stop the method
from attempting to traverse symlinked directories.
This also pins a more recent version of pre-commit, as the older
version was making a pre-commit file with errors.
Issue: RELENG-2576
Change-Id: I2265c9895444892877f684b807287ebe8319083a
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Aric Gardner [Fri, 20 Mar 2020 20:59:46 +0000 (16:59 -0400)]
Not sure why lftools requires virtualenv
In any case this breaks all tox runs.
as tox-install updates the virtualenv package.
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I78b07ee249c7201e99a3dbf2881910443dc94161
Aric Gardner [Mon, 16 Mar 2020 19:40:54 +0000 (19:40 +0000)]
Merge "Tweak initial project yaml file"
Aric Gardner [Wed, 11 Mar 2020 21:58:58 +0000 (17:58 -0400)]
Tweak initial project yaml file
Adds a project view
drop branch (it is ignored)
dont append the file with -info
we will treat this as the projects first jjb file
projects can build off of it.
also change default builder if we are o-ran
also fix example text, we use fqdn now
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ie7b56ad88769cd6ed6494162fc80ab8e8426392f
Andrew Grimberg [Fri, 13 Mar 2020 20:51:42 +0000 (13:51 -0700)]
Fix initialization value of stack_cost
After having recently fixed the job-cost script in
1c99557ff1b60356f85193d00e23020bb9e53467 I realized that I should check
over the stack_cost generation. Much to my surprise I found that the
cost was being initialized to 0.7! This means that all stack costs that
we've generated so far have been increased by $0.70 in the reported
cost, job runs that have no stacks are still properly reporting $0 for
the cost.
This change fixes the initialization to be properly 0.0.
Issue: RELENG-2719
Change-Id: I8474604dbe0e2b32165b086754e29fac17f79785
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
DW Talton [Fri, 28 Feb 2020 20:30:59 +0000 (13:30 -0700)]
Replace python test harness
This commit is a comprehensive overhaul of the test harness used
in lftools. First, it removes coala. Black is added as a defacto
code formatter to ensure consistency across all commits. Black is
added as a pre-commit hook. Flake8 is added in conjunction with
black, and configured according to Black's specification.
Issue-ID: RELENG-2766
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: Ib688b7cf9af8a22b7d067d3417a8e03f35f0bd64
DW Talton [Tue, 10 Mar 2020 16:41:45 +0000 (09:41 -0700)]
Format lftools code with the Black code formatter
This patch auto-formats most files contained in the lftools software.
Format changes are trivial, such as replacing a single quote with
double-quotes, and rearranging the length of certain lines. Lftools
will use Black for code formatting once this patch, and the test
harness overhaul patch, are merged.
Issue-ID: RELENG-2766
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I74318e0685aa44617af4575016c84caafa68b5c5
Eric Ball [Tue, 3 Mar 2020 17:18:39 +0000 (09:18 -0800)]
Revert "Change the copy_archives method to use..."
This change is causing logs to not be copied correctly. See IT-19109,
IT-19131.
This reverts commit
3fad9329a6de8384446fb63215311d29e8aab363.
Change-Id: I7b134d2ac5f8c3b93808c1b863f3f01df0cc0a95
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Aric Gardner [Wed, 26 Feb 2020 17:29:22 +0000 (12:29 -0500)]
Remove garbage var
No memory of how this snuck in here, should be removed
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I8fa62db1ff6b481f865229643ac319413c8660f4
Andrew Grimberg [Wed, 26 Feb 2020 17:16:13 +0000 (17:16 +0000)]
Merge "Change the copy_archives method to use copy2"
Eric Ball [Sat, 22 Feb 2020 03:15:32 +0000 (19:15 -0800)]
Change the copy_archives method to use copy2
Uses copy2 with follow_symlinks set to False, which will copy
symlinks as symlinks, rather than following their path. Since glob
does not have an option to not follow symlinks, we instead just check
for file existence to ignore issues with globbing symlinked paths
that "open" fails to find.
Issue: RELENG-2576
Change-Id: Ie5a6ffba5c1cb79cff7dd9c691d9b9bd2eaeeba3
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
DW Talton [Wed, 12 Feb 2020 20:59:53 +0000 (13:59 -0700)]
Test released staging repo for closure
Add long-polling to ensure that a released staging repo is fully
closed prior to exiting.
Issue-ID: RELENG-2695
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I032cccabe28ba7f5783b2ec82d73725d228ab797
DW Talton [Fri, 10 Jan 2020 21:03:59 +0000 (14:03 -0700)]
Add nexus2 API endpoints
Change-Id: I8ba68e819fbbcda8208e88d83007880cf543a194
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Andrew Grimberg [Tue, 4 Feb 2020 21:48:14 +0000 (21:48 +0000)]
Merge "Add GitLint which replaces GitCommitBear"
DW Talton [Wed, 29 Jan 2020 21:59:59 +0000 (14:59 -0700)]
Remove extra request error checking
Remove additional requests error checking. Current lftools needs
the ability to continue despite a http session error.
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: Icad82ce16f7df7466e43be7e9035abac3fe63033
Thanh Ha [Tue, 28 Jan 2020 15:58:50 +0000 (10:58 -0500)]
Add GitLint which replaces GitCommitBear
This replaces GitCommitBear with GitLint which does similar
linting for git commit messages but using pre-commit hooks.
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I3480ad81702fa6b5015523a7274e219ba2f240d0
Aric Gardner [Thu, 23 Jan 2020 20:11:11 +0000 (15:11 -0500)]
Make output parsable
Need valid JSON in the output for automation work
-thanks DW
ISSUE-ID: RELENG-2683
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ie85adcec15577f432299846075068ed2a72783db
Aric Gardner [Tue, 21 Jan 2020 18:37:26 +0000 (13:37 -0500)]
Enable project_version_update
Typo in the rtd docs api lead this to not be implemented.
corrected endpoint and implemented in CLI
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I09aee9e06e1b54c8f6c143762c501286627098d5
DW Talton [Thu, 12 Dec 2019 00:26:02 +0000 (17:26 -0700)]
Implement the Nexus 3 REST API in lftools
Issue: RELENG-308
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I3dc1b1b7280fe6a2d69a75e2f8c7f5a9d64c5126
Aric Gardner [Thu, 9 Jan 2020 20:37:32 +0000 (15:37 -0500)]
lftools github create-team no longer requires repo
repo is now optional
ISSUE-ID: RELENG-2646
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I3cb31d17bb2fbe9e0ec7c419c99ad5cc2397d0f6
Aric Gardner [Mon, 16 Dec 2019 19:04:42 +0000 (14:04 -0500)]
Fix reviewid logic
When ammending a review we need to skip
create change not publish
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I3e2832ee3d228af12f337d63b49b66cb91357915
Aric Gardner [Thu, 12 Dec 2019 21:30:41 +0000 (21:30 +0000)]
Merge "Fix jenkins info job creation"
Aric Gardner [Thu, 12 Dec 2019 01:50:10 +0000 (01:50 +0000)]
Merge "Fix for empty INFO file creation"
Aric Gardner [Tue, 10 Dec 2019 17:36:55 +0000 (12:36 -0500)]
Fix jenkins info job creation
Fix jenkins info job creation.
Auto created info jobs need to go to the
"ci-managment" repo.
Also fix description on project creation.
ISSUE-ID: RELENG-2574
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Iab6860152c4623989997588b9a46804fc5693c2f
Aric Gardner [Tue, 10 Dec 2019 17:16:10 +0000 (12:16 -0500)]
Github cli project specific tokens
Github cli commands should first search for a
[github.organization] section.
Then fall back to a [github] section
ISSUE-ID: RELENG-2574
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I23957b43c50b4f6fda3483eec956f04afe9509e8
Aric Gardner [Thu, 5 Dec 2019 21:15:20 +0000 (16:15 -0500)]
Fix for empty INFO file creation
Do not require and lftools
[gerrit]
section.
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I118b4a945d6dd03231cdd7a2cbcf8cb82d35acee
Aric Gardner [Wed, 4 Dec 2019 16:58:36 +0000 (16:58 +0000)]
Merge "Create gerrit project via gerrit api"
Aric Gardner [Wed, 30 Oct 2019 20:58:41 +0000 (16:58 -0400)]
Create gerrit project via gerrit api
internal jenkins implementation:
gerrit.linuxfoundation.org/infra/c/ci-management/+/62158
Extend lfidapi to exit with good message and code if
group does not exist. (check if group exists functionality)
Remove pythonsix compatability for config parser
Usage: lftools gerrit [OPTIONS] COMMAND [ARGS]...
GERRIT TOOLS.
Options:
--help Show this message and exit.
Commands:
abandonchanges Abandon all OPEN changes for a gerrit
project.
addfile Add an file for review to a Project.
addgithubrights Grant Github read for a project.
addgitreview Add git review to a project.
addinfojob Add an INFO job for a new Project.
createproject Create a project via the gerrit API.
list-project-inherits-from List who a project inherits from.
list-project-permissions List Owners of a Project.
ISSUE: RELENG-2370
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I3a24751eeb2e739dee183074bbef6742df58a793
Aric Gardner [Tue, 3 Dec 2019 21:57:21 +0000 (16:57 -0500)]
Support python 3.5.x
lftools must pin xdg to 3.x to support python 3.5
needed for onap ubuntu 16.04 builders
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: If97846bbb5a52410cfe23a66fb0f25c87b6bc5b4
Bengt Thuree [Wed, 27 Nov 2019 02:07:01 +0000 (02:07 +0000)]
Merge "Add missing releasenote"
Tim Johnson [Fri, 22 Nov 2019 18:32:51 +0000 (10:32 -0800)]
Add openstack cost command
Issue: RELENG-2550
Change-Id: I9bee88cd5cb3d89f3f554470e86925d8f24dbb5a
Signed-off-by: Tim Johnson <tijohnson@linuxfoundation.org>
DW Talton [Tue, 26 Nov 2019 03:32:04 +0000 (20:32 -0700)]
Add .venv to .gitignore
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I953896c94868ad1fbb39288be5a221c5541f9d26
DW Talton [Tue, 5 Nov 2019 20:53:51 +0000 (13:53 -0700)]
Add HTTPBasicAuth support to REST client
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I6467c8aeb2b2f24800ab1912ec6b485693a3378f
Bengt Thuree [Sun, 24 Nov 2019 22:39:07 +0000 (09:39 +1100)]
Add missing releasenote
Patch Id4efd6eaf870c9ce60d670094b6916b3c7b407cd did not have releasenote
Print rule failures for unclosed repos
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Change-Id: Ibc624e6e7c52716178e8e16995d7065982539271
Aric Gardner [Fri, 22 Nov 2019 19:47:30 +0000 (19:47 +0000)]
Merge "Fix help text for github create team"
Aric Gardner [Fri, 22 Nov 2019 19:03:04 +0000 (14:03 -0500)]
Fix help text for github create team
let user know to set privacy to closed
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I9f2b583eb266cb6678a56ce6c4678f3f23e1af76
Bengt Thuree [Fri, 22 Nov 2019 00:13:06 +0000 (11:13 +1100)]
Add --exact to releasedockerhub
Enables working only on one repo, compared to wildcards or all.
-e, --exact Only exact match repo name will be used.
If used, --repo parameter can not be empty
ISSUE: RELENG-2551
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Change-Id: I774d2b206305229af76d1e78570d09849b6c836f
Andrew Grimberg [Thu, 21 Nov 2019 22:28:59 +0000 (22:28 +0000)]
Merge "Add "create role" subcommand for nexus"
Eric Ball [Thu, 21 Nov 2019 19:39:58 +0000 (19:39 +0000)]
Merge "Catch errors thrown by check_response_code"
Bengt Thuree [Wed, 13 Nov 2019 02:05:26 +0000 (13:05 +1100)]
Add extra output for lftools nexus release
Add "Missing" value to the failure condition, as well as log it.
Refactored the code for finding errors as well.
ISSUE: RELENG-2453
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Change-Id: Id4efd6eaf870c9ce60d670094b6916b3c7b407cd
Eric Ball [Wed, 20 Nov 2019 02:02:24 +0000 (18:02 -0800)]
Catch errors thrown by check_response_code
For the current methods in lfidapi, we don't need to see the stack
trace or raise the error, so we should just catch and print it, and
then exit 1.
This also includes some minor cleanup.
Issue: RELENG-2024
Change-Id: Ife7f9b82be148a208c66134ef912bc091afac041
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
DW Talton [Mon, 18 Nov 2019 22:39:03 +0000 (15:39 -0700)]
Add RTD project details update support
Add the ability to update RTD project details from the CLI.
Issue: RELENG-2543
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: Ic165376cfcd7a81acb4ac602882d9d7925827c6a
Aric Gardner [Wed, 30 Oct 2019 16:20:00 +0000 (12:20 -0400)]
Drop support for python2.7 and 3.4(EOL)
Drop support for python2.7 and python3.4(EOL)
lftools now requires python >= 3.6 This allows us to remove
remaining pins, and to move from glob2 to builtin glob
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I48ffc27376e3a7b56f0627e8eaeb9ac3dff908dc
DW Talton [Thu, 7 Nov 2019 17:52:21 +0000 (17:52 +0000)]
Merge "Rewrite DCO check in python"
DW Talton [Tue, 5 Nov 2019 16:55:01 +0000 (09:55 -0700)]
Refactor API client to support more auth types
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: Ic74499afa718f276db0007813cf77bf3c61d1ad8
Eric Ball [Wed, 22 May 2019 21:03:10 +0000 (14:03 -0700)]
Add "create role" subcommand for nexus
This enables users to create Nexus roles outside of project creation.
Add docs/commands/nexus.rst to WriteGoodLintBear ignore list, due to
issues with it failing the check based on example yaml code.
Issue: RELENG-290
Change-Id: Ie51e2bcd4d03221661b3cec79f8f62ea6d3e8c39
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
DW Talton [Wed, 25 Sep 2019 17:38:57 +0000 (10:38 -0700)]
Rewrite DCO check in python
Move to python for more control and readability
Issue: RELENG-2374
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: If3c60718579fcb8f966c36fa212d23360c637864
Aric Gardner [Thu, 24 Oct 2019 19:59:43 +0000 (15:59 -0400)]
lftools info file generator
Creates a preliminary info file to streamline workflow.
Missing fields are to be corrected in review.
Includes needed fixes for
lftools ldap for python3
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I3af2d768c356c8cd6950c527b67623e15032080e
Aric Gardner [Tue, 22 Oct 2019 14:43:24 +0000 (14:43 +0000)]
Merge "Update lftools docs to install pyhton3"
Anil Belur [Fri, 18 Oct 2019 15:45:27 +0000 (21:15 +0530)]
Update lftools docs to install pyhton3
Python2 is EOL soon, and its receommended to install pyhton3.
Issue:IT-17903
Change-Id: I8f81539846777f50ee219e656300dfa6c81109ec
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Anil Belur [Thu, 17 Oct 2019 08:50:39 +0000 (14:20 +0530)]
Update pytest to python3
Change-Id: I46c6d4ad90c0e9e6874481a93a08d6c961dc81ae
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
DW Talton [Wed, 9 Oct 2019 18:26:31 +0000 (11:26 -0700)]
Add support for RTD subprojects
Add new rtd subprojects commands to allow for subproject CRUD operations.
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I9004a1e9a38dd939d3752dfb9b2efb3ba366d113
DW Talton [Tue, 20 Aug 2019 19:24:48 +0000 (12:24 -0700)]
Refactored multithreaded nexus upload
Refactored multithreaded nexus upload to use concurrent.futures
which allows for easy information gathering. Added additional
error code trapping to various requests methods, and overall
improved message output of the entire upload process from end
to end.
ISSUE: RELENG-2264
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I0707342b2017cdfe0139738ddff5123b110ddec2
Mohammed Naser [Thu, 3 Oct 2019 14:45:35 +0000 (10:45 -0400)]
Switch volume delete to use the ID
At the moment, the current code that's used when deleting volumes
uses a function that attempts to pull by ID, and then falls back
to name.
This is potentially dangerous because it might match more than one
volume and therefore potentially delete the wrong volume. This
patch revises the code to enforce retrieving the volume based on
the ID instead which is globally unique.
The tools that leverage this inside releng-global-jjb already
provide an ID at the moment[1].
[1]: https://github.com/lfit/releng-global-jjb/blob/
d22d069117b64cee73be1165861013de9e79ab19/shell/openstack-cleanup-orphaned-volumes.sh#L18
Change-Id: I078c8c4447ac308c66678e86ae31e8947f40340e
Signed-off-by: Mohammed Naser <mnaser@vexxhost.com>
Aric Gardner [Fri, 27 Sep 2019 17:00:09 +0000 (13:00 -0400)]
Remove python27 from tox env.
Remove py27 pins
fix LFTOOLS_CONFIG_FILE to use os.path
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Id237c116cecf80c7d827413ac3d63c58960f7b45
DW Talton [Thu, 29 Aug 2019 01:29:37 +0000 (18:29 -0700)]
Add support for Read the Docs API v3
Add support for RTD APIv3, which allows for PUT and POST
operations (v2 was read-only). Support added via modular
API client with RTD module
Issue: RELENG-2239
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: Iaad90ed360eaac0de0e7a6c5a4c6fa77ab349f70
DW Talton [Thu, 22 Aug 2019 01:59:06 +0000 (18:59 -0700)]
Rework DCO with check and match commands
The DCO shell script now has two commands, 'check' and 'match'.
Check will scan a git repo for missing DCO signatures and output
the results to the console. Match checks commits that do have
DCO signatures and confirms that the signature matches the commit
author's email address
Issue: RELENG-1980
Signed-off-by: DW Talton <dtalton@contractor.linuxfoundation.org>
Change-Id: I36f57ccdc993df9923dc2d9102b7bac4c92e2739
Anil Belur [Wed, 28 Aug 2019 06:01:58 +0000 (11:31 +0530)]
Add lxml to requirements.txt to lftools
'lftools nexus release -v --server https://nexus.opendaylight.org
odlparent-xxxx' returns the following error.
Error:
bs4.FeatureNotFound: Couldn't find a tree builder with the features
you requested: xml. Do you need to install a parser library?
Change-Id: Icf92537c338870cfa2508702ecbaaa5d4cf67edb
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Thanh Ha [Tue, 27 Aug 2019 19:11:02 +0000 (15:11 -0400)]
Configure basepython to python3
More and more libraries that we are pulling in require Python 3 so
to avoid having to play the pin dependencies whack-a-mole game let's
bump or base python where possible to python3.
This also renames the pre-commit env to the standard "pre-commit" name
that we use in other projects.
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I66d1e72390ba682d0e7bd6fdbbb1a7cc3c8f929d
Bengt Thuree [Fri, 23 Aug 2019 14:19:55 +0000 (19:49 +0530)]
Do not copy sub-directory
Create the missing paths as needed when copying files.
Fixes
lftools.deploy:265: Missing path, will create it <dir>
ERROR: [Errno 17] File exists: '<dir>'
Issue: IT-17315
Change-Id: Ic1dd29f7cb51ef1ca7a8740974e5b877a1348522
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Andrew Grimberg [Wed, 14 Aug 2019 16:20:34 +0000 (16:20 +0000)]
Merge "Changes needed in lftools for github orgs"
Aric Gardner [Fri, 9 Aug 2019 21:04:45 +0000 (17:04 -0400)]
exit if errors are found in a nexus repo
nexus release now checks "{}/staging/repository/{}/activity"
Ensures that Repository is in closed state
Checks if Repository is already released (exit 0)
Check for failures, if found (exit 1)
Added
click.option('-v', '--verify-only', is_flag=True, required=False)
if -v is passed, only checks for errors, skips release.
This could be better, but I have officially spent too much time
fiddling with it.
Its a bit of a brute force, but as
curl -v -u <lfid:pass>
'https://nexus.opendaylight.org/\
service/local/staging/repository/honeycomb-vbd-1629/activity'
Actually returns two xml pages.
not being too familliar with soup I used
soup.find_all("value")
and then used a re.search
to check for needed values
Removing the pytests, they don't make sense imo.
ISSUE: RELENG-2291
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I7ae6c487b2aaf0e1df01effa7da15a1b28b624ad
Aric Gardner [Thu, 18 Jul 2019 20:42:51 +0000 (16:42 -0400)]
Changes needed in lftools for github orgs
Part of the effort for self serve user management
check-votes now works against a a githubpr
Note: You can't vote on your own pr so I add the pr owner as
a voter automatically.
infofile match-ldap-to-info now works against github groups
github submit-pr will submit a pr if mergable
github votes will count votes on a pr
Fixes a bug in github team invitation.
Teams needs to be array of :class:`github.Team.Team`
adds: --add_team and --remove_team to github update-repo
so that the replication user can be added to each new repo
in an automated way
ISSUE: RELENG-1864
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I65d30dab1d1638123fdb46bc705ec6190cb3182a
Aric Gardner [Tue, 6 Aug 2019 17:32:42 +0000 (13:32 -0400)]
Support multiple nexus sections in lftools.ini
[nexus] section is taken from -s "server" passed to release job.
The "https://" part of passed url is stripped before match.
example:
lftools nexus release -s https://nexus.example.com
Would require auth section in lftools.ini of
[nexus.example.org]
username=
password=
Issue: RELENG-2129
Issue: RELENG-1666
Change-Id: I64ab0a18a4d0e9060f1273f5acc1971fcffe497c
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Aric Gardner [Mon, 5 Aug 2019 22:35:34 +0000 (18:35 -0400)]
Remove drop of staging repos on release
The api returns that the relese is completed.
in the background java threads are still running.
Then we call drop and nexus has threads promoting and dropping
at the same time.
In this way we lose data.
Something else needs to drop, the api does not correctly
handle this.
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I3ad52502d6ac44960f697561eb82e2b6896ac781
Eric Ball [Fri, 2 Aug 2019 21:08:06 +0000 (14:08 -0700)]
Include auth credentials with release requests
Also includes a small logging improvement. During troubleshooting, I
found that it wasn't clear whether the credentials were being picked
up successfully from lftools.ini. Having some feedback during a
handled error will be much more helpful.
Change-Id: I2fa49f11ccb242296db9875e3018a8adf8b70f18
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Aric Gardner [Wed, 31 Jul 2019 19:04:08 +0000 (15:04 -0400)]
Fix commit message length
Also provide better error message on failure
to push .gitreview to a new repository
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: I758d35ba9ca6d95828d2b1a341cc44e39476d742
Andrew Grimberg [Tue, 16 Jul 2019 20:59:46 +0000 (13:59 -0700)]
Re-add display of group members
Change I76a5db63d584ab84a1ad12bfde1daffde0b619b0 removed the print out
of the members of groups from the search-members due to having moved to
the logging module and the output having been debug only output.
Since the point of search-members is to list out the members we need to
output the list.
Change-Id: Ib7ae9107a2db8da96067eb904982fafb8c7f10d7
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Andrew Grimberg [Fri, 12 Jul 2019 15:02:15 +0000 (15:02 +0000)]
Merge "Fix ldap check in gerrit create"
Aric Gardner [Wed, 10 Jul 2019 18:10:26 +0000 (14:10 -0400)]
Fix ldap check in gerrit create
ldap csv is not needed as we have lfidapi
Signed-off-by: Aric Gardner <agardner@linuxfoundation.org>
Change-Id: Ie3c29138ebd5562155afd43958da94bf95714652
Andrew Grimberg [Tue, 2 Jul 2019 22:36:42 +0000 (15:36 -0700)]
Change out lfidapi module print statements
Switch to using the logging facility so that we can more easily switch
our output streams and also not through debugging output when we don't
need it.
Also add in some useful log.info where it is appropriate.
Change-Id: I76a5db63d584ab84a1ad12bfde1daffde0b619b0
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Andrew Grimberg [Tue, 2 Jul 2019 22:23:01 +0000 (15:23 -0700)]
Fix retyping issue of function
There was a subtle bug where a function call was being overwritten by a
local variable of the same name and then a call to the function was
being attempted.
Change-Id: I7316df0c773db347ccb1e7eb79e2f40fee3a9db2
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Andrew Grimberg [Tue, 2 Jul 2019 22:55:51 +0000 (15:55 -0700)]
Pin pytest to 4.6.x
PyTest 5 was released on June 28th that now requires Python >= 3.5
so pin to the 4.6.x stream until we are ready to drop 2.7 support.
Change-Id: If25020214f778390c4e2705fbf454ce4db59791c
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Eric Ball [Mon, 1 Jul 2019 17:12:49 +0000 (17:12 +0000)]
Merge "Use requests.put for Nexus 2+3 compatibility"
Eric Ball [Thu, 20 Jun 2019 17:39:24 +0000 (10:39 -0700)]
Use requests.put for Nexus 2+3 compatibility
While both post and put methods work for deploying to Nexus 2, Nexus 3
is not compatible with post. Change to put for all deploy_nexus
uploads.
Change-Id: Ib07ba6b72b0c05afe226f7ce4c36f021535d4bba
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Andrew Grimberg [Wed, 26 Jun 2019 20:23:59 +0000 (13:23 -0700)]
Revert plugin listing print change
Change
5df955f24bc0154f7069ecfc188311052e67febc had a minor change that
slipped through the code review changing the Jenkins plugin listing from
long names to using the short names. This however, is not utilized at
all by the security listing function that was the primary portion of
this change.
This modification causes the jenkins configuration management merge job
to fail on updates to the OpenStack cloud configuration.
Change-Id: I04cfbfe5a49904875a734e223bda13c6164df664
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Anil Belur [Wed, 19 Jun 2019 00:42:08 +0000 (10:42 +1000)]
Add --force option to stack delete
Add --force option to stack delete to improve error
handling in the builder-openstack-cron job.
Issue: RELENG-2058
Change-Id: I3f9918000ce615d765c27d9334c161a1cdf2443c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Eric Ball [Thu, 13 Jun 2019 18:25:39 +0000 (11:25 -0700)]
Fix deploy_logs unicode handling for Python 2 & 3
six.text_type will always return a utf-8 encoded string, while
str.encode will return a bytes object in Python 3. io.open is
used to ensure that the file is being opened for utf-8 writing in
Python 2. decode('utf-8') is used on the requests response due to
how requests encodes unicode responses.
Change-Id: If58f8a3b9617f0be4effc58414d22f0cf09946cb
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Bengt Thuree [Thu, 13 Jun 2019 00:51:35 +0000 (10:51 +1000)]
lftools deploy archives handle dup patterns/file
If the pattern is not properly done, the resulting file list might
contain duplicated files.
This fix will remove the duplicated patterns, as well as
duplicated matched files.
This fix should fix the OSError exception in mkdir
Change-Id: Ib7ac21ecdec2fdefe84b710a90666b70d966be12
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>