Andrew Grimberg [Thu, 12 Oct 2023 19:44:16 +0000 (12:44 -0700)]
Refactor: Add annotations to lftools.cli.version
Issue: RELENG-4933
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: If44a5dac471ef07c609bc8f26d70dbb34cc07def
Andrew Grimberg [Thu, 12 Oct 2023 19:34:38 +0000 (12:34 -0700)]
CI: Bring testing of lftools.cli.version to 100%
Finish off the testing for lftools.cli.version
Issue: RELENG-4944
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: Id5ea7118f25e066c173b5a56f705d638f40689af
Andrew Grimberg [Thu, 12 Oct 2023 16:56:22 +0000 (09:56 -0700)]
Chore: Add coverage reporting
Issue: RELENG-4938
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: Ic5589cd05033ee2f6758f10a84cb235317cc55df
Andrew Grimberg [Mon, 9 Oct 2023 15:49:31 +0000 (08:49 -0700)]
Refactor: Add annoations to lftools.oauth2_helper
Issue: RELENG-4933
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I5d80203fa91196de29e77c55721150668545aeee
Andrew Grimberg [Mon, 9 Oct 2023 15:39:02 +0000 (08:39 -0700)]
Refactor: Add annotations to lftools.license
Issue: RELENG-4933
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: Id3c7e98378b4a5251f266ed78a21e4311dea5aa5
Andrew Grimberg [Mon, 9 Oct 2023 15:18:12 +0000 (08:18 -0700)]
Refactor: Add annotations to lftools.jenkins
Issue: RELENG-4933
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I94db9930f39c4421b20dc74d678c685a5b1ce0c5
Andrew Grimberg [Fri, 6 Oct 2023 21:52:27 +0000 (14:52 -0700)]
Refactor: Add annotations to base lftools
Issue: RELENG-4933
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I9d793ebe4db3ca47dafee56c9016789f8bcd3f1f
Andrew Grimberg [Fri, 6 Oct 2023 20:07:44 +0000 (13:07 -0700)]
Chore: Remove defunct requirements files
The requirements files being removed was missed in the previous change
to convert to using pyproject.toml
Issue: RELENG-4932
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I3e887e754413899d63f5b320904b275dcf3a94bc
Jessica Wagantall [Thu, 5 Oct 2023 19:29:38 +0000 (19:29 +0000)]
Merge "Refactor: Migrate to pyproject.toml"
Andrew Grimberg [Thu, 5 Oct 2023 14:06:44 +0000 (07:06 -0700)]
Refactor: Migrate to pyproject.toml
* Convert build configuration to a pure pyproject.toml project.
* Clamps reno notes generation to greater than v0.36.0
Issue: RELENG-4932
Change-Id: I10ebe156bb7712569fb06bd0c4b203074c3aed2b
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Andrew Grimberg [Wed, 4 Oct 2023 13:42:39 +0000 (06:42 -0700)]
Chore: Bump base python to 3.8
Finalize the move to python 3.8 as our base python
Issue: RELENG-4891
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I4d3cea74a171baa6bdb540f8d4a29ac8d5cb4e02
Andrew Grimberg [Tue, 3 Oct 2023 17:10:44 +0000 (10:10 -0700)]
Chore: Remove Python 3.5 - 3.7
Remove Python 3.5 - 3.7 from the defined compatible versions of lftools
Issue: RELENG-4891
Change-Id: Ia51f0dcce25b1c0394d690156d693df3f4f0e7b5
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Andrew Grimberg [Tue, 3 Oct 2023 17:00:30 +0000 (10:00 -0700)]
CI: Remove Python 3.7 from tox validation
Issue: RELENG-4891
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I4d8bd39c553f6804beada41f7fcbfaec756af75f
Andrew Grimberg [Tue, 3 Oct 2023 16:54:29 +0000 (09:54 -0700)]
CI: Remove python 3.7 validation from GHA
Issue: RELENG-4891
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I0224c6a43c677aa2fba5d7450c5cd50500f11bd5
Andrew Grimberg [Mon, 2 Oct 2023 21:51:18 +0000 (14:51 -0700)]
CI: Update python validation
* Drop python 3.6
* Set min version to 3.8
* Add python 3.9
Issue: RELENG-4891
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I6c5dfa4a7730783cdb1e2181e33503241f90109f
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Eric Ball [Wed, 27 Sep 2023 23:59:08 +0000 (16:59 -0700)]
Fix: Update github_cli call from deprecated method
The "set_repo_permission" method has been replaced by
"update_team_repository", which has the same signature, but
uses "pull" and "push" rather than "read" and "write" as the
permission strings.
Issue: RELENG-4481
Change-Id: I5167b07f85cbc063e4a4affabbd5ccf60a7dfa42
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Eric Ball [Wed, 27 Sep 2023 03:02:09 +0000 (20:02 -0700)]
Chore: Change "print" to "log" in github_cli
print statements should not be used for logging. These have been
replaced with log.info and log.error, where appropriate.
Change-Id: I919e5972dd887a0ed647b00250cbbe7a020f7d7e
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Andrew Grimberg [Fri, 21 Jul 2023 12:58:54 +0000 (05:58 -0700)]
CI: Add RTD configuration file
RTD will soon be requiring a configuration file for builds.
Additionally, they will soon stop honoring the unshallow feature flag
that they have on the repository and to get an unshallow checkout we
must do so via the config file
Issue: RELENG-4815
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I41ae572570dcfb1b82a8457a5f008cb736ca3c0b
Kevin Sandi [Wed, 19 Jul 2023 23:28:41 +0000 (17:28 -0600)]
Chore: add new lines required by pre-commit
Signed-off-by: Kevin Sandi <ksandi@contractor.linuxfoundation.org>
Change-Id: I659776622fefceb233f3a444ec38f5174a5d5054
Anil Belur [Mon, 8 May 2023 03:28:34 +0000 (13:28 +1000)]
Fix: pin urllib3<2.0.0
The latest version of module breaks compatibility with
python-jenkins.
Issue: RELENG-4715
Change-Id: If0d942696820b8c867b56735ca3036aa3aa3b37c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Andrew Grimberg [Mon, 24 Apr 2023 18:14:00 +0000 (11:14 -0700)]
Fix: Correct version label for GHA call
Issue: RELENG-4563
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: Ia041c727d4787368a0eb26302a89110c047dbb9c
Andrew Grimberg [Tue, 18 Apr 2023 16:23:21 +0000 (09:23 -0700)]
Fix: Correct bad actions call
Issue: RELENG-4563
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I5e960e68368a3471d73776b330d4bb12f489d043
Andrew Grimberg [Mon, 17 Apr 2023 17:03:46 +0000 (17:03 +0000)]
Merge "Fix: Correct bad vars definition"
Andrew Grimberg [Mon, 17 Apr 2023 16:55:34 +0000 (09:55 -0700)]
Fix: Correct missing steps designator
Issue: RELENG-4563
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I7412fafa1d91d464e367209215c6a50861f27bd4
Andrew Grimberg [Sat, 15 Apr 2023 13:40:34 +0000 (06:40 -0700)]
Fix: Correct bad vars definition
Issue: RELENG-4563
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: Ie1a1ee5e12cdae56c5108824e3e755a536afd764
Andrew Grimberg [Wed, 12 Apr 2023 19:08:07 +0000 (12:08 -0700)]
CI: Optimize GHA verify job
Issue: RELENG-4563
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I6410121b35edf484a476ae0438c86fc8fbca2d28
Andrew Grimberg [Wed, 12 Apr 2023 17:35:02 +0000 (10:35 -0700)]
CI: Add release workflow for tags
Issue: RELENG-4563
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I32708d17966a7782e65bdf66c152e4e9ac89fda2
Andrew Grimberg [Wed, 12 Apr 2023 18:35:33 +0000 (11:35 -0700)]
CI: Enable sysstat
Test for lftools require that sysstat be running as sar is called
Issue: RELENG-4563
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I0e53f40a36f66a630d8d1a4469e39b811aab951b
Andrew Grimberg [Wed, 12 Apr 2023 17:42:43 +0000 (10:42 -0700)]
CI: Remove python 3.6 checks
GHA default runners will not install python 3.6, remove it as an option.
Issue: RELENG-4563
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: Ia9b54001fb1d6b774796735be0b0a7dde7f70ee3
Andrew Grimberg [Wed, 12 Apr 2023 17:02:16 +0000 (10:02 -0700)]
CI: Add GitHub Actions verify
Issue: RELENG-4563
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I5f37fa7d4cf278d48e4b6eae38f9f477781c36e3
Andrew Grimberg [Wed, 12 Apr 2023 16:59:57 +0000 (09:59 -0700)]
CI: Add tox publish environment
Issue: RELENG-4563
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I5a7158668f3418dbba2d09d71c0d32adbc17ddfe
Andrew Grimberg [Wed, 12 Apr 2023 16:58:24 +0000 (09:58 -0700)]
CI: Add tox build,clean environments
* Add the tox build and clean environments
* Add additional gitignore to deal with build artifacts
Issue: RELENG-4563
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I26df479ac9f0a6177c345fd3a9bdd3dbb4bc9f49
Andrew Grimberg [Wed, 12 Apr 2023 16:17:04 +0000 (09:17 -0700)]
Chore: Clean-up pre-commit issues
Pre-commit issues all need to be clean before GitHub Actions can be
setup.
Issue: RELENG-4689
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I4f08a7ed8f05caaa9430b50bc83aeceb8f4fd71e
Andrew Grimberg [Wed, 12 Apr 2023 14:44:39 +0000 (07:44 -0700)]
Chore: Update pre-commit hooks
* github.com/pre-commit/pre-commit-hooks: updating v4.3.0 -> v4.4.0
* github.com/jorisroovers/gitlint: v0.17.0 -> v0.19.1
* github.com/ambv/black: 22.8.0 -> 23.3.0
* github.com/PyCQA/isort: 5.10.1 -> 5.12.0
* github.com/PyCQA/flake8: 3.9.2 -> 6.0.0
* github.com/pre-commit/mirrors-mypy: v0.971 -> v1.2.0
Issue: RELENG-4387
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I244f85858511ea039e35afddfd05c29b59f7523f
Jessica Wagantall [Tue, 28 Mar 2023 16:58:31 +0000 (09:58 -0700)]
Chore: Use C8 build node for new project creation jobs
ONAP needs to use Centos8 build nodes in new automated project
yaml file creation.
Issue: RELENG-4659
Change-Id: I2dad2d7d26dc0690b98c5c15f42075a46b31fa97
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Anil Belur [Tue, 21 Mar 2023 11:24:16 +0000 (16:54 +0530)]
Chore: rm py2.7 and add py3.8 and py3.9 support
Update the PyPI setup config to remove Python 2.x support
since it is EOL and add support for Python 3.8, 3.9.
Change-Id: Ie7595bbd4a5673bdeef123a34a7d08b73a03d0c5
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Anil Belur [Fri, 17 Mar 2023 16:59:31 +0000 (22:29 +0530)]
Fix: Remove shade deps from tox
Shade libs are deprecated therefore no longer required
to be installed. Remove whitespaces in EOL.
Issue-ID: RELENG-4644
Change-Id: I06226186cc80d5cb6da41b35f7a9fdf8309bf32d
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Bengt Thuree [Fri, 17 Mar 2023 12:25:44 +0000 (23:25 +1100)]
Fix: Replace shade library with openstacksdk
The shade library has been depreciated. Replace shade library
with openstacksdk for remaining openstack commands.
Issue-ID: RELENG-4644
Ref: https://docs.openstack.org/releasenotes/shade/stein.html
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Change-Id: Ie45f6ba7862a1bd3f471d469286b042512604c54
Eric Ball [Wed, 7 Dec 2022 16:12:38 +0000 (16:12 +0000)]
Merge "Fix: Replace v1 dockerhub api with v2 json files"
Bengt Thuree [Thu, 10 Nov 2022 05:31:57 +0000 (16:31 +1100)]
Fix: Replace v1 dockerhub api with v2 json files
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Change-Id: Iac2e23671bb421745099a1b4cbc56187ab39474f
Matthew Watkins [Mon, 7 Nov 2022 16:55:51 +0000 (16:55 +0000)]
Fix: Improve parameter handling in os/image cleanup
Issue: RELENG-4467
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>
Change-Id: If91463f83c4be698415e695a83f10f416e379dac
Eric Ball [Tue, 27 Sep 2022 21:07:17 +0000 (14:07 -0700)]
Fix: Correct file path and url parsing
These two bugs are both blocking our info-master job, so they are
being grouped together. One is an issue with using an absolute rather
than relative file path when adding a file to git. The other is
parsing the url with safe slashes, which was causing issues with
creation of repos that have slashes in the name.
Issue: RELENG-4461
Change-Id: Ia63ffc5f157bfb5285e8c6bdb2274a91a384a3f4
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Matthew Watkins [Tue, 30 Aug 2022 14:43:44 +0000 (15:43 +0100)]
Fix: Mislabelling of function description
Issue: RELENG-4391
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>
Change-Id: I65b4b14604c8b536ee5663a43b3b004f701e5660
Andrew Grimberg [Fri, 26 Aug 2022 20:19:43 +0000 (13:19 -0700)]
CI: Enable pre-commit validation via MyPy
Start enforcing proper annotation defintions by way of MyPy validation
Issue: RELENG-4388
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I2776f4f578856da4478637a33711a47a4ba2ef1a
Andrew Grimberg [Fri, 26 Aug 2022 20:15:40 +0000 (13:15 -0700)]
Refactor: Cleanup MyPy errors
While preparing to start enforcing MyPy validation by way of pre-commit
several redeclarations of click command / functions were found. After
examining the errors from MyPy these were determined to be faulty in how
they were being implemented. They were not caught before as no linter
was in place that could easily catch it.
Issue: RELENG-4388
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I35c2502cb08f5502fb30e3d14ac083c30209635c
Andrew Grimberg [Fri, 26 Aug 2022 18:07:40 +0000 (11:07 -0700)]
Chore: Update pre-commit hooks
* github.com/pre-commit/pre-commit-hooks: v4.0.1 -> v4.3.0
* github.com/jorisroovers/gitlint: v0.15.1 -> v0.17.0
* github.com/ambv/black: 21.8b0 -> 22.6.0
* github.com/PyCQA/isort: 5.9.3 -> 5.10.1
* Update tox pre-commit environment to be py37 for base python matching
our global minver, this is required to get pre-commit to properly
install under tox
Issue: RELENG-4387
Change-Id: Iffaf00c6100237141140ba5524068964c406c8cc
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Andrew Grimberg [Wed, 17 Aug 2022 17:38:03 +0000 (10:38 -0700)]
CI: Require RELENG changes in commit messages
To make sure that JSD issues are not being referenced since that is not
public data, we will now start enforcing a requirement that commits have
a RELENG issue associated with them.
Issue: RELENG-4375
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: Ibd30e0f0f1b0ab5393dfd84bf080ab6671936c85
Eric Ball [Mon, 15 Aug 2022 21:10:42 +0000 (14:10 -0700)]
Fix: Bugs in git.gerrit on nexus3 and sanity_check
Issue: RELENG-4366
Change-Id: I88f71ea3ffe746b46a5ee599db99aeee0e9bd3e8
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Eric Ball [Mon, 8 Aug 2022 14:59:17 +0000 (07:59 -0700)]
Fix: Gerrit create_project check results
The create_project method is not hitting the correct URL, but will
still often receive HTML from the server and a 200 status. We need to
check the validity of the returned data, and use the status of a bad
return as the exit code so that scripts can properly react to the
error encountered.
Issue: RELENG-4355
Change-Id: Icc3a769f0fa9aba61142bef11bd81821e1fa5356
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Andrew Grimberg [Mon, 6 Jun 2022 21:56:14 +0000 (21:56 +0000)]
Merge "Fix: Print upload size with deploy nexus stage"
Anil Belur [Wed, 25 May 2022 01:35:09 +0000 (11:35 +1000)]
Feat: Add get-private-keys to lftools.jenkins
Add ``get-private-keys`` to lftools.jenkins.
The ``get-private-keys`` command retrives the private keys and passphrases
stored in the credential store. This feature will add the
command ``get-private-keys`` in order to print these values.
Issue-ID: RELENG-4245
Change-Id: Iff0cab0b14d43bea540ca513ade08a5e330836e4
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Anil Belur [Sun, 7 Nov 2021 06:18:04 +0000 (16:18 +1000)]
Fix: Print upload size with deploy nexus stage
Print upload size in the logs after deploying Nexus
stage repositories. This is useful while comparing
the repository sizes.
Change-Id: Ie8cff4ae6e026177cd43c309d4f3a36bab9c7beb
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Eric Ball [Fri, 15 Apr 2022 13:07:43 +0000 (06:07 -0700)]
Fix: INFO job template and sanity_check call
The INFO job template was improperly formatted. The sanity_check call
is to a class method, and requires and positional argument for "self"
(this does not need to contain anything; it is just filler in this
case).
Change-Id: I1dcbc456bf31e907821ab4c5aee7c324dbd70722
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Bengt Thuree [Tue, 12 Apr 2022 12:55:41 +0000 (22:55 +1000)]
Fix: Do not fail tox if gerrit.acumos.org is down
Issue-Id: RELENG-4181
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Change-Id: I3a1228e77fd7fdc0ae723c26cff420da8f299fa3
Bengt Thuree [Thu, 24 Mar 2022 08:54:50 +0000 (19:54 +1100)]
Fix: Force delay (take 2) between docker get calls
Docker has introduced a 429 error return code
if there site is called to quickly.
Retry after 60 seconds after every 429 return code,
up to 19 times. Then fail.
Issue-ID: RELENG-3711
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Change-Id: I51d23fa9a2da1dc7cc30afceae4f35942bb42182
Andrew Grimberg [Wed, 23 Mar 2022 20:16:19 +0000 (20:16 +0000)]
Merge "Fix: Force delay between docker get calls"
Bengt Thuree [Wed, 23 Mar 2022 04:09:06 +0000 (15:09 +1100)]
Fix: Force delay between docker get calls
Docker has introduced a 429 error return code
if there site is called to quickly.
Delay 0.5 seconds after every pull, to avoid to fast requests.
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>
Issue-ID: RELENG-3711
Change-Id: Ic5ac8da65903c0445092d74168ff9fd1dc0068bd
Eric Ball [Fri, 4 Mar 2022 20:21:19 +0000 (12:21 -0800)]
Feat: New method add_maven_config for JCasC config
This method allows for adding settings files for new projects via
JCasC, rather than the old method of using a Groovy script to manually
add the file to Jenkins.
This also removes the __del__ function. The function was not entirely
necessary, and was causing issues with unit testing. All work is done
in /tmp, so these files will be automatically cleaned by the system.
Issue: RELENG-3893
Change-Id: I03511b9c3e1c9f9c9f79054bd6f8991cd274e8f5
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Eric Ball [Wed, 2 Feb 2022 23:33:03 +0000 (15:33 -0800)]
Feat: Move git functions to use python git lib
Issue: RELENG-4052
Change-Id: Iafaa078b24838135dd67ec0ac53b1d83de13ae8b
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Eric Ball [Tue, 22 Feb 2022 18:58:04 +0000 (10:58 -0800)]
Test: Speed up Nexus tests
By patching the sleep function, we can remove about 25 seconds of
pauses during unit testing of release_staging_repos method.
Issue: RELENG-4052
Change-Id: I8692c1183d70873348616435074b9b69193dda41
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Eric Ball [Thu, 3 Feb 2022 00:13:31 +0000 (16:13 -0800)]
Feat: Add get-secrets to lftools.jenkins
Get-credentials only produces username/password credentials, which
does not include "secret text" stores. This feature will add the
command "get-secrets" in order to print these as well.
Change-Id: Iac1911b486be1dc125a42b23595431c7065adfce
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Anil Belur [Wed, 20 Oct 2021 03:32:44 +0000 (13:32 +1000)]
Fix: Create temp file in empty dirs for S3 log dir
Since S3 buckets does not have an underlying file system to view empty
directories, an temp file has to be created when referencing the directory.
This allows empty folders to be viewed through cloudfront, without
returning a 404 error.
Create temp file in the format: "_%d%m%Y_%H%M%S", would
help with future clean or debugging.
Issue-ID: RELENG-4016
Change-Id: I0a635cd32da860deaada0ac0cc15bf1aa5fc2d4a
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Thanh Ha [Sat, 31 Jul 2021 11:47:00 +0000 (07:47 -0400)]
Chore: Configure Flake8 linter
This adds Flake8 to pre-commit and removes the unused config in tox.ini.
Not sure why Flake8 was disabled but config in tox.ini suggests that
it was there at one point.
Change-Id: I93c9a6e4562f957f75f28a3842ae29eb8df67fb9
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Thanh Ha [Sat, 9 Oct 2021 14:39:41 +0000 (10:39 -0400)]
Refactor: Resolve remaining flake8 violations
- E304 blank lines found after function decorator
- E402 module level import not at top of file
- E741 ambiguous variable name 'l'
- F522 '...'.format(...) has unused named argument(s): end
- W601 .has_key() is deprecated, use 'in'
Change-Id: I2607e7bb195f6175a7b331e89870fa23e225e31c
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Thanh Ha [Sat, 9 Oct 2021 14:33:11 +0000 (10:33 -0400)]
Refactor: Resolve F811 and E712 and E501 in tests
- E501 line too long
- E712 comparison to True should be 'if cond is True:' or 'if cond:'
- E712 comparison to False should be 'if cond is False:' or 'if not cond:'
- F811 redefinition of unused 'test_log_and_exit' from line 26
Change-Id: I896e994b26d62c7bdd85ca61ebf1b9d2df9e8b1e
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Thanh Ha [Tue, 5 Oct 2021 23:25:12 +0000 (19:25 -0400)]
Refactor: Resolve Flake8 F401 and F403 and F405
It's bad practice to use * imports (F405) since the methods or
variables being imported could be undefined. Explicitly importing
makes this more clear. Also F403 violation makes it difficult to
detect undefined names.
Additionally F401 violation to allow an indirect module import makes
things harder to debug. It's better to be explicit.
https://www.flake8rules.com/rules/F401.html
https://www.flake8rules.com/rules/F403.html
https://www.flake8rules.com/rules/F405.html
Change-Id: Ib938f865d3ac6456216a119c830d3b013932bf88
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Andrew Grimberg [Tue, 5 Oct 2021 20:09:08 +0000 (20:09 +0000)]
Merge "Refactor: Resolve flake8 E722 bare except"
Thanh Ha [Tue, 5 Oct 2021 16:17:02 +0000 (12:17 -0400)]
Refactor: Resolve flake8 E722 bare except
It is considered bad practice to use a bare except as this kind
of except statement can have untended consequences such as disabling
ctrl+c interrupts. At the vary least we should use Exception however
a more specific one would be better. This PR uses the general Exception
to resolve the linter which we can add more specific ones later when
we can appropriately unit test the code.
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I9afcff49f34326d279a63ce972d12f9aa72f18e2
Thanh Ha [Tue, 5 Oct 2021 16:27:09 +0000 (12:27 -0400)]
Refactor: Resolve W605 Invalid escape sequence
W605 is deprecated and will become a syntax error in Python 3.8
(https://www.flake8rules.com/rules/W605.html).
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: Ic2a51f089ef9d6fc45a3db5e96a8f5196d59f02c
Bengt Thuree [Thu, 2 Sep 2021 05:25:52 +0000 (05:25 +0000)]
Merge "Fix: Incorrect indent skips calling deploy_s3()"
Anil Belur [Thu, 2 Sep 2021 02:55:57 +0000 (12:55 +1000)]
Fix: Incorrect indent skips calling deploy_s3()
When Unix style glob patterns are passed through the CLI option
the deploy_s3() function skips the step to push logs to the S3 bucket.
Issue-Id: RELENG-3866
Change-Id: I58d6ef8dab41213fdbc431294ddd21854adc60dc
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Andrew Grimberg [Wed, 1 Sep 2021 18:07:33 +0000 (11:07 -0700)]
Chore: Upgrade pre-commit hooks
* github.com/ambv/black: updating 21.7b0 -> 21.8b0
Change-Id: I4a2374c5e589c4adbfdc9e8e82cb7fc099acaa8c
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
pmikus [Mon, 23 Aug 2021 10:37:58 +0000 (10:37 +0000)]
Fix: typo in deploy.py
Typo error in xml mimetype deploy to S3 storage. Thais patch remove the
extra sign to correctly map mimetype to content-type.
Signed-off-by: pmikus <peter.mikus@protonmail.ch>
Change-Id: Ia88ac75f483b31312ae72cc104021ab9efda6455
Andrew Grimberg [Mon, 16 Aug 2021 22:19:38 +0000 (22:19 +0000)]
Merge "Fix: Refactor image clean"
Anil Belur [Fri, 13 Aug 2021 13:20:08 +0000 (23:20 +1000)]
Fix: Refactor image clean
Refactor image cleanup code to handle attribute types and
multi-clouds config correctly.
Error:
AttributeError: 'Connection' object has no attribute 'cloud_config'
Issue-ID: RELENG-3869
Change-Id: If6dd04507f6eb94786f6589299ce31f62a009b84
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Eric Ball [Sat, 14 Aug 2021 00:54:06 +0000 (00:54 +0000)]
Merge "Refactor: Fix flake8 F841"
Thanh Ha [Fri, 13 Aug 2021 01:17:36 +0000 (21:17 -0400)]
Refactor: Fix flake8 F841
Resolves "F841 local variable ____ is assigned to but never used".
Change-Id: I4ab246dd96c96f2fdf33bae1d7d6b2d2c6fe6190
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Eric Ball [Fri, 13 Aug 2021 01:40:17 +0000 (18:40 -0700)]
Fix: Move text/html mimetype check to after None
When checking mimetype during _upload_to_s3, the newly-added
text/html type was checking before it is checked for None, which is a
potential TypeError.
By moving it after the variable is checked for None, the feature is
safe without requiring any further modifications.
Change-Id: Ia945893adb8973608cb9527b4ea811a5610d6aba
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
pmikus [Tue, 10 Aug 2021 11:31:16 +0000 (11:31 +0000)]
Fix: Add text/html mimetype to handle s3 logs
Currently html files in S3 bucket are not rendered properly
via Cloudfront as they are sent with text/plain content-type.
This patch is setting up content-tyepe for html for text/html.
Signed-off-by: pmikus <peter.mikus@protonmail.ch>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Change-Id: Id8a26f77446890320b212522236b68552d73c425
Andrew Grimberg [Tue, 3 Aug 2021 14:49:36 +0000 (14:49 +0000)]
Merge "Test: Improve unit test for nexus"
Eric Ball [Fri, 30 Jul 2021 22:45:05 +0000 (15:45 -0700)]
Test: Improve unit test for nexus
When waiting for a repo to complete releasing, a more realistic
"activity" return is to have the "release" status in place, but
without the "stopped" property (which is not added until the repo is
finished releasing). This improved unit test would have caught the
bug addressed by commit
e083184d.
Change-Id: I0827cdaad07a8fb94d0f266dc8e53fd69c4aab2d
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Andrew Grimberg [Sat, 31 Jul 2021 14:19:10 +0000 (14:19 +0000)]
Merge "Refactor: Fix F401 module imported but unused"
Andrew Grimberg [Sat, 31 Jul 2021 14:18:04 +0000 (14:18 +0000)]
Merge "Chore: Remove botocore from requirements.txt"
Andrew Grimberg [Sat, 31 Jul 2021 14:17:48 +0000 (14:17 +0000)]
Merge "Refactor: Fix E713 form should be "not in""
Thanh Ha [Sat, 31 Jul 2021 12:01:18 +0000 (08:01 -0400)]
Refactor: Fix F401 module imported but unused
A module has been imported but is not used anywhere in the file.
The module should either be used or the import should be removed.
Ref: https://www.flake8rules.com/rules/F401.html
Change-Id: I3e73260628bdbeccab2a7dd78215aa83a2ee71ff
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Thanh Ha [Sat, 31 Jul 2021 11:44:23 +0000 (07:44 -0400)]
Chore: Remove botocore from requirements.txt
This is a transitive dependency of boto3, no need to explicitly
require it unless we are trying to pin the version.
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I0ce2d72082707a969d8b4841957f6dd761c7c6a0
Thanh Ha [Sat, 31 Jul 2021 03:53:40 +0000 (23:53 -0400)]
Refactor: Fix E713 form should be "not in"
Tests for membership should use the form x not in the_list
rather than not x in the_list. The former example is simply
more readable.
Ref: https://www.flake8rules.com/rules/E713.html
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I3ab1811c2a6b7e6135397e694b4ec4a1ab980c93
Thanh Ha [Fri, 30 Jul 2021 22:36:22 +0000 (18:36 -0400)]
Refactor: Fix Flake8 E711
Comparisons to the singleton objects, like True, False, and None,
should be done with identity, not equality. Use “is” or “is not”
Ref: https://www.flake8rules.com/rules/E711.html
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I33e80228341b93bdeb6cae1b587b8b19275c589d
Thanh Ha [Fri, 30 Jul 2021 22:31:07 +0000 (18:31 -0400)]
Chore: Enable black for lftools
Not sure why lftools was excluded but it effectively disabled black
in all of lftools defeating the purpose of having it in pre-commit
in the first place.
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I60b3832cb180574e114b2dca78c0376a14c6d66e
Thanh Ha [Fri, 30 Jul 2021 18:26:47 +0000 (14:26 -0400)]
Chore: Add isort to Python linters
ISort is an automatic import sorter for Python. More details at
the ISort GitHub page (https://github.com/PyCQA/isort).
Signed-off-by: Thanh Ha <zxiiro@gmail.com>
Change-Id: I2eaff523fab0966ea7196037669a1c603c68cd3e
Eric Ball [Wed, 28 Jul 2021 13:33:12 +0000 (06:33 -0700)]
Fix: Add check for "stopped" before reading text
nexus.release_staging_repo has an error condition if the staging repo
is not released when checked, as a non-existent "stopped" field will
return a NoneType, causing an error when .text is read. We avoid this
by checking for the existence of the "stopped" field before attempting
to read the text.
Issue: RELENG-3813
Change-Id: I0fae854a3ee52118806f70ea8b6faacc5fda430b
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Dave Wallace [Thu, 22 Jul 2021 18:54:11 +0000 (14:54 -0400)]
Fix: Don't exit on error while collecting data for _sys-info.log
Change-Id: Ic3ed23b727b4663f01e52205641b23505518d3b9
Signed-off-by: Dave Wallace <dwallacelf@gmail.com>
Eric Ball [Wed, 21 Jul 2021 23:13:45 +0000 (16:13 -0700)]
Fix: Nexus release logic and naming, add unit test
nexus.release_staging_repos was failing with an error due to an
undeclared variable. Several other issues were identified and fixed
(see release note for details), and a unit test has been added to
help prevent similar issues in the future.
Issue: RELENG-3804
Change-Id: I8b7e4adf4561541acffa1dc81dbeba35d7783e51
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Eric Ball [Wed, 14 Jul 2021 21:21:39 +0000 (14:21 -0700)]
Chore: Unpin requirements
Having all requirements pinned was causing problems with dependency
resolution in pip. This worked fairly well in older versions, but
newer versions of pip are more strict in how they handle dependencies
of multiple packages. This was causing v0.31.1, the last version with
unpinned requirements, to be installed in cases where lftools was
installed alongside packages with conflicting requirements.
Issue: RELENG-3792
Change-Id: Ie8fb0dac45da5d3fba803ef66e216341516bc3ab
Signed-off-by: Eric Ball <eball@linuxfoundation.org>
Eric Ball [Fri, 9 Jul 2021 23:39:13 +0000 (23:39 +0000)]
Merge "Refactor: Update setup.cfg due to deprecations"
Andrew Grimberg [Tue, 29 Jun 2021 20:53:08 +0000 (13:53 -0700)]
Refactor: Update setup.cfg due to deprecations
While working on a few other updates it was noticed that tox was giving
deprecations warnings related to values in setup.cfg. This therefore
updates the keys with the new non-deprecated values.
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Change-Id: I04c526c5799a6a5da1ea6aae8a56ab4fc07d7465
Andrew Grimberg [Mon, 28 Jun 2021 17:32:58 +0000 (10:32 -0700)]
CI: Properly run gitlint in CI
When running pre-commit in a CI system the COMMIT_EDITMSG does not
(normally) get created as that is an artifact of editing the commit
message. If the file doesn't exist then gitlint will skip which makes it
possible for pre-commit checks that should fail, to pass.
Since we want tox to run in a consistent manner both locally and in CI
we need to play around a little with how we are checking the commit
message.
Change-Id: I25c1815ea3195e47a67b529a06497ad3594b2c84
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Andrew Grimberg [Mon, 28 Jun 2021 17:30:05 +0000 (10:30 -0700)]
CI: Configure gitlint for Conventional Commits
Turn on Conventional Commit message subject validation. This will affect
CI specifically, but developers can avoid having CI fail by assuring
that they have installed pre-commit and that they have also run
pre-commit install --hook-type commit-msg
The above is needed because pre-commit does not install commit-msg hooks
by default
Change-Id: Ifa301f8f6b8fa5c5838934b8ffcfa47e746cd95d
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Andrew Grimberg [Mon, 28 Jun 2021 17:28:24 +0000 (10:28 -0700)]
Chore: Upgrade pre-commit hooks
This change also includes style updates enforced by black that have to
come in _with_ the upgrade.
Change-Id: I68eebed11c5aca19d530104ae1164c42371d4851
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>
Chandra Dasari [Fri, 14 May 2021 13:54:55 +0000 (09:54 -0400)]
Refactor: Migrate shade library to openstacksdk
Replacing shade library with openstacksdk for openstack image commands.
Signed-off-by: Chandra Dasari <cdasari@contractor.linuxfoundation.org>
Change-Id: I23f901de37d502674edc617784ad3095dbc38288
Issue: RELENG-3392