CI: Switch to using reuse for license checking

The tox based license checker is failing due to some issues related to
distutils. As we do not really maintain the validator and there is a
better upstream system we're going to switch to it.

This change brings in all the needed modifications to switch tools and
also take care of the license and copyright information without having
to touch all of the files in the repository.

Issue: RELENG-5356
Change-Id: Ibaecb20f922adf5e7a050be9f03f945076dd2dfe
Ref: https://github.com/fsfe/reuse-tool
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Chore: Upgrade Jenkins-job-builder to 6.3.0

JJB 6.3.0 is released includes bug fixes and improvements on job views.

Ref:
https://groups.google.com/g/jenkins-job-builder/c/d4PPKsGGab0

Issue: RELENG-5285
Change-Id: I0ccd07e80e0394ab9e3a22e7e52794cc5e63cab3
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Chore: Upgrade Jenkins-job-builder to 6.2.0

JJB 6.2.0 is released includes bug fixes and improvements on job views.

Ref:
https://groups.google.com/g/jenkins-job-builder/c/CXXF5U-05BI/m/cRu0nU-BAgAJ

Issue: RELENG-5285
Change-Id: Id58222e29c587fb787b778c00816f455b65bff3a
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Chore: Update gerrit-review-action to latest v0.8

Issue: RELENG-5251
Change-Id: I20d0d1679bced982b84a0e0e6419d45478c4d7ac
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Allow deploy job for unmerged changes

This will allow the deploy job to be used
as part of verification process before merge.

Issue: RELENG-5184
Change-Id: Ie7e06f7301e4f90ff8eb0b1c3dae6eb94d57dbb4
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

Merge "CI: Remove mailing list notification"

CI: Remove mailing list notification

Issue: RELENG-5258
Change-Id: Ib98718f1b1bb9c13916024ba9ee8107e8d74c30b
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Fix: Bump the default version of jjb to 6.0.0

I highly doubt any project still works
with the old 2.8.0 value.

Issue: RELENG-5184
Change-Id: I1bb33a87349e5e7cdad024c84eb45d67b3e6a6a5
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

Merge "Fix: Add comment as parameter to deploy job"

Fix: Add comment as parameter to deploy job

While testing deploy jobs on Sandbox is not impossible,
currently it needs manual editing of job configuration.
With this, the comment text can be added
when starting job manually in Sandbox.

Issue: RELENG-5184
Change-Id: I38528702b6da3dd359d058521ee39469b17f4265
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

Fix: Update submodules in jjb-deploy-job

The jjb-deploy-job can be used to test upgrades
in ci-management related repositories.
Those contain submodules, e.g. global-jjb.

The parent way to checkout the change under test
does not update the submodules,
so the job uses a combination of unmerged main repo
with current master HEAD submodules.
That is unhelpful and dangerous.

This change adds a command
to update the submodules after checkout,
so they match the commit under test.

In the Issue field I am pointing
to otherwise unrelated issue
that prompted me to start verifying
submodule upgrades in Sandbox.
That is because I cannot find a way
to open a new ticket in:
https://jira.linuxfoundation.org/projects/RELENG/issues

Issue: RELENG-5184
Change-Id: I407ccf86e404bd01675439bb4df8c85a804267bc
Signed-off-by: Vratko Polak <vrpolak@cisco.com>

Chore: Update gerrit-review-action to v0.7

- Pulls in change to skip voting locally

Issue: RELENG-5251
Change-Id: I5ab2930ed4bb0cbe41d2fca4be13c41f704381a2
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Add vex v3 starter flavors for create scripts

Update the create script to include V3 starter flavors on the cloud
provider. Newer V3 starter flavors run on newer hardware and could
potentially reduce costs by half and ideal for running jobs or tests
as compared to standard flavors that are not required for basic jobs.

Issue: RELENG-5237
Change-Id: I8b8c59b82a995298a0c892d134b5df68a77ed0ef
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Workaround for yaml multiline string

Fix JaCoCo exclude patterns format. JJB 6.x does not process
the job config when double quotes are used and returns the
following error.

Error:
requests.exceptions.HTTPError: 500 Server Error: Server Error for url:
https://jenkins.opendaylight.org/releng/job/distribution-\
merge-managed-argon/config.xml

Using YAML multiline '>-' introduces spaces between lines that causes
JJB updates to fail when the job exists on Jenkins. Therefore as
workaround use double-quoted string with newline escape.

Using '>-' introduces space that causes consecutive JJB updates
to fail with the same 500 internal error.

Expected pattern:
"**/gen/**,**/generated-sources/**,**/yang-gen**,**/pax/**"

Generated by JJB with YAML multiline '>-':
"**/gen/**, **/generated-sources/**, **/yang-gen**, **/pax/**"

Therefore use the workaround of double quoted string with
newline escape.

Issue: RELENG-5118
Change-Id: Id26a78280768dbc43a74d3d393907da946c15dd4
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Chore: Update GitHub setup-python action from v4 -> v5

Issue: RELENG-5176
Change-Id: Id9b6ad45a2fd0d42816075964552c5f0a59d75e9
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>

Fix: Address various shellcheck linting errors in scripts

Issue: RELENG-5183
Change-Id: I068c27b215fd487985d6204af05fe6619c9d8f15
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>

Fix: Remove stale opensdk version pinning

Issue: RELENG-5176
Change-Id: Id6dd1fdac68e3105f938a225c67ca0bed6813f89
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>

Chore: Comment recent tox.ini changes

Issue: RELENG-5179
Change-Id: Ic7561b5c339dadb2d0b71c64dbd63b69fbf84188
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>

Chore: Update pre-commit dependencies

pre-commit-hooks v4.2.0 -> v4.5.0
mirrors-prettier v2.6.2 -> v4.0.0-alpha.8
gitlint] v0.17.0 -> v0.19.1
yamllint.git v1.26.3 -> v1.35.1
pre-commit-hooks 2.1.6 -> 3.0.0

Issue: RELENG-5144
Change-Id: Ibb0505a52d8baff7239e49caef410ff6fad35cb4
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

CI: Update workflow to pull latest gerrit checkout

Issue: RELENG-5144
Change-Id: I8603b94b072afba73f2de16e982fbd2a37b28322
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Rename view name to All

Rename view name 'All'. JJB 6x has introduced a
regression with view name that causing view updates
to fail. The workaround it to use 'All'.

Error:
Unrecognized view-type: All (supported types are: all,
delivery_pipeline, list, nested, pipeline, sectioned)

Update the JJB version in tox for comparing xml output.

Issue: RELENG-5144
Change-Id: I409d2ef2c14b5c5e70c8edb13a802b3c4aae410c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: JaCoCo exclude patterns format

Fix JaCoCo exclude patterns format. JJB 6.x does
not process the job config when double quotes are used and returns
the following error.

Error:
requests.exceptions.HTTPError: 500 Server Error: Server Error for url:
https://jenkins.opendaylight.org/releng/job/distribution-
merge-managed-argon/config.xml

Issue: RELENG-5118
Change-Id: Id8b2c02c7feb1e5f2ed443eb59dd61e040a31d1c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Workaround missing setuptools dependency in tox.ini

JJB6 uses stevedore, which has an undeclared dependency on setuptools

Issue: RELENG-5179
Change-Id: Ie59f28b75e86e2e80421840a5ebd6eb982f165f3
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>

Fix: Rename tox job and remove pre-commit

This commit 0ff7cbacb76c80c8e2222 previously removed the tox job
however did not remove the depenencies.

This causes the verify workflow to fail with the following message:
Invalid workflow file: .github/workflows/gerrit-verify.yaml#L85
The workflow is not valid.
.github/workflows/gerrit-verify.yaml (Line: 85, Col: 22):
Job 'vote' depends on unknown job 'pre-commit'.
.github/workflows/gerrit-verify.yaml (Line: 85, Col: 34):
Job 'vote' depends on unknown job 'tox'.

Issue: RELENG-5138
Change-Id: I079fa354d70046df1083c08166f33b9ce2073d15
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Merge "CI: Use the tox-verify workflow to test all envs"

CI: Use the tox-verify workflow to test all envs

The previous tox CI config was not running "docs", which is not part
of the envlist. We will now run all necessary envs.

Due to how the tox-verify workflow functions, pre-commit no longer
needs to be separated out. All envs will be run in parallel.

Issue: RELENG-5138
Change-Id: I2c5e732b4fd0a28c800d7a3819fb8fb32a29e5fd
Signed-off-by: Eric Ball <eball@linuxfoundation.org>

Fix: Update Sphinx to fix RTD build

The old version pinned in requirements.txt was causing build failures
on RTD. This has been updated, along with a few small fixes that
showed up once the build was working again.

Issue: RELENG-5138
Change-Id: Icaa18d3a39540148b328654bddb9d7fb98ecbd31
Signed-off-by: Eric Ball <eball@linuxfoundation.org>

Merge "Chore: Update to gerrit-review-action@v0.5"

Merge "Docs: Quick links to guides"

Chore: Update to gerrit-review-action@v0.5

The gerrit-review-action@v0.3 shows the deprecation warnings that is
fixed in v0.5.

Warning: Node.js 16 actions are deprecated. Please update the
following actions to use Node.js 20: shimataro/ssh-key-action@v2.5.1.
For more information see:
Ref: https://github.blog/changelog/2023-09-22-github-actions
-transitioning-from-node-16-to-node-20/

Issue: RELENG-5136
Change-Id: I0b931a12e73eb75bdfbf040d57b056764d834f56
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Feat!: Upgrade Jenkins-job-builder to 6x

Jenkins Job Builder 6.0.0 is released and breaks backward
compatibility with 5.1.0.

The breaking changes with previous 5.1.0 release:

- Jobs are now expanded the same way as job templates.
- Macros without parameters are now expanded the same way as macros
  with parameters.
- Tags !include-raw: and !include-raw-escape: should now be used the
  same way in jobs and macros without parameters as they are used in
  job templates and macros with parameters.

Note: This breaks backward compatibility with older version of JJB
therefore care must be taken when upgrading the ci-man repositories.

Ref:
https://storyboard.openstack.org/#!/story/2010963
https://storyboard.openstack.org/#!/story/2010588

Issue: RELENG-5118
Change-Id: Ib9acd0b9e7c264ac3531af858431e738397833b5
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Add Java update alternatives step

This is to allow users to use a different Java
version for their Sonarqube builds.

Issue: RELENG-5099
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I7a0915bb41ec18e9eeed57f5049e72cacf4333f6

Fix: Pin JJB to 5.1.0

JJB 6.x is released makes it incompatible with 5.x.

Issue: RELENG-5118
Change-Id: I1736c3298421dc847bb8e4600dee43c2cd08ba1b
Ref: https://groups.google.com/g/jenkins-job-builder/c/miYGnWgYZYM
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Docs: Quick links to guides

Issue: RELENG-4732
Change-Id: I11c14cd4fa2ba9b9f87a05d8c1f82d427cf2208d
Signed-off-by: Ayush <ayushayush.sadwal@gmail.com>
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Update lf-infra-sonar macro to use JDK version

Issue: RELENG-5099
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I90875727fd93767496f72a26d3acdb3055dc7e8e

Feat: Allow SBOM to run in docker-maven jobs

Issue: RELENG-5062
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: Ifd685f1c6266397a0086093fcb3680a02725d999

Merge "Fix: Add HCL file extentions to regex"

Feat: Use Sigstore Cosign to sign docker images and push signature

In order to enable, the project needs to create their keypair and
credentials in Jenkins for cosign-password (keypair password) and
cosign-private-key.

Issue: RELENG-5014
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: Ie3e73cb10445b5139417c62fb534ba883a0ad499

Fix: Add HCL file extentions to regex

Add regex with HCL file extensions to trigger merge jobs when
HCL file format is found.

Issue: RELENG-5048
Change-Id: I9396cc252e70939f97b9fddbffb68f4f82051b26
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Make sure reno has enough commits

Issue: RELENG-4975
Change-Id: I8f25c8c91278508bf086760ef094f79b9c7f359b
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Fix: Update release jobs triggers

Currently, all active branches verify jobs are triggering
causing verify conflicts when projects are releasing more
than one branch simultaniously.
Update triggers to only trigger the branch in question.

Issue: RELENG-4979
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: Ifff3470671df4b6d05d78575d809c1befa7056de

Fix: Move the git fetch depth

It is not possible to specify a depth larger than 1 with the tag
checkout that is performed. Move the depth increase to where it's
actually needed.

Issue: RELENG-4975
Change-Id: I8a33ccb244bb5cff784934fe7bd75e690f6ff222
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

CI: Generate GitHub releases

* Increase the checkout depth from 1 to 30 to handle reno needing to be
  able to find all relevant tags (current and current -1) and also pull
  the tags
* Setup Python to run reno
* Generate a reno report for the given tag and then modify the report to
  strip out all the extra garbage that is added by the report process to
  get it into a valid markdown file and not RST file
* Use `gh` cli tool to generate the release notes using the
  auto-changelog generation plus the reno release notes

Issue: RELENG-4975
Change-Id: Ie05579dfd757e7d3be7169da987bf67229f4f818
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Fix: Handle error in job-cost.sh without failing

We have seen builds get marked as "unstable" due to transient network
failures while retrieving instance type info. This is not an
acceptable criteria for marking a build unstable, so this change will
handle such an error more cleanly.

Issue: RELENG-4970
Change-Id: Ia146e2771df638e8410fb187d730d4faadf132c2
Signed-off-by: Eric Ball <eball@linuxfoundation.org>

Merge "Chore: Update Sonar OpenJDK version"

Chore: Update Sonar OpenJDK version

Update to OpenJDK17 for Sonar jobs
Also revert from JDK13->JDK11 where previous scope was incorrect

Issue: RELENG-4962
Change-Id: I300c9ccbc47a5fe03354c6a088fd75dff8c55519
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>

Feat: Define naming of K8S clusters to preserve

Issue: RELENG-4963
Change-Id: If7e283b6b2d8bb55a8f4a08b6406b85530f7eedd
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>

Chore: Update Sonar OpenJDK version

Issue: RELENG-4962
Change-Id: I3aa37e4dc09d9efbbc99f5f12e61c29ce86d24d6
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>

Fix: Pin setuptools in jjb-deploy-job.sh

This is in order to fix the OpenSSL issue:
urllib3 v2.0 only supports OpenSSL 1.1.1+

Issue: RELENG-4952
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I389c29fab9f892950f424d92eff65b8900d3e6e8

Merge "CI!: Create variable for jacoco exclusion pattern"

Merge "Feat: Upgrade Jenkins-job-builder to 5.0.4"

CI!: Create variable for jacoco exclusion pattern

This changes the signature of lf-jacoco-report macro. All
implementations within global-jjb have been updated, but any uses of
this macro outside of global-jjb will need to be updated.

Issue: RELENG-4856
Change-Id: I3db8ab048fb9d79d4503f455e967a4a26a4a0411
Signed-off-by: Eric Ball <eball@linuxfoundation.org>

Fix: gradle jobs triggers and scm

Issue: RELENG-4763
Signed-off-by: Kevin Sandi <ksandi@contractor.linuxfoundation.org>
Change-Id: I773d891518d4de8bf6fc0d5148664ba03958c3e1

Feat: Upgrade Jenkins-job-builder to 5.0.4

Supports urllib3 to newer DEFAULT_TIMEOUT, therefore unpin urllib3<2.0.0
Ref: https://review.opendev.org/c/jjb/python-jenkins/+/882757

Fixes Jenkins version 2.387.1 or earlier can return 'all' as view name when
requested is 'All'.

Issue: RELENG-4853
Change-Id: Idcfae769f5e5c4268380f531ad86e57aab4fbf03
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Rename view name to lower case

Jenkins version 2.387.1 or earlier can return 'all'
as view name when requested is 'All'.
Add workaround and rename the view name to lower case.

Issue: RELENG-4849
Ref: https://review.opendev.org/c/jjb/python-jenkins/+/888285
Change-Id: I374de78a497b4ba38b495a547fcdbbfd176b068e
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Feat: Add gradle publish job

Issue: RELENG-4763
Change-Id: Ifca59da7e6321815ad1301baa28756461f53ce30
Signed-off-by: Fiete Ostkamp <Fiete.Ostkamp@telekom.de>

CI!: Remove unused WhiteSource templates

Issue: RELENG-4817
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I3bae425c75dfe1df1694f323d76f4559c3252221

Docs: Remove pip install method from RTD config

RTD seems to have a problem with the install method and path with how
they were defined. Remove them as they are the default anyway

Issue: RELENG-4816
Change-Id: I5e8a2a925ee81b5284fc44cf4ec72ef1fa8761d6
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Docs: Add RTD config file

RTD will start requiring configuration files on September 25, 2023.
This also makes sure that we properly unshallow the clone as that is a
feature flag that is going away as well.

Issue: RELENG-4816
Change-Id: Ia3015cd6f519158ac03ddcb3453d3ac4fb233d51
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Feat: Switch gerrit-gradle-build template to the gradle builder

Issue: RELENG-4763
Change-Id: Id31d9e23c8df1470c41eab58a91a005b5bd9a4db
Signed-off-by: Fiete Ostkamp <Fiete.Ostkamp@telekom.de>

Merge "Fix: Run docker login for gradle build jobs"

Fix: Add {stream} in release verify and merge jobs

Issue: RELENG-4782
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: Ib319f90e5e1b64b837588f46314766e55431e4fb

Fix: Run docker login for gradle build jobs

Issue: RELENG-4763
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I24aa3a8a8982a048d760d3012b9645e2689e8996

Fix: Add target filename for config file

Pass target file where the config file should be created. Newer
version of packer accepts only .json or .hcl extension filenames.

Error:
Could not guess format of
/w/workspace/builder-packer-verify@tmp/config5102323344744095713tmp
A var file must be suffixed with `.hcl` or `.json`.'

Issue: RELENG-4764
Change-Id: Ief8b703246667c38c216ede6ac94e5d549cfe46b
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Feat: Rewrite packer jobs to work with hcl2 format

Rewrite packer jobs to work with HCL2 format.

As of packer version 1.7.0 HCL2 is the preferred way to write Packer
templates. HCL2 preserves existing workflows while leveraging HCL2’s
advanced features like variable interpolation and configuration
composability.

Upgrade packer version to v1.9.1. JSON format templates are
deprecated and no longer work with packer version > 1.9.x.
Project specific templates require to be upgraded to HCL2 format.

Support for '.json' templates will be removed from common-packer in
subsequent releases. Therefore, the jobs are expected to work with
older templates.

Ref: https://gerrit.linuxfoundation.org/infra/c/releng/common-packer/+/71859
Issue: RELENG-4764
Change-Id: Ie591343ac87caca217ff7125a84f4b769bb3a40c
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Add JDK configuration step in Gradle jobs

Issue: RELENG-4769
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I50eb766d7b60e8da8ff3eafbda1d9fc4f2e817ca

CI: Add initial gradle jobs file

Issue: RELENG-4763
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I2d3d8627c0b9fa1e76b3d83ce5755c25fae79f8a

Fix: Pin urllib3 to <2.0.0 for the JJB cleanup

Issue: RELENG-4715
Change-Id: I8d5ef5e30682654b4212a320ffa20f061e11d828
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Pin urllib3 in pypi-upload script

Issue: RELENG-4715
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I0f8b34516d00d7eec95422c8552f2ec4ee404294

Fix: Pin urllib3~=1.26.15 in pypi dist jobs

Issue: RELENG-4715
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Change-Id: I8acbb9250a3224c29b6582940f60f6f8ebe19586

Fix: Pin urllib3~=1.26.15 in jjb-deploy job

Issue: RELENG-4731
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I7efdb713bec085356d972d5f9213aec335959c63

Fix: Pin urllib3 for RTD builds

RTD needs urllib3 pinned in the requirements.txt for proper building of
the RTD site itself.

Issue: RELENG-4715
Change-Id: Id125e99c8dff5239e95567e7148c595e10096044
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Fix: pin urllib3 to <2.0.0 for RTD jobs

The latest version of module breaks compatibility
with RTDv3 jobs during tox install and run.

Note: The pip upgrade eager option pulls in the newer
version of urllib3>2.0.0 unless the exact version is passed
with lftools.

Issue: RELENG-4715
Change-Id: I8aae2505eb20e7dc8bef02f5687f9dad9b0e8ef0
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: pin urllib3 to <2.0.0 for verify jobs

The latest version of module breaks compatibility
with python-jenkins.

Note: The pip upgrade eager option pulls in the newer
version of urllib3>2.0.0 unless the exact version is passed
with lftools.

Issue: RELENG-4715
Change-Id: Ib0136e8155b6a654d1b15a558796774e78a31a4d
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: pin urllib3 to <2.0.0

The latest version of module breaks compatibility
with python-jenkins.

Note: The pip upgrade eager option pulls in the newer
version of urllib3>2.0.0 unless the exact version is passed
with lftools.

Issue: RELENG-4715
Change-Id: Ie2fbc7fe1eb57e8936a50b176f7aae6e47efa7e6
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Correct action version

Issue: RELENG-4563
Change-Id: Ie3129c744d3774d88ed042609e6de7044f7f4de7
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Fix: Use refspec to push tag and code

The CR 365d0c0fe65 for the ODL release custom bits
uses the GERRIT_BRANCH while pushing the tag and code,
however that does not work returns an error.

error: src refspec stable/chlorine does not match any

Push the code bundle and tag as separate operations.

Issue: RELENG-4696
Change-Id: I62d472b998a2d799990605482e5433fb7c721e5f
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Correct bad action call

Issue: RELENG-4563
Change-Id: Idb27ec5805d2a52bb2e30e72931745e023b5ea80
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Fix: Remove recursive vars for release jobs

JJB 5x treats recursive parameters as an error.

Issue: RELENG-4686
Change-Id: I07df6e5662d1301caaafb628b0d884b7990e6809
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Fix: Update JJB to 5.0.2

JJB's 5.0.1 has a bug that return an error when a macro is
set as null. This issue is resolved in 5.0.2.

TypeError: 'NoneType' object is not a mapping

Issue: RELENG-4686
Change-Id: Iba6680ca7e3d54ca88b4854d9ba40006efe3f3e2
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Merge "Refactor!: JJB code to comply with v5.x"

CI: Add release GitHub Action

Issue: RELENG-4563
Change-Id: I5164d1e76df6dbd74c1b00b7e98b67519940bcda
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Refactor!: JJB code to comply with v5.x

There has been an update to the YAML parser with more fine-tuned
control over YAML parsing. YAML objects and parameter expansion logic
is in accordance with the latest YAML specification 1.2.2 (released
in Oct 2021).

Notable changes include:
- Aggressive parameter expansion. This may lead to parameters expanded
in places where they were not expanded before.
- Recursive parameters usage is being treated as error.
- Strict checking for missing parameters.
- Files included using '!include-raw:' elements and having formatting
  it path ('lazy-loaded' in previous implementation) are now expanded too.
  Use '!include-raw-escape:' for them instead.

NOTE: this is a beaking change would have potential impact many of jobs
in global-jjb and in ci-man repo that DO NOT use global-jjb templates.

Issue: RELENG-4686
Change-Id: Ie7417956bdcf5f4d6477dbff87baca6b1b8aeeaf
Ref: https://groups.google.com/u/2/g/jenkins-job-builder/c/1fx7PmADgXw
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Merge "Refactor: pre-commit cleanups"

Refactor: pre-commit cleanups

Cleanup errors being caught by pre-commit that is preventing GitHub
Actions runs of pre-commit (aka pre-commit run -a) from passing

Issue: RELENG-4687
Change-Id: I09636538d1d34e5ab2deeb05c112632540a17368
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Fix: Use the python3 module opt to call pip

This fixes the issue where pip is not available in the
venv path.

Error: /tmp/venv-knWF/bin/pip: No such file or directory

Issue-ID: RELENG-4688
Change-Id: I6d6edeb8c3cdf0c36f271412d3d2e96db96f6e4b
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

CI: Disable jjb-latest tox test

JJB v5.x released this morning (April 12, 2023) and started breaking the
jjb-latest validation test. As the test is a canary test, it did the
correct thing and broke our builds alerting us to the issue.

This disables the test so that other work is unblocked until such time
as we can determine the correct way forward with fixes and can turn this
test back on.

Issue: RELENG-4686
Change-Id: I94f6e944356c5aabb75434165a738b4d9336d818
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

CI: Add GHA for verification

Issue: RELENG-4564
Change-Id: I61c91c14b2e2211fae5062ac0b5354d6e84d3feb
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Fix: Newer tox needs allowlist vs whitelist

Issue: RELENG-4539
Change-Id: Id137b386671258f2c4fbdc115ea38429abfa3f43
Signed-off-by: Andrew Grimberg <agrimberg@linuxfoundation.org>

Merge "Chore: Remove daily cron on stage jobs"

Fix: condition before pushing the object.

Signed-off-by: Anil Belur <abelur@linuxfoundation.org>
Change-Id: If1d091bd9e94fa5337053094b0101541bb056ca4

Fix: Ensure tag and object are pushed together

Address the problem where the tag is not pushed to the mainline
branch therefore causing the tag missing in the git history.

To fix this check commit count between the HEAD and
origin/${GERRIT_BRANCH} before the fetch and merge operation.
This is done to ensure that the tag lands on the target branch.
If the branch has already moved forward from the tagging point,
then a spur commit is created for the tag.

Change-Id: I675c815d4ac39244adf5168c1fbd051f8c626290
Signed-off-by: Anil Belur <abelur@linuxfoundation.org>

Chore: Remove daily cron on stage jobs

Remove daily cron on maven-stage and maven-docker-stage
jobs. Cron triggers stay configurable.

Issue: RELENG-4666
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I15931a8d691bffa77dd0d50f638fc38590d33255

Merge "Fix!: Un-pin tox version from 3.27.1 and remove tox-pyenv"

FIX: Revert clm-maven plugin

Latest version of clm-maven plugin (clm-maven-plugin:2.42.0-01)
is not working in our environment.
We get the following error

Failed to execute goal com.sonatype.clm:clm-maven-plugin:2.42.0-01:index
(default-cli) on project babel: Failed to invoke Maven build.
Maven execution failed, exit code: 1 -> [Help 1]

This patch pins the previous working plugin which is 2.41.0-02

Issue-ID: IT-25225
Change-Id: I3d87d5adc954baf703f29816c0e295e0203b4f40
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>

Merge "Fix: Update path/location of update-alternatives"

Feat: Add Docker Snyk CLI Scanner jobs

Introduce Docker Snyk CLI scanner jobs. These jobs can be triggered
to download the latest version of Snyk's CLI scanner and trigger a
scan for Docker based repos. These jobs produce a report which is
published into Snyk's dashboard. These reports are fetched and
reflected back into the LFX Security tool.

Issue: RELENG-4609
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: Ifc9ab4c51393e893b22b06844f3701caaca06c6f

Feat: Add Python Snyk CLI Scanner jobs

Introduce Python Snyk CLI scanner jobs. These jobs can be triggered
to download the latest version of Snyk's CLI scanner and trigger a
scan for Python based repos. These jobs produce a report which is
published into Snyk's dashboard. These reports are fetched and
reflected back into the LFX Security tool.

Issue: RELENG-4609
Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
Change-Id: I5414d04ccc7472a6b3cd2576da1cb6bc36d1ea25

Fix!: Un-pin tox version from 3.27.1 and remove tox-pyenv

Un-pin tox version from 3.27.1 and remove tox-pyenv. Testing has
demonstrated that tox-pyenv is no longer required to obtain correct
Python runtime versions when running tests. Also, removed Python 3.8
from the VENV setup where it was being specifically requested.

Due to unpinning of the tox version, tox.ini configuration files may
need modifying to reflect a change in configuration syntax; where
whitelist_externals needs to be replaced with allowlist_externals.

Issue-ID: RELENG-4539
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>
Change-Id: Ifdd49de2a8e5054dad4a5d52125ede537049ad8e

Fix: Update path/location of update-alternatives

The path is different between CentOS7/8 and was incorrect for Ubuntu

Issue-ID: IT-25261
Signed-off-by: Matthew Watkins <mwatkins@linuxfoundation.org>
Change-Id: I91bfed027dcbe10c0dc39beac2ae71fb4f5f00ad
Signed-off-by: Bengt Thuree <bthuree@linuxfoundation.org>