8 .. include:: ../job-groups.rst
10 Below is a list of Python job groups:
12 .. literalinclude:: ../../jjb/lf-python-job-groups.yaml
22 Runs CLM scanning against a Python project.
26 :clm-project-name: Project name in Nexus IQ to send results to.
31 Installs Tox into a virtualenv.
35 :python-version: Version of Python to invoke the pip install of the tox-pyenv
36 package that creates a virtual environment, either "python2" or "python3".
41 Creates a Tox virtual environment and invokes tox.
45 :parallel: Boolean. If true use detox (distributed tox);
55 CLM scans for Python based repos. This job will call the Nexus IQ CLI
56 directly to run the scans.
58 A new credential named "nexus-iq-xc-clm" needs to exist in the Jenkins
59 credentials. The credential should contain the username and password
60 to access Nexus IQ Server.
64 - {project-name}-python-clm-{stream}
65 - gerrit-python-xc-clm
66 - github-python-xc-clm
68 :Comment Trigger: run-clm
72 :build-node: The node to run build on.
73 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
74 get configured in defaults.yaml)
78 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
79 :nexus-iq-cli-version: Nexus IQ CLI package version to download and use.
81 :nexus-iq-namespace: Insert a namespace to project AppID for projects that
82 share a Nexus IQ system to avoid project name collision. We recommend
83 inserting a trailing - dash if using this parameter.
84 For example 'odl-'. (default: '')
85 :build-timeout: Timeout in minutes before aborting build. (default: 60)
86 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
87 :java-version: Version of Java to use for the build. (default: openjdk8)
88 :pre-build-script: Shell script to execute before the CLM builder.
89 For example, install prerequisites or move files to the repo root.
90 (default: a string with a shell comment)
91 :stream: Keyword used to represent a release code-name.
92 Often the same as the branch. (default: master)
93 :submodule-recursive: Whether to checkout submodules recursively.
95 :submodule-timeout: Timeout (in minutes) for checkout operation.
97 :submodule-disable: Disable submodule checkout operation.
99 :gerrit_clm_triggers: Override Gerrit Triggers.
100 :gerrit_trigger_file_paths: Override file paths used to filter which file
101 modifications trigger a build. Refer to JJB documentation for "file-path" details.
102 https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit
105 Python Sonar with Tox
106 ---------------------
108 Sonar scans for Python based repos. This job invokes tox to run tests
109 and gather coverage statistics from the test results, then invokes
110 Maven to publish the results to either a Sonar server or SonarCloud.
112 To get the Sonar coverage results, file tox.ini must exist and contain
113 coverage commands to run.
115 The coverage commands define the code that gets executed by the test
116 suites. Checking coverage does not guarantee that the tests execute
117 properly, but it identifies code that is not executed by any test.
119 .. comment Start ignoring WriteGoodLintBear
121 This job reuses the Sonar builders used for Java/Maven projects which
122 run maven twice. The first invocation does nothing for Python
123 projects, so the job uses the goal 'validate' by default. The second
124 invocation publishes results using the goal 'sonar:sonar' by default.
126 .. comment Stop ignoring
134 coverage run --module pytest --junitxml xunit-results.xml
135 coverage xml --omit=".tox/py27/*","tests/*"
136 coverage report --omit=".tox/py27/*","tests/*"
138 For more details refer to coverage and sonar documentation:
140 https://coverage.readthedocs.io/
142 https://docs.sonarqube.org/display/PLUG/Python+Coverage+Results+Import
146 - {project-name}-tox-sonar
150 :Comment Trigger: run-sonar
152 :Required parameters:
154 :build-node: The node to run build on.
155 :jenkins-ssh-credential: Credential to use for SSH. (Generally should
156 get configured in defaults.yaml)
157 :mvn-settings: The name of the settings file with credentials for the project.
159 .. comment Start ignoring WriteGoodLintBear
161 :Optional parameters:
163 :branch: Git branch, should be master (default: master)
164 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
165 :build-timeout: Timeout in minutes before aborting build. (default: 60)
166 :cron: Cron schedule when to trigger the job. This parameter also
167 supports multiline input via YAML pipe | character in cases where
168 one may want to provide more than 1 cron timer. (default: H 11 * * *
170 :disable-job: Whether to disable the job (default: false)
171 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
172 :github-url: URL for Github. (default: https://github.com)
173 :java-version: Version of Java to use for the build. (default: openjdk8)
174 :mvn-global-settings: The name of the Maven global settings to use
175 :mvn-goals: The Maven goal to run first. (default: validate)
176 :mvn-version: Version of maven to use. (default: mvn35)
177 :parallel: Boolean indicator for tox to run tests in parallel or series.
178 (default: false, in series)
179 :pre-build-script: Shell script to execute before the Sonar builder.
180 For example, install prerequisites or move files to the repo root.
181 (default: a string with a shell comment)
182 :python-version: Python version to invoke pip install of tox-pyenv
184 :sonarcloud: Boolean indicator to use SonarCloud ``true|false``.
186 :sonarcloud-project-key: SonarCloud project key. (default: '')
187 :sonarcloud-project-organization: SonarCloud project organization.
189 :sonarcloud-api-token: SonarCloud API Token. (default: '')
190 :sonar-mvn-goal: The Maven goal to run the Sonar plugin. (default: sonar:sonar)
191 :stream: Keyword used to represent a release code-name.
192 Often the same as the branch. (default: master)
193 :submodule-recursive: Whether to checkout submodules recursively.
195 :submodule-timeout: Timeout (in minutes) for checkout operation.
197 :submodule-disable: Disable submodule checkout operation.
199 :tox-dir: Directory containing the project's tox.ini relative to
200 the workspace. The default uses tox.ini at the project root.
202 :tox-envs: Tox environments to run. If blank run everything described
203 in tox.ini. (default: '')
204 :gerrit_sonar_triggers: Override Gerrit Triggers.
205 :gerrit_trigger_file_paths: Override file paths used to filter which file
206 modifications trigger a build. Refer to JJB documentation for "file-path" details.
207 https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit
209 .. comment Stop ignoring
215 Tox runner to verify a project on creation of a patch set. This job
216 is pyenv aware so if the image contains an installation of pyenv at
217 /opt/pyenv it will pick it up and run Python tests with the
218 appropriate Python versions. This job will set the following pyenv
219 variables before running.
223 export PYENV_ROOT="/opt/pyenv"
224 export PATH="$PYENV_ROOT/bin:$PATH"
228 - {project-name}-tox-verify-{stream}
232 :Comment Trigger: recheck|reverify
234 :Required Parameters:
236 :build-node: The node to run build on.
237 :jenkins-ssh-credential: Credential to use for SSH. (Generally set
240 :Optional Parameters:
242 :branch: The branch to build against. (default: master)
243 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
244 :build-timeout: Timeout in minutes before aborting build. (default: 10)
245 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
246 :pre-build-script: Shell script to execute before the Tox builder.
247 For example, install prerequisites or move files to the repo root.
248 (default: a string with a shell comment)
249 :parallel: Boolean indicator for tox to run tests in parallel or series.
250 (default: false, in series)
251 :python-version: Python version to invoke pip install of tox-pyenv
253 :stream: Keyword representing a release code-name.
254 Often the same as the branch. (default: master)
255 :submodule-recursive: Whether to checkout submodules recursively.
257 :submodule-timeout: Timeout (in minutes) for checkout operation.
259 :submodule-disable: Disable submodule checkout operation.
261 :tox-dir: Directory containing the project's tox.ini relative to
262 the workspace. The default uses tox.ini at the project root.
264 :tox-envs: Tox environments to run. If blank run everything described
265 in tox.ini. (default: '')
266 :gerrit_trigger_file_paths: Override file paths used to filter which file
267 modifications trigger a build. Refer to JJB documentation for "file-path" details.
268 https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit
274 Tox runner to verify a project after merge of a patch set. This job
275 is pyenv aware so if the image contains an installation of pyenv at
276 /opt/pyenv it will pick it up and run Python tests with the
277 appropriate Python versions. This job will set the following pyenv
278 variables before running.
282 export PYENV_ROOT="/opt/pyenv"
283 export PATH="$PYENV_ROOT/bin:$PATH"
287 - {project-name}-tox-merge-{stream}
291 :Comment Trigger: remerge
293 :Required Parameters:
295 :build-node: The node to run build on.
296 :jenkins-ssh-credential: Credential to use for SSH. (Generally set
299 :Optional Parameters:
301 :branch: The branch to build against. (default: master)
302 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
303 :build-timeout: Timeout in minutes before aborting build. (default: 10)
304 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
305 :pre-build-script: Shell script to execute before the CLM builder.
306 For example, install prerequisites or move files to the repo root.
307 (default: a string with a shell comment)
308 :python-version: Python version to invoke pip install of tox-pyenv
310 :stream: Keyword representing a release code-name.
311 Often the same as the branch. (default: master)
312 :submodule-recursive: Whether to checkout submodules recursively.
314 :submodule-timeout: Timeout (in minutes) for checkout operation.
316 :submodule-disable: Disable submodule checkout operation.
318 :tox-dir: Directory containing the project's tox.ini relative to
319 the workspace. The default uses tox.ini at the project root.
321 :tox-envs: Tox environments to run. If blank run everything described
322 in tox.ini. (default: '')
323 :gerrit_trigger_file_paths: Override file paths used to filter which file
324 modifications trigger a build. Refer to JJB documentation for "file-path" details.
325 https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit
331 Verifies a Python library project on creation of a patch set. Runs tox
332 then builds a source distribution and (optionally) a binary
333 distribution. The project repository must have a setup.py file with
334 configuration for packaging the component.
336 The tox runner is pyenv aware so if the image contains an installation
337 of pyenv at /opt/pyenv it will pick it up and run Python tests with
338 the appropriate Python versions. The tox runner sets the following
339 pyenv variables before running.
343 export PYENV_ROOT="/opt/pyenv"
344 export PATH="$PYENV_ROOT/bin:$PATH"
348 - {project-name}-pypi-verify-{stream}
352 :Comment Trigger: recheck
354 :Required Parameters:
356 :build-node: The node to run the build on.
357 :jenkins-ssh-credential: Credential to use for SSH. (Generally set
359 :project: Git repository name
360 :project-name: Jenkins job name prefix
362 :Optional Parameters:
364 :branch: The branch to build against. (default: master)
365 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
366 :build-timeout: Timeout in minutes before aborting build. (default: 15)
367 :disable-job: Whether to disable the job (default: false)
368 :dist-binary: Whether to build a binary wheel distribution. (default: true)
369 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
370 :parallel: Boolean indicator for tox to run tests in parallel or series.
371 (default: false, in series)
372 :pre-build-script: Shell script to execute before the tox builder. For
373 example, install system prerequisites. (default: a shell comment)
374 :python-version: Python version to invoke pip install of tox-pyenv
376 :stream: Keyword representing a release code-name.
377 Often the same as the branch. (default: master)
378 :submodule-recursive: Whether to checkout submodules recursively.
380 :submodule-timeout: Timeout (in minutes) for checkout operation.
382 :submodule-disable: Disable submodule checkout operation.
384 :tox-dir: Directory containing the project's tox.ini relative to
385 the workspace. The default uses tox.ini at the project root.
387 :tox-envs: Tox environments to run. If blank run everything described
388 in tox.ini. (default: '')
389 :gerrit_trigger_file_paths: Override file paths used to filter which file
390 modifications trigger a build. Refer to JJB documentation for "file-path" details.
391 https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit
397 Creates and uploads distribution files on merge of a patch set. Runs
398 tox, builds a source distribution and (optionally) a binary
399 distribution, and uploads the distribution(s) to a PyPI repository.
400 This job should use a staging repository like testpypi.python.org,
401 which sets up use of release jobs to promote the distributions later.
402 This job can also use a public release area like the global PyPI
403 repository. The project git repository must have a setup.py file
404 with configuration for packaging the component.
406 The tox runner is pyenv aware so if the image contains an installation
407 of pyenv at /opt/pyenv it will pick it up and run Python tests with
408 the appropriate Python versions. The tox runner sets the following
409 pyenv variables before running.
413 export PYENV_ROOT="/opt/pyenv"
414 export PATH="$PYENV_ROOT/bin:$PATH"
417 Requires a .pypirc configuration file in the Jenkins builder home directory,
418 an example appears next that uses API tokens. Note that in the [pypi] entry
419 the repository key-value pair is optional, it defaults to pypi.org.
423 [distutils] # this tells distutils what package indexes you can push to
424 index-servers = pypi-test pypi
427 repository: https://test.pypi.org/legacy/
429 password: pypi-test-api-token-goes-here
433 password: pypi-api-token-goes-here
438 - {project-name}-pypi-merge-{stream}
442 :Comment Trigger: remerge
444 :Required Parameters:
446 :build-node: The node to run the build on.
447 :jenkins-ssh-credential: Credential to use for SSH. (Generally set
449 :project: Git repository name
450 :project-name: Jenkins job name prefix
452 :Optional Parameters:
454 :branch: The branch to build against. (default: master)
455 :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
456 :build-timeout: Timeout in minutes before aborting build. (default: 15)
457 :cron: Cron schedule when to trigger the job. Supports daily builds.
458 This parameter also supports multiline input via YAML pipe | character in
459 cases where one may want to provide more than 1 cron timer. (default: empty)
460 :disable-job: Whether to disable the job (default: false)
461 :dist-binary: Whether to build a binary wheel distribution. (default: true)
462 :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
463 :parallel: Boolean indicator for tox to run tests in parallel or series.
464 (default: false, in series)
465 :pre-build-script: Shell script to execute before the tox builder. For
466 example, install system prerequisites. (default: a shell comment)
467 :pypi-repo: Key for the PyPI target repository in the .pypirc file,
468 ideally a server like test.pypy.org. (default: pypi-test)
469 :python-version: Python version to invoke pip install of tox-pyenv
471 :stream: Keyword representing a release code-name.
472 Often the same as the branch. (default: master)
473 :submodule-recursive: Whether to checkout submodules recursively.
475 :submodule-timeout: Timeout (in minutes) for checkout operation.
477 :submodule-disable: Disable submodule checkout operation.
479 :tox-dir: Directory containing the project's tox.ini relative to
480 the workspace. The default uses tox.ini at the project root.
482 :tox-envs: Tox environments to run. If blank run everything described
483 in tox.ini. (default: '')
484 :gerrit_trigger_file_paths: Override file paths used to filter which file
485 modifications trigger a build. Refer to JJB documentation for "file-path" details.
486 https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit