jobs:
- gerrit-tox-sonarqube
+
Tox Verify
----------
PyPI Merge
----------
-Creates and uploads distribution files on merge of a patch set. Runs
-tox, builds a source distribution and (optionally) a binary
+Creates and uploads package distribution files on merge of a patch set.
+Runs tox, builds a source distribution and (optionally) a binary
distribution, and uploads the distribution(s) to a PyPI repository.
The project git repository must have a setup.py file
with configuration for packaging the component.
+Projects can choose **either** this template to publish on merge,
+**or** the Stage template to publish on command.
+
This job should use a staging repository like testpypi.python.org,
which sets up use of release jobs to promote the distributions later.
This job can also use a public release area like the global PyPI
export PYENV_ROOT="/opt/pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
-Installable package projects should use the directory layout shown
-below. All Python files are in a repo subdirectory separate from
-non-Python files like documentation. This layout allows highly
-specific build-job triggers in Jenkins using the subdirectory
-paths. For example, a PyPI merge job should not run on a non-Python
-file change such as documentation, because the job cannot upload the
-same package twice.
-
-To make the document files available for building a Python package
-long description in setup.py, add a symbolic link "docs" in the
-package subdirectory pointing to the top-level docs directory.
+See the recommended directory layout documented in the PyPI Verify job.
-.. code-block:: bash
-
- git-repo-name/
- │
- ├── docs/
- │ ├── index.rst
- │ └── release-notes.rst
- │
- ├── helloworld-package/
- │ │
- │ └── helloworld/
- │ │ ├── __init__.py
- │ │ ├── helloworld.py
- │ │ └── helpers.py
- │ │
- │ ├── tests/
- │ │ ├── helloworld_tests.py
- │ │ └── helloworld_mocks.py
- │ │
- │ ├── requirements.txt
- │ └── setup.py
- │ └── tox.ini
- │
- ├── releases/
- │ └── pypi-helloworld.yaml
- │
- ├── .gitignore
- ├── LICENSE
- └── README.md
-
-
-Jobs built from the PyPI templates depend on a .pypirc configuration file
+Jobs using this PyPI template depend on a .pypirc configuration file
in the Jenkins builder home directory. An example appears next that uses
API tokens. Note that in the [pypi] entry the repository key-value pair
is optional, it defaults to pypi.org.
:branch: The branch to build against. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
:build-timeout: Timeout in minutes before aborting build. (default: 15)
- :cron: Cron schedule when to trigger the job. Supports daily builds.
- This parameter also supports multiline input via YAML pipe | character in
- cases where one may want to provide more than 1 cron timer. (default: empty)
+ :cron: Cron schedule when to trigger the job. Supports regular builds.
+ Not useful when publishing to pypi.org because that rejects a package
+ if the version exists. (default: empty)
:disable-job: Whether to disable the job (default: false)
:dist-binary: Whether to build a binary wheel distribution. (default: true)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
:pre-build-script: Shell script to execute before the tox builder. For
example, install system prerequisites. (default: a shell comment)
:pypi-repo: Key for the PyPI target repository in the .pypirc file,
- ideally a server like test.pypy.org. (default: pypi-test)
+ ideally a server like test.pypi.org. (default: pypi-test)
:python-version: Python version to invoke pip install of tox-pyenv
(default: python3)
:stream: Keyword representing a release code-name.
https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit
+PyPI Stage
+----------
+
+Creates and uploads package distribution files on receipt of a comment.
+Runs tox, builds a source distribution and (optionally) a binary
+distribution, and uploads the distribution(s) to a PyPI repository.
+The project git repository must have a setup.py file with configuration
+for packaging the component.
+
+Projects can choose **either** this template to publish on command,
+**or** the Merge template to publish on merge.
+
+This job should use a staging repository like testpypi.python.org,
+which sets up use of release jobs to promote the distributions later.
+This job can also use a public release area like the global PyPI
+repository if the release process is not needed. These PyPI
+repositories allow upload of a package at a specific version once,
+they do not allow overwrite of a package. This means that a job
+will fail in the upload step if the package version already exists in
+the target repository.
+
+The tox runner is pyenv aware so if the image contains an installation
+of pyenv at /opt/pyenv it will pick it up and run Python tests with
+the appropriate Python versions. The tox runner sets the following
+pyenv variables before running.
+
+.. code:: bash
+
+ export PYENV_ROOT="/opt/pyenv"
+ export PATH="$PYENV_ROOT/bin:$PATH"
+
+See the recommended directory layout documented in the PyPI Verify job.
+
+Jobs using this PyPI template depend on a .pypirc configuration file
+in the Jenkins builder home directory. An example appears next that uses
+API tokens. Note that in the [pypi] entry the repository key-value pair
+is optional, it defaults to pypi.org.
+
+.. code-block:: bash
+
+ [distutils] # this tells distutils what package indexes you can push to
+ index-servers = pypi-test pypi
+
+ [pypi-test]
+ repository: https://test.pypi.org/legacy/
+ username: __token__
+ password: pypi-test-api-token-goes-here
+
+ [pypi]
+ username: __token__
+ password: pypi-api-token-goes-here
+
+
+:Template Names:
+
+ - {project-name}-pypi-stage-{stream}
+ - gerrit-pypi-stage
+ - github-pypi-stage
+
+:Comment Trigger: **stage-release** post a comment with the trigger to launch
+ this job manually. Do not include any other text or vote in the
+ same comment.
+
+:Required Parameters:
+
+ :build-node: The node to run the build on.
+ :jenkins-ssh-credential: Credential to use for SSH. (Generally set
+ in defaults.yaml)
+ :mvn-settings: The settings file with credentials for the project
+ :project: Git repository name
+ :project-name: Jenkins job name prefix
+
+:Optional Parameters:
+
+ :branch: The branch to build against. (default: master)
+ :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
+ :build-timeout: Timeout in minutes before aborting build. (default: 15)
+ :cron: Cron schedule when to trigger the job. Supports regular builds.
+ Not useful when publishing to pypi.org because that rejects a package
+ if the version exists. (default: empty)
+ :disable-job: Whether to disable the job (default: false)
+ :dist-binary: Whether to build a binary wheel distribution. (default: true)
+ :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+ :mvn-opts: Sets MAVEN_OPTS to start up the JVM running Maven. (default: '')
+ :mvn-params: Parameters to pass to the mvn CLI. (default: '')
+ :mvn-version: Version of maven to use. (default: mvn35)
+ :parallel: Boolean indicator for tox to run tests in parallel or series.
+ (default: false, in series)
+ :pre-build-script: Shell script to execute before the tox builder. For
+ example, install system prerequisites. (default: a shell comment)
+ :pypi-repo: Key for the PyPI target repository in the .pypirc file,
+ ideally a server like test.pypi.org. (default: pypi-test)
+ :python-version: Python version to invoke pip install of tox-pyenv
+ (default: python3)
+ :stream: Keyword representing a release code-name.
+ Often the same as the branch. (default: master)
+ :submodule-recursive: Whether to checkout submodules recursively.
+ (default: true)
+ :submodule-timeout: Timeout (in minutes) for checkout operation.
+ (default: 10)
+ :submodule-disable: Disable submodule checkout operation.
+ (default: false)
+ :tox-dir: Directory containing the project's tox.ini relative to
+ the workspace. The default uses tox.ini at the project root.
+ (default: '.')
+ :tox-envs: Tox environments to run. If blank run everything described
+ in tox.ini. (default: '')
+ :gerrit_trigger_file_paths: Override file paths used to filter which file
+ modifications trigger a build. Refer to JJB documentation for "file-path" details.
+ https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit
+
PyPI Verify
-----------
distribution. The project repository must have a setup.py file with
configuration for packaging the component.
+Installable package projects should use the directory layout shown
+below. All Python files are in a repo subdirectory separate from
+non-Python files like documentation. This layout allows highly
+specific build-job triggers in Jenkins using the subdirectory
+paths. For example, a PyPI publisher job should not run on a non-Python
+file change such as documentation, because the job cannot upload the
+same package twice.
+
+To make the document files available for building a Python package
+long description in setup.py, add a symbolic link "docs" in the
+package subdirectory pointing to the top-level docs directory.
+
+.. code-block:: bash
+
+ git-repo-name/
+ │
+ ├── docs/
+ │ ├── index.rst
+ │ └── release-notes.rst
+ │
+ ├── helloworld-package/
+ │ │
+ │ └── helloworld/
+ │ │ ├── __init__.py
+ │ │ ├── helloworld.py
+ │ │ └── helpers.py
+ │ │
+ │ ├── tests/
+ │ │ ├── helloworld_tests.py
+ │ │ └── helloworld_mocks.py
+ │ │
+ │ ├── requirements.txt
+ │ └── setup.py
+ │ └── tox.ini
+ │
+ ├── releases/
+ │ └── pypi-helloworld.yaml
+ │
+ ├── .gitignore
+ ├── LICENSE
+ └── README.md
+
+
The tox runner is pyenv aware so if the image contains an installation
of pyenv at /opt/pyenv it will pick it up and run Python tests with
the appropriate Python versions. The tox runner sets the following
parallel: "{parallel}"
- shell: !include-raw-escape: ../shell/pypi-dist-build.sh
-- lf_pypi_merge_builders: &lf_pypi_merge_builders
- name: lf-pypi-merge-builders
+- lf_pypi_publish_builders: &lf_pypi_publish_builders
+ name: lf-pypi-publish-builders
builders:
- lf-infra-pre-build
name: "{project-name}-pypi-merge-{stream}"
id: gerrit-pypi-merge
<<: *lf_pypi_common
- <<: *lf_pypi_merge_builders
+ <<: *lf_pypi_publish_builders
- cron: ""
+ cron: "" # avoid for pypi which rejects duplicates
pypi-repo: pypi-test
scm:
name: "{project-name}-pypi-merge-{stream}"
id: github-pypi-merge
<<: *lf_pypi_common
- <<: *lf_pypi_merge_builders
+ <<: *lf_pypi_publish_builders
cron: ""
pypi-repo: pypi-test
white-list-target-branches:
- "{branch}"
included-regions: "{obj:github_included_regions}"
+
+- job-template:
+ name: "{project-name}-pypi-stage-{stream}"
+ id: gerrit-pypi-stage
+ <<: *lf_pypi_common
+ <<: *lf_pypi_publish_builders
+
+ cron: ""
+ pypi-repo: pypi-test
+
+ gerrit_stage_triggers:
+ - comment-added-contains-event:
+ comment-contains-value: '^Patch Set\s+\d+:\s+stage-release\s*$'
+
+ scm:
+ - lf-infra-gerrit-scm:
+ jenkins-ssh-credential: "{jenkins-ssh-credential}"
+ git-url: "{git-url}"
+ refspec: "$GERRIT_REFSPEC"
+ branch: "$GERRIT_BRANCH"
+ submodule-recursive: "{submodule-recursive}"
+ submodule-timeout: "{submodule-timeout}"
+ submodule-disable: "{submodule-disable}"
+ # stage jobs always build from tip
+ choosing-strategy: default
+
+ triggers:
+ - timed: "{obj:cron}"
+ - gerrit:
+ server-name: "{gerrit-server-name}"
+ trigger-on: "{obj:gerrit_stage_triggers}"
+ projects:
+ - project-compare-type: ANT
+ project-pattern: "{project}"
+ branches:
+ - branch-compare-type: ANT
+ branch-pattern: "**/{branch}"
+ file-paths: "{obj:gerrit_trigger_file_paths}"
+
+- job-template:
+ name: "{project-name}-pypi-stage-{stream}"
+ id: github-pypi-stage
+ <<: *lf_pypi_common
+ <<: *lf_pypi_publish_builders
+
+ cron: ""
+ pypi-repo: pypi-test
+
+ properties:
+ - github:
+ url: "{github-url}/{github-org}/{project}"
+
+ scm:
+ - lf-infra-github-scm:
+ url: "{git-clone-url}{github-org}/{project}"
+ refspec: ""
+ branch: "refs/heads/{branch}"
+ submodule-recursive: "{submodule-recursive}"
+ submodule-timeout: "{submodule-timeout}"
+ submodule-disable: "{submodule-disable}"
+ choosing-strategy: default
+ jenkins-ssh-credential: "{jenkins-ssh-credential}"
+
+ triggers:
+ - timed: "{obj:cron}"
+ - github-pull-request:
+ trigger-phrase: "^stage-release$"
+ only-trigger-phrase: true
+ status-context: "Release"
+ permit-all: true
+ github-hooks: true
+ white-list-target-branches:
+ - "{branch}"
+ included-regions: "{obj:github_included_regions}"
Code Review / releng / global-jjb.git / commitdiff