# Global JJB
-The purpose of this repository is store generically define reusable JJB
-templates that can be deployed across LF projects.
+The purpose of this repository is store generically defined, reusable JJB
+templates, deployable across LF projects.
-The following variables are necessary to be defined in the Jenkins server as
+Define the following variables in the Jenkins server as
global environment variables as scripts in this repo expect these variables to
be available.
For example:
```
+GERRIT_URL=https://git.opendaylight.org/gerrit
GIT_URL=ssh://jenkins-$SILO@git.opendaylight.org:29418
+GIT_CLONE_URL=git@github.com:
JENKINS_HOSTNAME=jenkins092
LOGS_SERVER=https://logs.opendaylight.org
NEXUS_URL=https://nexus.opendaylight.org
SILO=releng
+SONAR_URL=https://sonar.opendaylight.org
```
+Note: Use **GIT_CLONE_URL** for GitHub projects as this
+will be different from the URL used the poperties
+configuration.
+
## Jenkins Plugin Requirements
**Required**
-- Config File Provider
-- Description Setter
-- Gerrit Trigger
-- Post Build Script
-- SSH Agent
-- Workspace Cleanup
+- [Config File Provider](https://plugins.jenkins.io/config-file-provider)
+- [Description Setter](https://plugins.jenkins.io/description-setter)
+- [Environment Injector Plugin](https://plugins.jenkins.io/envinject)
+- [Git plugin](https://plugins.jenkins.io/git)
+- [Post Build Script](https://plugins.jenkins.io/postbuildscript)
+
+ Due to security concerns Post Build Script is not available through
+ the Jenkins update center and must install it manually. You can
+ download it
+ [here](https://updates.jenkins-ci.org/download/plugins/postbuildscript/).
+
+- [SSH Agent](https://plugins.jenkins.io/ssh-agent)
+- [Workspace Cleanup](https://plugins.jenkins.io/ws-cleanup)
+
+**Required for Gerrit connected systems**
+
+- [Gerrit Trigger](https://plugins.jenkins.io/gerrit-trigger)
+
+**Required for GitHub connected systems**
+
+- [GitHub plugin](https://plugins.jenkins.io/github)
+- [GitHub Pull Request Builder](https://plugins.jenkins.io/ghprb)
**Optional**
-- Mask Passwords
-- MsgInject
-- OpenStack Cloud
-- Timestamps
+- [Mask Passwords](https://plugins.jenkins.io/mask-passwords)
+- [MsgInject](https://plugins.jenkins.io/msginject)
+- [OpenStack Cloud](https://plugins.jenkins.io/openstack-cloud)
+- [Timestamps](https://plugins.jenkins.io/timestamper)
## Installing global-jjb
-global-jjb should be deployed in the ci-management repository's jjb directory as
-a submodule. global-jjb is versioned and tagged in Gerrit so installing,
-upgrading, and rolling back changes should be simple via the Gerrit tag system.
+Deploy global-jjb in the ci-management repository's jjb directory as
+a submodule. Installing, upgrading, and rolling back changes is simple via the
+versioned git tags.
```
# Choose a global-jjb version to install
GLOBAL_JJB_VERSION=v0.1.0
# Add the new submodule to ci-management's jjb directory.
- # Note: Only needs to be performed once per ci-management repo.
+ # Note: Perform once per ci-management repo.
cd jjb/
- git submodule add https://gerrit.linuxfoundation.org/infra/releng/global-jjb
+
+ # For production deployments:
+ git submodule add https://github.com/lfit/releng-global-jjb global-jjb
+ # For test deployments comment the above and uncomment the below
+ # git submodule add https://gerrit.linuxfoundation.org/infra/releng/global-jjb
# Checkout the version of global-jjb you wish to deploy.
cd global-jjb
## Parameters stored in defaults.yaml
-There are a few project specific parameters that should be stored in the
-ci-management repo's defaults.yaml file.
+Configure the following parameters in the ci-management repo's
+defaults.yaml file.
+
+**gerrit-server-name**: The name of the Gerrit Server as defined
+in Gerrit Trigger global configuration.
-**gerrit-server-name**: The name of the Gerrit Server as defined in Gerrit
-Trigger global configuration.
+**jenkins-ssh-credential**: The name of the Jenkins Credential to
+use for ssh connections.
-**jenkins-ssh-credential**: The name of the Jenkins Credential to use for ssh
-connections.
+If you are using GitHub then configure the following parameters
+in defaults.yaml
-If you are using GitHub then there are two more parameters which
-will need to be placed in the defaults.yaml
+**git-url**: Set this to the base URL of your GitHub repo. In
+general this should be <https://github.com>. If you are using
+GitHub Enterprise, or some other GitHub-style system, then it
+should be whatever your installation base URL is.
-**github-url**: The URL of GitHub. (Default: https://github.com)
+**git-clone-url**: This is the clone prefix used by GitHub jobs.
+Set this to either the same thing as **git-url** or the
+'git@github.com:' including the trailing ':'
-**github-org**: The name of the GitHub organization.
+**github-org**: The name of the GitHub organization interpolated
+into the scm config.
+
+**github_pr_org**: The name of the GitHub organization. All members
+of this organization will be able to trigger any job using the
+`lf-infra-github-pr` macro.
+
+**github_pr_whitelist**: List of GitHub members you wish to be able to
+trigger any job that uses the `lf-infra-github-pr-trigger` macro.
+
+**github_pr_admin_list**: List of GitHub members that will have admin
+privileges on any job using the `lf-infra-github-pr-trigger`
+macro.
+
+**lftools-version**: Version of lftools to install. Can be a specific version
+like '0.6.1' or a PEP-440 definition. <https://www.python.org/dev/peps/pep-0440/>
+For example `<1.0.0` or `>=1.0.0,<2.0.0`.
+
+**mvn-site-id**: Maven Server ID from settings.xml containing the credentials
+to push to a Maven site repository.
+
+**mvn-staging-id**: Maven Server ID from settings.xml containing the credentials
+to push to a Maven staging repository.
defaults.yaml:
jenkins-ssh-credential: opendaylight-jenkins-ssh
gerrit-server-name: OpenDaylight
github-org: lfit
+ github_pr_whitelist:
+ - jpwku
+ - tykeal
+ - zxiiro
+ github_pr_admin_list:
+ - tykeal
+ lftools-version: '<1.0.0'
+ mvn-site-id: opendaylight-site
```
## Config File Management
## Deploying ci-jobs
-The CI job group contains multiple jobs that should be deployed in all LF
+The CI job group contains jobs that should deploy in all LF
Jenkins infra. The minimal configuration needed to deploy the ci-management
jobs is as follows which deploys the **{project-name}-ci-jobs** job group as
defined in **lf-ci-jobs.yaml**.
Required parameters:
-**project**: is the project repo as defined in Gerrit.
+**project**: is the project repo as defined in source control.
+**project-name**: is a custom name to call the job in Jenkins.
+**build-node**: is the name of the builder to use when building (Jenkins label).
+
+Optional parameters:
+
+**branch**: is the git branch to build from.
+**jjb-version**: is the version of JJB to install in the build minion.
+
+## Deploying packer-jobs
+
+The packer job group contains jobs to build custom minion images. The minimal
+configuration needed to deploy the packer jobs is as follows which deploys the
+**{project-name}-packer-jobs** job group as defined in **lf-ci-jobs.yaml**.
+
+ci-management.yaml:
+
+```
+- project:
+ name: packer-jobs
+
+ jobs:
+ - '{project-name}-packer-jobs'
+
+ project: ci-management
+ project-name: ci-management
+ branch: master
+ build-node: centos7-basebuild-2c-1g
+
+ platforms:
+ - centos
+ - ubuntu-14.04
+ - ubuntu-16.04
+
+ templates:
+ - devstack
+ - docker
+ - gbp
+ - java-builder
+ - mininet
+
+ exclude:
+ - platforms: centos
+ templates: gbp
+ - platforms: centos
+ templates: mininet
+```
+
+Required parameters:
+
+**project**: is the project repo as defined in source control.
**project-name**: is a custom name to call the job in Jenkins.
**build-node**: is the name of the builder to use when building (Jenkins label).
+**platforms**: is a list of supported platforms.
+**templates**: is a list of supported templates.
+
+Optional parameters:
+
+**branch**: is the git branch to build from.
+**packer-version**: is the version of packer to install in the build minion,
+when packer is not available.
+**exclude**: is a combination of platforms and templates which are not required
+to build.
+
+## Deploying Python jobs
+
+We provide the following Python jobs templates:
+
+### {project-name}-tox-verify-{stream}
+
+Use this job to call python-tox to run builds and tests. The most common
+usage of this job is to run the Coala linter against projects.
+
+```
+- project:
+ name: builder
+ jobs:
+ - '{project-name}-tox-verify-{stream}'
+
+ project-name: builder
+ project: releng/builder
+ build-node: centos7-java-builder-2c-4g
+ stream: master
+```
+
+Required parameters:
+
+**project**: is the project repo as defined in source control.
+**project-name**: is a custom name to call the job in Jenkins.
+**build-node**: is the name of the builder to use when building (Jenkins label).
+**stream**: typically `master` or matching the build branch. This
+ is a useful keywords to map a release codename to a branch. For
+ example OpenDaylight uses this to map stream=carbon to
+ branch=stable/carbon.
Optional parameters:
**branch**: is the git branch to build from.
**jjb-version**: is the version of JJB to install in the build minion.
+**tox-dir**: directory containing tox.ini file (default: '')
+**tox-envs**: tox environments to run (default: '')
## Archiving logs in Jobs
1) Job creates $WORKSPACE/archives directory and places logs there
-In this method the entire archives directory will be pushed to the log server
+This method pushes the entire archives directory to the log server
in the same structure as configured in the archives directory.
2) Via job variable ARCHIVE_ARTIFACTS using globstar patterns.
-In this method a job can define a globstar for example ``**/*.log`` which then
+In this method a job can define a globstar for example `**/*.log` which then
causes the archive script to do a globstar search for that pattern and archives
any files it finds matching.
+
+## Overriding merge and verify triggers
+
+The default trigger conditions for Merge and Verify job types are overrideable
+in a project configuration by overriding the following variables:
+
+- gerrit_merge_triggers
+- gerrit_verify_triggers
+
+These variables take a list of trigger-on values as defined in JJB docs here:
+<https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit>
+
+## Appendix
+
+### ShellCheck
+
+When using ShellCheck to lint global-jjb or any projects that include
+global-jjb as part of their project (common with ci-management repos) then
+we require version 0.4.x of ShellCheck installed on the build vms. This version
+introduces annotations used by shell scripts in this repo.
Code Review / releng / global-jjb.git / blobdiff