[all.YAML]
bears = YAMLLintBear
-files = **.yaml
+files = **.yaml, **.yml
document_start = True
yamllint_config = yamllint.conf
# Global JJB
-The purpose of this repository is store generically defined, reusable JJB
-templates, deployable across LF projects.
+Global-JJB is a library project containing reusable Jenkins Job Builder
+templates. Developed for LFCI to deploy management Jenkins jobs to an LF
+managed Jenkins instance, there are other jobs defined which may be helpful
+to projects that use the same build technology. The intention is to help
+projects save time from having to define their own job templates.
-Define the following variables in the Jenkins server as
-global environment variables as scripts in this repo expect these variables to
-be available.
-
-For example:
-
-```
-GERRIT_URL=https://git.opendaylight.org/gerrit
-GIT_URL=ssh://jenkins-$SILO@git.opendaylight.org:29418
-GIT_CLONE_URL=git@github.com:
-JENKINS_HOSTNAME=jenkins092
-LOGS_SERVER=https://logs.opendaylight.org
-NEXUS_URL=https://nexus.opendaylight.org
-SILO=releng
-SONAR_URL=https://sonar.opendaylight.org
-```
-
-Note: Use **GIT_CLONE_URL** for GitHub projects as this
-will be different from the URL used the poperties
-configuration.
-
-## Jenkins Plugin Requirements
-
-**Required**
-
-- [Config File Provider](https://plugins.jenkins.io/config-file-provider)
-- [Description Setter](https://plugins.jenkins.io/description-setter)
-- [Environment Injector Plugin](https://plugins.jenkins.io/envinject)
-- [Git plugin](https://plugins.jenkins.io/git)
-- [Post Build Script](https://plugins.jenkins.io/postbuildscript)
-
- Due to security concerns Post Build Script is not available through
- the Jenkins update center and must install it manually. You can
- download it
- [here](https://updates.jenkins-ci.org/download/plugins/postbuildscript/).
-
-- [SSH Agent](https://plugins.jenkins.io/ssh-agent)
-- [Workspace Cleanup](https://plugins.jenkins.io/ws-cleanup)
-
-**Required for Gerrit connected systems**
-
-- [Gerrit Trigger](https://plugins.jenkins.io/gerrit-trigger)
-
-**Required for GitHub connected systems**
-
-- [GitHub plugin](https://plugins.jenkins.io/github)
-- [GitHub Pull Request Builder](https://plugins.jenkins.io/ghprb)
-
-**Optional**
-
-- [Mask Passwords](https://plugins.jenkins.io/mask-passwords)
-- [MsgInject](https://plugins.jenkins.io/msginject)
-- [OpenStack Cloud](https://plugins.jenkins.io/openstack-cloud)
-- [Timestamps](https://plugins.jenkins.io/timestamper)
-
-## Installing global-jjb
-
-Deploy global-jjb in the ci-management repository's jjb directory as
-a submodule. Installing, upgrading, and rolling back changes is simple via the
-versioned git tags.
-
-```
- # Choose a global-jjb version to install
- GLOBAL_JJB_VERSION=v0.1.0
-
- # Add the new submodule to ci-management's jjb directory.
- # Note: Perform once per ci-management repo.
- cd jjb/
-
- # For production deployments:
- git submodule add https://github.com/lfit/releng-global-jjb global-jjb
- # For test deployments comment the above and uncomment the below
- # git submodule add https://gerrit.linuxfoundation.org/infra/releng/global-jjb
-
- # Checkout the version of global-jjb you wish to deploy.
- cd global-jjb
- git checkout $GLOBAL_JJB_VERSION
-
- # Commit global-jjb version to the ci-management repo.
- cd ../..
- git add jjb/global-jjb
- git commit -sm "Install global-jjb $GLOBAL_JJB_VERSION"
-
- # Push the patch to ci-management for review
- git review
-```
-
-## Parameters stored in defaults.yaml
-
-Configure the following parameters in the ci-management repo's
-defaults.yaml file.
-
-**gerrit-server-name**: The name of the Gerrit Server as defined
-in Gerrit Trigger global configuration.
-
-**jenkins-ssh-credential**: The name of the Jenkins Credential to
-use for ssh connections.
-
-If you are using GitHub then configure the following parameters
-in defaults.yaml
-
-**git-url**: Set this to the base URL of your GitHub repo. In
-general this should be <https://github.com>. If you are using
-GitHub Enterprise, or some other GitHub-style system, then it
-should be whatever your installation base URL is.
-
-**git-clone-url**: This is the clone prefix used by GitHub jobs.
-Set this to either the same thing as **git-url** or the
-'git@github.com:' including the trailing ':'
-
-**github-org**: The name of the GitHub organization interpolated
-into the scm config.
-
-**github_pr_org**: The name of the GitHub organization. All members
-of this organization will be able to trigger any job using the
-`lf-infra-github-pr` macro.
-
-**github_pr_whitelist**: List of GitHub members you wish to be able to
-trigger any job that uses the `lf-infra-github-pr-trigger` macro.
-
-**github_pr_admin_list**: List of GitHub members that will have admin
-privileges on any job using the `lf-infra-github-pr-trigger`
-macro.
-
-**lftools-version**: Version of lftools to install. Can be a specific version
-like '0.6.1' or a PEP-440 definition. <https://www.python.org/dev/peps/pep-0440/>
-For example `<1.0.0` or `>=1.0.0,<2.0.0`.
-
-**mvn-site-id**: Maven Server ID from settings.xml containing the credentials
-to push to a Maven site repository.
-
-**mvn-staging-id**: Maven Server ID from settings.xml containing the credentials
-to push to a Maven staging repository.
-
-defaults.yaml:
-
-```
-- defaults:
- name: global
-
- # lf-infra defaults
- jenkins-ssh-credential: opendaylight-jenkins-ssh
- gerrit-server-name: OpenDaylight
- github-org: lfit
- github_pr_whitelist:
- - jpwku
- - tykeal
- - zxiiro
- github_pr_admin_list:
- - tykeal
- lftools-version: '<1.0.0'
- mvn-site-id: opendaylight-site
-```
-
-## Config File Management
-
-### Logs
-
-The logs account requires a Maven Settings file created called
-**jenkins-log-archives-settings** with a server ID of **logs** containing the
-credentials for the logs user in Nexus.
-
-## Deploying ci-jobs
-
-The CI job group contains jobs that should deploy in all LF
-Jenkins infra. The minimal configuration needed to deploy the ci-management
-jobs is as follows which deploys the **{project-name}-ci-jobs** job group as
-defined in **lf-ci-jobs.yaml**.
-
-ci-management.yaml:
-
-```
-- project:
- name: ci-jobs
-
- jobs:
- - '{project-name}-ci-jobs'
-
- project: ci-management
- project-name: ci-management
- build-node: centos7-basebuild-2c-1g
-```
-
-Required parameters:
-
-**project**: is the project repo as defined in source control.
-**project-name**: is a custom name to call the job in Jenkins.
-**build-node**: is the name of the builder to use when building (Jenkins label).
-
-Optional parameters:
-
-**branch**: is the git branch to build from.
-**jjb-version**: is the version of JJB to install in the build minion.
-
-## Deploying packer-jobs
-
-The packer job group contains jobs to build custom minion images. The minimal
-configuration needed to deploy the packer jobs is as follows which deploys the
-**{project-name}-packer-jobs** job group as defined in **lf-ci-jobs.yaml**.
-
-ci-management.yaml:
-
-```
-- project:
- name: packer-jobs
-
- jobs:
- - '{project-name}-packer-jobs'
-
- project: ci-management
- project-name: ci-management
- branch: master
- build-node: centos7-basebuild-2c-1g
-
- platforms:
- - centos
- - ubuntu-14.04
- - ubuntu-16.04
-
- templates:
- - devstack
- - docker
- - gbp
- - java-builder
- - mininet
-
- exclude:
- - platforms: centos
- templates: gbp
- - platforms: centos
- templates: mininet
-```
-
-Required parameters:
-
-**project**: is the project repo as defined in source control.
-**project-name**: is a custom name to call the job in Jenkins.
-**build-node**: is the name of the builder to use when building (Jenkins label).
-**platforms**: is a list of supported platforms.
-**templates**: is a list of supported templates.
-
-Optional parameters:
-
-**branch**: is the git branch to build from.
-**packer-version**: is the version of packer to install in the build minion,
-when packer is not available.
-**exclude**: is a combination of platforms and templates which are not required
-to build.
-
-## Deploying Python jobs
-
-We provide the following Python jobs templates:
-
-### {project-name}-tox-verify-{stream}
-
-Use this job to call python-tox to run builds and tests. The most common
-usage of this job is to run the Coala linter against projects.
-
-```
-- project:
- name: builder
- jobs:
- - '{project-name}-tox-verify-{stream}'
-
- project-name: builder
- project: releng/builder
- build-node: centos7-java-builder-2c-4g
- stream: master
-```
-
-Required parameters:
-
-**project**: is the project repo as defined in source control.
-**project-name**: is a custom name to call the job in Jenkins.
-**build-node**: is the name of the builder to use when building (Jenkins label).
-**stream**: typically `master` or matching the build branch. This
- is a useful keywords to map a release codename to a branch. For
- example OpenDaylight uses this to map stream=carbon to
- branch=stable/carbon.
-
-Optional parameters:
-
-**branch**: is the git branch to build from.
-**jjb-version**: is the version of JJB to install in the build minion.
-**tox-dir**: directory containing tox.ini file (default: '')
-**tox-envs**: tox environments to run (default: '')
-
-## Archiving logs in Jobs
-
-There are 2 ways supported for archiving log information:
-
-1) Job creates $WORKSPACE/archives directory and places logs there
-
-This method pushes the entire archives directory to the log server
-in the same structure as configured in the archives directory.
-
-2) Via job variable ARCHIVE_ARTIFACTS using globstar patterns.
-
-In this method a job can define a globstar for example `**/*.log` which then
-causes the archive script to do a globstar search for that pattern and archives
-any files it finds matching.
-
-## Overriding merge and verify triggers
-
-The default trigger conditions for Merge and Verify job types are overrideable
-in a project configuration by overriding the following variables:
-
-- gerrit_merge_triggers
-- gerrit_verify_triggers
-
-These variables take a list of trigger-on values as defined in JJB docs here:
-<https://docs.openstack.org/infra/jenkins-job-builder/triggers.html#triggers.gerrit>
-
-## Appendix
-
-### ShellCheck
-
-When using ShellCheck to lint global-jjb or any projects that include
-global-jjb as part of their project (common with ci-management repos) then
-we require version 0.4.x of ShellCheck installed on the build vms. This version
-introduces annotations used by shell scripts in this repo.
+- Documentation <http://docs.releng.linuxfoundation.org/projects/global-jjb>
--- /dev/null
+- triggers:
+ - github-pull-request:
+ trigger-phrase: ^remerge$
+ only-trigger-phrase: true
+ status-context: JJB Merge
+ permit-all: false
+ github-hooks: true
+ org-list:
+ - '{github-org}'
+ white-list: '{obj:github_pr_whitelist}'
+ admin-list: '{obj:github_pr_admin_list}'
+ included_regions: '{obj:github_included_regions}'
--- /dev/null
+########
+Appendix
+########
+
+ShellCheck
+==========
+
+When using ShellCheck to lint global-jjb or any projects that include
+global-jjb as part of their project (common with ci-management repos) then
+we require version 0.4.x of ShellCheck installed on the build vms. This version
+introduces annotations used by shell scripts in this repo.
2. Run the build scripts in this case lftools-install.sh and logs-deploy.sh
3. Remove credentials provided by config-file-provider
+.. _preserve-variable-refs:
+
Preserving Objects in Variable References
=========================================
.. code-block:: yaml
- - triggers:
- - lf-infra-github-pr-trigger:
- trigger-phrase: ^remerge$
- status-context: JJB Merge
- permit-all: false
- github-hooks: true
- github-org: '{github-org}'
- github_pr_whitelist: '{obj:github_pr_whitelist}'
- github_pr_admin_list: '{obj:github_pr_admin_list}'
-
-In the above example note the use of underscores in `github_pr_admin_list` and
-`github_pr_admin_list`.
+ .. literalinclude:: _static/github-pr-trigger.example
+
+In the above example note the use of underscores in ``github_pr_whitelist``,
+``github_pr_admin_list``, and ``github_included_regions``.
Using single quotes around variables
====================================
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
import os
+import subprocess
import sys
import sphinx_bootstrap_theme
+
sys.path.insert(0, os.path.abspath('..'))
+
+def format_version(version):
+ fmt = '{tag}.dev{commitcount}+{gitsha}'
+ parts = version.split('-')
+ assert len(parts) in (3, 4)
+ dirty = len(parts) == 4
+ tag, count, sha = parts[:3]
+ if count == '0' and not dirty:
+ return tag
+ return fmt.format(tag=tag, commitcount=count, gitsha=sha.lstrip('g'))
+
+
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
# built documents.
#
# The short X.Y version.
-version = '0.19.2'
+command = 'git describe --tags --long --dirty'
+try:
+ git_version = format_version(
+ subprocess.check_output(command.split()).decode('utf-8').strip())
+except subprocess.CalledProcessError: # Handle docs builds from tarball
+ git_version = "v0.0.9999-local"
+version = git_version
# The full version, including alpha/beta/rc tags.
release = version
-.. _global-jjb-configuration:
-
#############
Configuration
#############
+.. _defaults-yaml:
+
+defaults.yaml
+=============
+
+This file lives in the ci-management repo typically under the path
+``jjb/defaults.yaml``. The purpose of this file is to store default variable
+values used by global-jjb templates.
+
+**Required**
+
+:jenkins-ssh-credential: The name of the Jenkins Credential to
+ use for ssh connections. (ex: jenkins-ssh)
+
+:lftools-version: Version of lftools to install. Can be a specific version
+ like '0.6.1' or a `PEP-440 definition <https://www.python.org/dev/peps/pep-0440/>`_
+ For example `<1.0.0` or `>=1.0.0,<2.0.0`.
+
+:mvn-site-id: Maven Server ID from settings.xml containing the credentials
+ to push to a Maven site repository.
+
+:mvn-staging-id: Maven Server ID from settings.xml containing the credentials
+ to push to a Maven staging repository.
+
+**Gerrit required parameters**:
+
+:gerrit-server-name: The name of the Gerrit Server as defined in Gerrit
+ Trigger global configuration. (ex: Primary)
+
+**GitHub required parameters**:
+
+:git-url: Set this to the base URL of your GitHub repo. In
+ general this should be https://github.com. If you are using
+ GitHub Enterprise, or some other GitHub-style system, then it
+ should be whatever your installation base URL is.
+
+:git-clone-url: This is the clone prefix used by GitHub jobs.
+ Set this to either the same thing as **git-url** or the
+ 'git@github.com:' including the trailing ':'
+
+:github-org: The name of the GitHub organization interpolated
+ into the scm config.
+
+:github_pr_org: The name of the GitHub organization. All members
+ of this organization will be able to trigger jobs.
+
+:github_pr_whitelist: List of GitHub members you wish to be able to
+ trigger jobs.
+
+:github_pr_admin_list: List of GitHub members that will have admin
+ privileges on the jobs.
+
+Example Gerrit Infra:
+
+.. code-block:: yaml
+
+ - defaults:
+ name: global
+
+ # lf-infra defaults
+ jenkins-ssh-credential: jenkins-ssh
+ gerrit-server-name: OpenDaylight
+ lftools-version: '<1.0.0'
+ mvn-site-id: opendaylight-site
+ mvn-staging-id: opendaylight-staging
+
+Example GitHub Infra:
+
+.. code-block:: yaml
+
+ - defaults:
+ name: global
+
+ # lf-infra defaults
+ jenkins-ssh-credential: jenkins-ssh
+ github-org: lfit
+ github_pr_whitelist:
+ - jpwku
+ - tykeal
+ - zxiiro
+ github_pr_admin_list:
+ - tykeal
+ lftools-version: '<1.0.0'
+ mvn-site-id: opendaylight-site
+ mvn-staging-id: opendaylight-staging
+
+.. _jenkins-files:
+
Jenkins Files
=============
default configurations for certain tools. This section details the files to
define in Jenkins' **Manage Files** configuration.
+.. _npmrc:
+
npmrc
-----
Create a "Custom file" with contents:
-.. code::
+.. code-block:: ini
registry = https://nexus.opendaylight.org/content/repositories/npmjs/
+.. _pipconf:
+
pipconf
-------
Create a "Custom file" with contents:
-.. code::
+.. code-block:: ini
[global]
timeout = 60
index-url = https://nexus3.opendaylight.org/repository/PyPi/simple
+.. _jjbini:
+
+jjbini
+------
+
+This file contains the Jenkins Job Builder `configuration
+<https://docs.openstack.org/infra/jenkins-job-builder/execution.html#configuration-file>`_
+for :doc:`jjb/lf-ci-jobs`.
+
+:Required: This file MUST exist.
+:type: Custom file
+
+Create a "Custom file" with contents:
+
+.. code-block:: ini
+
+ [job_builder]
+ ignore_cache=True
+ keep_descriptions=False
+ include_path=.:scripts:~/git/
+ recursive=True
+
+ [jenkins]
+ user=jenkins-jobbuilder
+ password=1234567890abcdef1234567890abcdef
+ url=https://jenkins.example.org
+ query_plugins_info=False
+
+ [production]
+ user=jenkins-jobbuilder
+ password=1234567890abcdef1234567890abcdef
+ url=https://jenkins.example.org
+ query_plugins_info=False
+
+ [sandbox]
+ user=jenkins-jobbuilder
+ password=1234567890abcdef1234567890abcdef
+ url=https://jenkins.example.org/sandbox
+ query_plugins_info=False
+
+The last 2 sections are for the ``jenkins-cfg`` job use, they should match the
+``silo`` names for the respective Jenkins systems, typically ``production`` and
+``sandbox``.
+
+.. _jenkins-log-archives-settings:
+
+jenkins-log-archives-settings
+-----------------------------
+
+See :ref:`lf-infra-ship-logs` for usage. If not archiving logs then keep this
+file with default settings, global-jjb needs the file to exist to function.
+
+Requires a *credential* named 'logs' of type 'Username and Password' created in
+the Jenkins Credentials system.
+
+#. Add Server Credentials
+#. Set ``ServerId`` to ``logs``
+#. Set ``Credentials`` to the ``logs`` user created in the Credentials System
+
+:Required: This file MUST exist if using log archiving.
+:type: Maven settings.xml
+
+.. code-block:: xml
+
+ <?xml version="1.0" encoding="UTF-8"?>
+ <settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
+ xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+ xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
+ </settings>
+
+.. note::
+
+ This example is the default boilerplate generated by Jenkins with
+ the comments stripped out. We can also use the default generated by Jenkins
+ without modifying it.
+
+.. _packer-cloud-env:
+
+packer-cloud-env
+----------------
+
+Cloud environment configuration variables for Packer jobs. These can
+contain credentials and configuration for whichever clouds packer jobs
+are using.
+
+:Required: This file MUST exist to use packer jobs.
+:type: Json file
+
+.. code-block:: json
+
+ {
+ "cloud_auth_url": "https://auth.vexxhost.net/v3/",
+ "cloud_tenant": "TENANT_ID",
+ "cloud_user": "CLOUD_USERNAME",
+ "cloud_pass": "CLOUD_PASSWORD",
+ "cloud_network": "CLOUD_NETWORK",
+ "ssh_proxy_host": ""
+ }
+
+.. _jenkins-ci-jobs:
Jenkins CI Jobs
===============
+.. _jenkins-cfg-merge:
+
jenkins-cfg-merge
-----------------
This job manages Jenkins Global configuration. Refer to
the :ref:`CI Documentation <lf-global-jjb-jenkins-cfg-merge>` for job
configuration details.
+
+.. _log-archiving:
+
+Log Archiving
+=============
+
+The logs account requires a Maven Settings file created called
+**jenkins-log-archives-settings** with a server ID of **logs** containing the
+credentials for the logs user in Nexus.
--- /dev/null
+########
+Glossary
+########
+
+.. glossary::
+
+ ciman
+ Short for :term:`ci-management`.
+
+ ci-management
+ Refers to the SCM repository containing the :term:`JJB` configuration
+ files. In most LF Projects this is the repository named
+ ``ci-management``, but is ``releng/builder`` in the OpenDaylight project
+ and ``releng`` in the OPNFV project.
+
+ JJB
+ Short for Jenkins Job Builder (JJB) a tool used to convert YAML
+ definitions into XML as a way to define Jenkins job configuration.
Documentation.
Global-JJB is a library project containing reusable Jenkins Job Builder
-templates. Mainly used by LFCI to deploy management Jenkins jobs to an LF
-managed Jenkins instance, there are other jobs defined for which may be helpful
-to projects whom use the same build technology. The intention is to save time
-for projects from having to define their own job templates.
+templates. Developed for LFCI to deploy management Jenkins jobs to an LF
+managed Jenkins instance, there are other jobs defined which may be helpful
+to projects that use the same build technology. The intention is to help
+projects save time from having to define their own job templates.
Release Notes
-------------
.. toctree::
:maxdepth: 2
- best-practices
+ install
configuration
+ best-practices
+ glossary
+ appendix
+
Global JJB Templates
--------------------
--- /dev/null
+#######
+Install
+#######
+
+global-jjb requires configuration in 2 places; ``Jenkins`` and the
+:term:`ci-management` repository.
+
+.. _jenkins-config:
+
+Jenkins configuration
+=====================
+
+On the Jenkins side, we need to prep ``environment variables`` and
+``plugins`` required by the jobs in global-jjb before we can start our first
+jobs.
+
+.. _jenkins-install-plugins:
+
+Install Jenkins plugins
+-----------------------
+
+Install the following required Jenkins plugins and any optional ones as
+necessary by the project.
+
+**Required**
+
+- `Config File Provider <https://plugins.jenkins.io/config-file-provider>`_
+- `Description Setter <https://plugins.jenkins.io/description-setter>`_
+- `Environment Injector Plugin <https://plugins.jenkins.io/envinject>`_
+- `Git plugin <https://plugins.jenkins.io/git>`_
+- `Post Build Script <https://plugins.jenkins.io/postbuildscript>`_
+- `SSH Agent <https://plugins.jenkins.io/ssh-agent>`_
+- `Workspace Cleanup <https://plugins.jenkins.io/ws-cleanup>`_
+
+**Required for Gerrit connected systems**
+
+- `Gerrit Trigger <https://plugins.jenkins.io/gerrit-trigger>`_
+
+**Required for GitHub connected systems**
+
+- `GitHub plugin <https://plugins.jenkins.io/github>`_
+- `GitHub Pull Request Builder <https://plugins.jenkins.io/ghprb>`_
+
+**Optional**
+
+- `Mask Passwords <https://plugins.jenkins.io/mask-passwords>`_
+- `MsgInject <https://plugins.jenkins.io/msginject>`_
+- `OpenStack Cloud <https://plugins.jenkins.io/openstack-cloud>`_
+- `Timestamper <https://plugins.jenkins.io/timestamper>`_
+
+.. _jenkins-envvars:
+
+Environment Variables
+---------------------
+
+The :ref:`lf-global-jjb-jenkins-cfg-merge` job can manage environment variables
+job but we must first bootstrap them in Jenkins so that the job can run and
+take over.
+
+**Required**::
+
+ GIT_URL=ssh://jenkins-$SILO@git.opendaylight.org:29418
+ JENKINS_HOSTNAME=jenkins092
+ NEXUS_URL=https://nexus.opendaylight.org
+ SILO=production
+ SONAR_URL=https://sonar.opendaylight.org
+
+**Gerrit**::
+
+ GERRIT_URL=https://git.opendaylight.org/gerrit
+
+**GitHub**::
+
+ GIT_URL=https://github.com
+ GIT_CLONE_URL=git@github.com:
+
+.. note::
+
+ Use ``GIT_CLONE_URL`` for GitHub projects as this will be different from the
+ URL used in the properties configuration.
+
+**Optional**::
+
+ LOGS_SERVER=https://logs.opendaylight.org
+
+Steps
+
+#. Navigate to https://jenkins.example.org/configure
+#. Configure the environment variables as described above
+#. Configure the same environment variables in the :term:`ci-management` repo
+
+.. _jenkins-ci-management:
+
+ci-management
+=============
+
+:term:`ci-management` is a git repository containing :term:`JJB` configuration
+files for Jenkins Jobs. Deploying Global JJB here as a submodule allows us easy
+management to install, upgrade, and rollback changes via git tags. Install
+Global JJB as follows:
+
+#. Install Global JJB
+
+ .. code-block:: bash
+
+ GLOBAL_JJB_VERSION=v0.1.0
+ git submodule add https://github.com/lfit/releng-global-jjb.git jjb/global-jjb
+ cd jjb/global-jjb
+ git checkout $GLOBAL_JJB_VERSION
+ cd ../..
+ git add jjb/global-jjb
+ git commit -sm "Install global-jjb $GLOBAL_JJB_VERSION"
+
+ .. note::
+
+ We are purposely using github for production deploys of global-jjb so that
+ uptime of LF Gerrit does not affect projects using global-jjb. In a test
+ environment we can use
+ https://gerrit.linuxfoundation.org/infra/releng/global-jjb if desired.
+
+#. Setup ``jjb/defaults.yaml``
+
+ Create and configure the following parameters in the
+ ``jjb/defaults.yaml`` file as described in the
+ `defaults.yaml configuration docs <defaults-yaml>`.
+
+ Once configured commit the modifications:
+
+ .. code-block:: bash
+
+ git add jjb/defaults.yaml
+ git commit -sm "Setup defaults.yaml"
+
+#. Push patches to Gerrit / GitHub using your favourite push method
+
+At this point global-jjb installation is complete in the :term:`ci-management`
+repo and is ready for use.
+
+.. _deploy-ci-jobs:
+
+Deploy ci-jobs
+==============
+
+The CI job group contains jobs that should deploy in all LF
+Jenkins infra. The minimal configuration to deploy the
+**{project-name}-ci-jobs** job group as defined in **lf-ci-jobs.yaml** is as
+follows:
+
+jjb/ci-management/ci-management.yaml:
+
+.. code-block:: yaml
+
+ - project:
+ name: ci-jobs
+
+ jobs:
+ - '{project-name}-ci-jobs'
+
+ project: ci-management
+ project-name: ci-management
+ build-node: centos7-builder-2c-1g
+
+**Required parameters**:
+
+:project: The project repo as defined in source control.
+:project-name: A custom name to call the job in Jenkins.
+:build-node: The name of the builder to use when building (Jenkins label).
+
+**Optional parameters**:
+
+:branch: The git branch to build from. (default: master)
+:jjb-version: The version of JJB to install in the build minion. (default:
+ <defined by the global-jjb project>)
+
+.. _deploy-packer-jobs:
+
+Deploy packer-jobs
+==================
+
+The packer job group contains jobs to build custom minion images. The minimal
+configuration needed to deploy the packer jobs is as follows which deploys the
+**{project-name}-packer-jobs** job group as defined in **lf-ci-jobs.yaml**.
+
+jjb/ci-management/packer.yaml:
+
+.. code-block:: yaml
+
+ - project:
+ name: packer-builder-jobs
+
+ jobs:
+ - '{project-name}-packer-jobs'
+
+ project: ci-management
+ project-name: ci-management
+ branch: master
+ build-node: centos7-builder-2c-1g
+
+ platforms:
+ - centos
+ - ubuntu-16.04
+
+ templates: builder
+
+ - project:
+ name: packer-docker-jobs
+
+ jobs:
+ - '{project-name}-packer-jobs'
+
+ project: ci-management
+ project-name: ci-management
+ branch: master
+ build-node: centos7-builder-2c-1g
+
+ templates: docker
+
+ platforms:
+ - centos
+ - ubuntu-16.04
+
+**Required parameters**:
+
+:project: The project repo as defined in source control.
+:project-name: A custom name to call the job in Jenkins.
+:build-node: The name of the builder to use when building (Jenkins label).
+:platforms: A list of supported platforms.
+:templates: A list of templates to build. We recommend setting one template per
+ ``project`` section so that we can control which platforms to build for
+ specific templates.
+
+**Optional parameters**:
+
+:branch: The git branch to build from. (default: master)
+:packer-version: The version of packer to install in the build minion,
+ when packer is not available. (default: <defined by global-jjb>)
- github-jjb-merge
- github-jjb-verify
+{project-name}-info-yaml-jobs
+-----------------------------
+
+Jobs to verify INFO.yaml file changes.
+
+:Includes:
+
+ - gerrit-info-yaml-verify
+
+{project-name}-github-info-yaml-jobs
+------------------------------------
+
+Jobs to verify INFO.yaml file changes using Github.
+
+:Includes:
+
+ - github-info-yaml-verify
+
{project-name}-packer-jobs
--------------------------
Macros
======
+lf-infra-jjb-parameters
+-----------------------
+
+:Required Parameters:
+
+ :jjb-version: Version of Jenkins Job Builder (JJB) to install and use in
+ the jjb jobs.
+
+lf-jenkins-cfg-clouds
+---------------------
+
+Deploys Jenkins Cloud configuration read from the ``jenkins-clouds`` directory
+in ci-management repositories.
+
+.. note::
+
+ Requires the jjbini file in Jenkins CFP to contain JJB 2.0 style
+ config definitions for "production" and "sandbox" systems.
+
+:Required Parameters:
+
+ :jenkins-silos: Space-separated list of Jenkins silos to update
+ configuration for as defined in ~/.config/jenkins_jobs/jenkins_jobs.ini
+
lf-jenkins-cfg-global-vars
--------------------------
:Required parameters:
- :jenkins-silos: Space separated list of Jenkins silos to update
+ :jenkins-silos: Space-separated list of Jenkins silos to update
configuration for as defined in ~/.config/jenkins_jobs/jenkins_jobs.ini
lf-infra-jjbini
.. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-merge-full.yaml
:language: yaml
+.. _jenkins-cfg-envvar:
+
Global Environment Variables
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OpenStack Cloud plugin version supported:
-* 2.30
-* 2.31
-* 2.32
+* 2.30 - 2.34
+* 2.35 - 2.37
Cloud configuration are managed via a directory structure in ci-management as
follows:
:RETENTION_TIME: Number of minutes to wait for an idle slave to be used
again before it's removed. If set to -1, the slave will be kept
forever. (default: 0)
+ :CONNECTION_TYPE: The connection type for Jenkins to connect to the build
+ minion. Valid options: JNLP, SSH. (default: "SSH")
For a live example see the OpenDaylight project jenkins-config directory.
https://github.com/opendaylight/releng-builder/tree/master/jenkins-config
:branch: Git branch to fetch for the build. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 10)
+ :build-timeout: Timeout in minutes before aborting build. (default: 10)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
:stream: Keyword that can be used to represent a release code-name.
Often the same as the branch. (default: master)
:branch: Git branch to fetch for the build. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 10)
+ :build-timeout: Timeout in minutes before aborting build. (default: 10)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
:stream: Keyword that can be used to represent a release code-name.
Often the same as the branch. (default: master)
filter which file modifications will trigger a build.
(default defined by lf_jjb_common)
+.. _info-yaml-verify:
+
+Info YAML Verify
+----------------
+
+Info YAML Verify job validates that INFO.yaml file changes are kept isolated from
+other file changes. Verifies INFO.yaml files follow the schema defined in
+`global-jjb/info-schema`.
+
+:Template Names:
+ - {project-name}-info-yaml-verify
+ - gerrit-info-yaml-verify
+ - github-info-yaml-verify
+
+:Required parameters:
+
+ :build-node: The node to run build on.
+ :jenkins-ssh-credential: Credential to use for SSH. (Generally should
+ be configured in defaults.yaml)
+
+:Optional parameters:
+
+ :branch: Git branch to fetch for the build. (default: master)
+ :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
+ :build-timeout: Timeout in minutes before aborting build. (default: 10)
+ :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+ :stream: Keyword that can be used to represent a release code-name.
+ Often the same as the branch. (default: master)
+ :submodule-recursive: Whether to checkout submodules recursively.
+ (default: true)
+ :gerrit_verify_triggers: Override Gerrit Triggers.
+
+.. _license-checker:
+
+License Checker
+---------------
+
+Job to scan projects for files missing license headers.
+
+:Template Names:
+ - {project-name}-license-check
+ - gerrit-license-check
+ - github-license-check
+
+:Optional parameters:
+
+ :build-timeout: Timeout in minutes before aborting build. (default: 15)
+ :file-patterns: Space-separated list of file patterns to scan.
+ (default: \*.go \*.groovy \*.java \*.py \*.sh)
+ :spdx-disable: Disable the SPDX-Identifier checker. (default: false)
+ :lhc-version: Version of LHC to use. (default: 0.2.0)
+ :license-exclude-paths: Comma-separated list of paths to exclude from the
+ license checker. The paths used here will be matched using a contains
+ rule so it is best to be as precise with the path as possible.
+ For example a path of '/src/generated/' will be searched as
+ '**/src/generated/**'.
+ Example: org/opendaylight/yang/gen,protobuff/messages
+ (default: '')
+ :licenses-allowed: Comma-separated list of allowed licenses.
+ (default: Apache-2.0,EPL-1.0,MIT)
.. _gjjb-packer-merge:
:Optional parameters:
+ :cron: Time when the packer image should be rebuilt (default: @monthly)
:branch: Git branch to fetch for the build. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 10)
+ :build-timeout: Timeout in minutes before aborting build. (default: 90)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
:packer-cloud-settings: Name of settings file containing credentials
for the cloud that packer will build on. (default: packer-cloud-env)
:branch: Git branch to fetch for the build. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 10)
+ :build-timeout: Timeout in minutes before aborting build. (default: 10)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
:packer-cloud-settings: Name of settings file containing credentials
for the cloud that packer will build on. (default: packer-cloud-env)
projects in the specified order. The order is determined by first patch
instance for a project in the patch list.
+lf-license-check
+----------------
+
+Checks files for
+
+:Required parameters:
+
+ :file-patterns: Space-separated list of file patterns to scan.
+ For example: \*.go \*.groovy \*.java \*.py \*.sh
+ :spdx-disable: Disable the SPDX-Identifier checker.
+ :lhc-version: Version of LHC to use.
+ :license-exclude-paths: Comma-separated list of paths to exclude from the
+ license checker. The paths used here will be matched using a contains
+ rule so it is best to be as precise with the path as possible.
+ For example a path of '/src/generated/' will be searched as
+ '**/src/generated/**'.
+ Example: org/opendaylight/yang/gen,protobuff/messages
+ :licenses-allowed: Comma-separated list of allowed licenses.
+ For example: Apache-2.0,EPL-1.0,MIT
+
lf-infra-create-netrc
---------------------
Verify gpg signature of the latest commit message in $WORKSPACE.
This command assumes that $WORKSPACE is a git repo.
+lf-infra-package-listing
+------------------------
+
+Lists distro level packages.
+
lf-infra-packer-build
---------------------
server as CLOUDENV environment variable.
:packer-version: Version of packer to use.
+lf-infra-push-gerrit-patch
+--------------------------
+
+Push a change through a Jenkins job to a Gerrit repository in an automated
+way using git-review.
+
+:Required parameters:
+
+ :gerrit-commit-message: Commit message to assign.
+ :gerrit-host: Gerrit hostname.
+ :gerrit-topic: Gerrit topic.
+ :gerrit-user: Gerrit user-id used for submitting the change.
+ :reviewers-email: Reviewers email. Space-separated list of
+ email addresses to CC on the patch.
+ :project: Gerrit project name.
+
+.. _lf-infra-ship-logs:
+
lf-infra-ship-logs
------------------
any macros that calles the
:ref:`lf-provide-maven-settings <lf-provide-maven-settings>` macro.
+.. _provide-sigul-config:
+
+lf-provide-sigul-configuration
+------------------------------
+
+Provides sigual configuration to the Jenkins build node.
+
+Requires the following Config File Provider managed files in Jenkins
+preconfigured.
+
+* sigul-config
+* sigul-password
+* sigul-pki
+
+.. _provide-sigul-config-cleanup:
+
+lf-provide-sigul-configuration-cleanup
+--------------------------------------
+
+Cleanup configuration provided by `provide-sigul-config`.
+
lf-rtd-trigger-build
--------------------
ReadTheDocs verify script.
+lf-sigul-install
+----------------
+
+Install Sigul.
+
+Requires ``SIGUL_BRIDGE_IP`` configured as a global envvar.
+
+lf-infra-provide-docker-cleanup
+-------------------------------
+
+Forcibly removes all of the docker images.
+
Parameters
==========
Provides parameters needed by Maven. Should be used by any jobs that need to
call the mvn cli.
+lf-infra-openstack-parameters
+-----------------------------
+
+Provides parameters needed by OpenStack client CLI. Use in jobs that need to
+call the openstack cli.
+
+:Required Parameters:
+
+ :os-cloud: Configures ``OS_CLOUD`` envvar as used by openstack cli.
+
lf-infra-parameters
-------------------
as the value. This will require that the job be triggered via
the GHPRB plugin and not manually.
-Triggers
-========
-
-lf-infra-github-pr-trigger
---------------------------
-
-Provides configuration for a GitHub PR Trigger.
-
Wrappers
========
Macros
======
+lf-infra-maven-sonar
+--------------------
+
+Runs Sonar against a Maven project.
+
+:Required Parameters:
+
+ :java-version: Version of Java to execute Sonar with.
+ :mvn-version: Version of Maven to execute Sonar with.
+ :mvn-settings: Maven settings.xml file containing credentials to use.
+
lf-maven-build
--------------
Calls the maven stage script to push artifacts to a Nexus staging repository.
+lf-update-java-alternatives
+---------------------------
+
+Setup Java alternatives for the Distro.
+
+:Required Parameters:
+
+ :java-version: Version of Java to set as the default Java.
+ Eg. openjdk8
+
Job Templates
=============
:branch: The branch to build against. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 60)
+ :build-timeout: Timeout in minutes before aborting build. (default: 60)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
:java-version: Version of Java to use for the build. (default: openjdk8)
:mvn-global-settings: The name of the Maven global settings to use for
:branch: The branch to build against. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 60)
+ :build-timeout: Timeout in minutes before aborting build. (default: 60)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
:java-version: Version of Java to use for the build. (default: openjdk8)
:mvn-global-settings: The name of the Maven global settings to use for
:branch: The branch to build against. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 60)
+ :build-timeout: Timeout in minutes before aborting build. (default: 60)
:deploy-path: The path in Nexus to deploy javadoc to. (default: $PROJECT/$STREAM)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
:java-version: Version of Java to use for the build. (default: openjdk8)
:branch: Git branch to fetch for the build. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 60)
+ :build-timeout: Timeout in minutes before aborting build. (default: 60)
:cron: Cron schedule when to trigger the job. This parameter also
supports multiline input via YAML pipe | character in cases where
one may want to provide more than 1 cron timer. (default: 'H H * * 0'
:archive-artifacts: Artifacts to archive to the logs server (default: '').
:branch: The branch to build against. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 60)
+ :build-timeout: Timeout in minutes before aborting build. (default: 60)
:cron: Cron schedule when to trigger the job. This parameter also
supports multiline input via YAML pipe | character in cases where
one may want to provide more than 1 cron timer. (default: '')
:Optional parameters:
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 60)
+ :build-timeout: Timeout in minutes before aborting build. (default: 60)
:cron: Cron schedule when to trigger the job. This parameter also
supports multiline input via YAML pipe | character in cases where
one may want to provide more than 1 cron timer. (default: 'H H * * 6'
:branch: Git branch to fetch for the build. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 60)
+ :build-timeout: Timeout in minutes before aborting build. (default: 60)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
:java-version: Version of Java to use for the build. (default: openjdk8)
:mvn-global-settings: The name of the Maven global settings to use for
:branch: The branch to build against. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 10)
+ :build-timeout: Timeout in minutes before aborting build. (default: 10)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
:node-dir: Path to a NodeJS project to run node test against
(default: '')
Macros
======
+lf-infra-clm-python
+-------------------
+
+Run CLM scanning against a Python project.
+
+:Required Parameters:
+
+ :clm-project-name: Project name in Nexus IQ to send results to.
+
+lf-infra-tox-install
+--------------------
+
+Install Tox into a virtualenv.
+
+:Required Parameters:
+
+ :python-version: Version of Python to install into the Tox virtualenv.
+ Eg. python2 / python3
+
+lf-infra-tox-sonar
+------------------
+
+Runs Sonar scanning against a Python project.
+
+:Required Parameters:
+
+ :java-version: Version of Java to use to run Sonar.
+ :mvn-version: Version of Maven to use to run Sonar.
+
lf-tox-install
--------------
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
:nexus-iq-cli-version: Nexus IQ CLI package version to download and use. (default: 1.44.0-01)
- :build-timeout: Timeout in seconds before aborting build. (default: 60)
+ :build-timeout: Timeout in minutes before aborting build. (default: 60)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
:java-version: Version of Java to use for the build. (default: openjdk8)
:stream: Keyword used to represent a release code-name.
:Optional parameters:
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 60)
+ :build-timeout: Timeout in minutes before aborting build. (default: 60)
:cron: Cron schedule when to trigger the job. This parameter also
supports multiline input via YAML pipe | character in cases where
one may want to provide more than 1 cron timer. (default: H 11 * * *
:branch: The branch to build against. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 10)
+ :build-timeout: Timeout in minutes before aborting build. (default: 10)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
:python-version: Version of Python to configure as a base in virtualenv.
(default: python3)
:branch: Git branch to fetch for the build. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
- :build-timeout: Timeout in seconds before aborting build. (default: 15)
+ :build-timeout: Timeout in minutes before aborting build. (default: 15)
:git-url: base URL of git project. (default: https://github.com)
:project-pattern: Project to trigger build against. (default: \*\*)
:stream: Keyword representing a release code-name.
:branch: Git branch to fetch for the build. (default: master)
:build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
:build-node: The node to run build on.
- :build-timeout: Timeout in seconds before aborting build. (default: 15)
+ :build-timeout: Timeout in minutes before aborting build. (default: 15)
:doc-dir: Directory where tox will place built docs.
as defined in the tox.ini (default: docs/_build/html)
:git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
undocumented_count=0
for file in "${jjb_files[@]}"; do
- docs_interests=$(grep -e '\- builder:' \
+ mapfile -t docs_interests < <(grep -e '\- builder:' \
-e '\- job-group:' \
-e '\- job-template:' \
-e '\- parameter:' \
-e '\- trigger:' \
-e '\- wrapper:' \
-A1 "$file" \
- | grep 'name: ' | awk -F': ' '{print $2}' | awk -F"'" '{print $2}' | sort | uniq)
+ | grep 'name: ' | awk -F': ' '{print $2}' | sort | uniq \
+ | tr -d "'")
for item in "${docs_interests[@]}"; do
if ! grep -q "$item" "docs/${file//.yaml/.rst}"; then
- echo "$item"
+ echo "$file:$item"
let "undocumented_count++"
fi
done
--- /dev/null
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2018 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+---
+$schema: 'http://json-schema.org/schema#'
+$id: 'https://github.com/lfit/releng-global-jjb/blob/master/info-schema'
+
+required:
+ - 'project'
+ - 'project_creation_date'
+ - 'project_category'
+ - 'lifecycle_state'
+ - 'project_lead'
+ - 'primary_contact'
+ - 'issue_tracking'
+ - 'mailing_list'
+ - 'realtime_discussion'
+ - 'repositories'
+ - 'committers'
+ - 'tsc'
+
+properties:
+ project:
+ type: 'string'
+ version:
+ type: 'string'
+ project_creation_date:
+ type: 'string'
+ project_category:
+ type: 'string'
+ lifecycle_state:
+ type: 'string'
+ enum: ['Incubation', 'Proposal', 'Mature', 'Core', 'Top Level', 'Archived', 'Null', 'Integration']
+ project_lead: &user_object
+ type: 'object'
+ properties:
+ name:
+ type: 'string'
+ email:
+ type: 'string'
+ format: 'email'
+ id:
+ type: 'string'
+ company:
+ type: 'string'
+ timezone:
+ type: 'string'
+ pattern: '([A-Z][a-z]*\s*)+\/([A-Za-z_-])+'
+ additionalProperties: false
+ primary_contact: *user_object
+ issue_tracking:
+ type: 'object'
+ properties:
+ type:
+ type: 'string'
+ url:
+ type: 'string'
+ format: 'uri'
+ key:
+ type: 'string'
+ required: ['type', 'url']
+ committers:
+ type: 'array'
+ items: *user_object
--- /dev/null
+The files in this directory are for initializing a Jenkins minion just before
+the cloud plugin hands the minion over to Jenkins.
+
+init.sh is the entry point script which then calls all other scripts.
+
+init.sh also provides a local init script entry point by checking for the
+existance of the file CIMAN_ROOT/jenkins-init-scripts/local-init.sh. If this
+file exists it can be used to further customize the server node before handing
+it to Jenkins for job use.
+
+To use this create a managed file in Jenkins with the contents:
+
+ #!/bin/bash
+ git clone --recurse-submodules https://gerrit.example.org/r/ci-management.git /opt/ciman
+ /opt/ciman/jjb/global-jjb/jenkins-init-scripts/init.sh
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2015 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+
+# Increase limits
+cat <<EOF > /etc/security/limits.d/jenkins.conf
+jenkins soft nofile 16000
+jenkins hard nofile 16000
+EOF
+
+cat <<EOF >/etc/sudoers.d/89-jenkins-user-defaults
+Defaults:jenkins !requiretty
+jenkins ALL = NOPASSWD: ALL
+EOF
+
+cat <<EOSSH >> /etc/ssh/ssh_config
+Host *
+ ServerAliveInterval 60
+ StrictHostKeyChecking no
+ UserKnownHostsFile /dev/null
+EOSSH
+
+cat <<EOKNOWN > /etc/ssh/ssh_known_hosts
+github.com,192.30.253.112 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAq2A7hRGmdnm9tUDbO9IDSwBK6TbQa+PXYPCPy6rbTrTtw7PHkccKrpp0yVhp5HdEIcKr6pLlVDBfOLX9QUsyCOV0wzfjIJNlGEYsdlLJizHhbn2mUjvSAHQqZETYP81eFzLQNnPHt4EVVUh7VfDESU84KezmD5QlWpXLmvU31/yMf+Se8xhHTvKSCZIFImWwoG6mbUoWf9nzpIoaSjB+weqqUUmpaaasXVal72J+UX2B+2RPW3RcT0eOzQgqlJL3RKrTJvdsjE3JEAvGq3lGHSZXy28G3skua2SmVi/w4yCE6gbODqnTWlg7+wC604ydGXA8VJiS5ap43JXiUFFAaQ==
+[140.211.169.26]:29418,[git.opendaylight.org]:29418 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRXyHEw/P1iZr/fFFzbodT5orVV/ftnNRW59Zh9rnSY5Rmbc9aygsZHdtiWBERVVv8atrJSdZool75AglPDDYtPICUGWLR91YBSDcZwReh5S9es1dlQ6fyWTnv9QggSZ98KTQEuE3t/b5SfH0T6tXWmrNydv4J2/mejKRRLU2+oumbeVN1yB+8Uau/3w9/K5F5LgsDDzLkW35djLhPV8r0OfmxV/cAnLl7AaZlaqcJMA+2rGKqM3m3Yu+pQw4pxOfCSpejlAwL6c8tA9naOvBkuJk+hYpg5tDEq2QFGRX5y1F9xQpwpdzZROc5hdGYntM79VMMXTj+95dwVv/8yTsw==
+[gerrit.onap.org]:29418,[198.145.29.92]:29418 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyAKv0UzEhpGKP/rW+yHpngl32Ppr5Uy42coz/sYZYxbtpI+9yaMqfoBb06ktmt6kV7OCT/Sc0OpyWmpcR0d7KZHxx/LE/nm7Gi+xkNHhb9G+Hn6DagP4V+LS6x1YlUt2InLCb8g07+/n6rfxqCI6emIJYu9aTpDhaARb+mMX1xzJuoa4wp59Yr1mkKK8lXHKGnPCemyl9a0vSRY58b7ZWG/N8giNvqYeptslIF1E/MEI5AP6nx7EupiVulAUdboAnDSD0urt9zdE8KRjboghB7PHguil6/OZhbqOb/uEt/rGCHn+02pig1K/vjFvCqNErNgS6EKj0IkH+cU/vjV6j
+[gerrit.opnfv.org]:29418,[198.145.29.81]:29418 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/jsHVV7453mz8D9tQp9t4gDZSYEnt7RTbm9dQCHvrjDxjKRaxCwFkEEF/wHgEm2DkiHLroRvcrJAN6aTH8SdMT4xpOIbD9iDw2ucHWjm3pJ0y4KlNcMnpg9uEWArwhR+pDWgxRCU77eCbCwv1ZEdqMfSgmYdO+MudNZXrldbgFAvsO1HbpwP/naCmUuYDfxlp3UCau19wR8BTKYGnROmQQXB5fSmNW7zrPsAdf7+rzktg1jp9JF7ss34T+gmxEigaC1WrpWRRlIsVTMHH3a9efcgJBS8sAcGRYMg5JRCArPP5u0dg6dXNqk8Zbd0CRpF72A9xVINRf7JZdea2yD+L
+EOKNOWN
+
+# To handle the prompt style that is expected all over the environment
+# with how use use robotframework we need to make sure that it is
+# consistent for any of the users that are created during dynamic spin
+# ups
+echo 'PS1="[\u@\h \W]> "' >> /etc/skel/.bashrc
+
+# vim: sw=2 ts=2 sts=2 et :
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2016 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+
+OS=$(facter operatingsystem | tr '[:upper:]' '[:lower:]')
+
+useradd -m -s /bin/bash jenkins
+
+if grep -q docker /etc/group; then
+ usermod -a -G docker jenkins
+fi
+
+# Used for building RPMs
+if grep -q mock /etc/group; then
+ usermod -a -G mock jenkins
+fi
+
+mkdir /home/jenkins/.ssh /w
+cp -r "/home/${OS}/.ssh/authorized_keys" /home/jenkins/.ssh/authorized_keys
+
+# Generate ssh key for use by Robot jobs
+echo -e 'y\n' | ssh-keygen -N "" -f /home/jenkins/.ssh/id_rsa -t rsa
+chown -R jenkins:jenkins /home/jenkins/.ssh /w
+chmod 700 /home/jenkins/.ssh
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2018 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+
+dd if=/dev/zero of=/swap count=1024 bs=1MiB
+chmod 600 /swap
+mkswap /swap
+swapon /swap
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2015 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+
+OS=$(facter operatingsystem)
+
+case "$OS" in
+ Fedora)
+ systemctl stop firewalld
+ ;;
+ CentOS|RedHat)
+ if [ "$(facter operatingsystemrelease | cut -d '.' -f1)" -lt "7" ]; then
+ service iptables stop
+ else
+ systemctl stop firewalld
+ fi
+ ;;
+ *)
+ # nothing to do
+ ;;
+esac
+
+# vim: ts=4 ts=4 sts=4 et :
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2016 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+
+INIT_SCRIPTS_DIR="/opt/ciman/jjb/global-jjb/jenkins-init-scripts"
+
+"$INIT_SCRIPTS_DIR/package-listing.sh"
+"$INIT_SCRIPTS_DIR/basic-settings.sh"
+"$INIT_SCRIPTS_DIR/disable-firewall.sh"
+"$INIT_SCRIPTS_DIR/create-swap-file.sh"
+
+# Entry point for additional local minion customization
+# Eg. OpenDaylight has additional bootstrap scripts depending on minion type.
+if [ -f "/opt/ciman/jenkins-init-scripts/local-init.sh" ]; then
+ /opt/ciman/jenkins-init-scripts/local-init.sh
+fi
+
+# Create the jenkins user last so that hopefully we DO NOT have to deal with
+# guard files
+"$INIT_SCRIPTS_DIR/create-jenkins-user.sh"
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2018 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+
+# Share script with JJB jobs so we only have to maintain it in one place
+JJB_SHELL_DIR="$(dirname $0)/../shell"
+
+# Make sure the script is executable and then run it
+chmod +x "${JJB_SHELL_DIR}/package-listing.sh"
+"${JJB_SHELL_DIR}/package-listing.sh"
- github-jjb-merge
- github-jjb-verify
+- job-group:
+ name: '{project-name}-info-yaml-jobs'
+
+ jobs:
+ - gerrit-info-yaml-verify
+
+- job-group:
+ name: '{project-name}-github-info-yaml-jobs'
+
+ jobs:
+ - github-info-yaml-verify
+
- job-group:
name: '{project-name}-packer-jobs'
- gerrit-packer-merge
- gerrit-packer-verify
-
- job-group:
name: '{project-name}-github-packer-jobs'
- file-id: jjbini-sandbox
target: '$HOME/.config/jenkins_jobs/jenkins_jobs.ini'
-- lf_packer_file_paths: &lf_packer_file_paths
- name: lf-packer-file-paths
- file-paths:
- - compare-type: ANT
- pattern: 'packer/vars/{platforms}.json'
- - compare-type: ANT
- pattern: 'packer/templates/{templates}.json'
- - compare-type: ANT
- pattern: 'packer/provision/*.sh'
- - compare-type: ANT
- pattern: 'packer/provision/{templates}.yaml'
- - compare-type: ANT
- pattern: 'packer/provision/local-{templates}.yaml'
- - compare-type: ANT
- pattern: 'packer/provision/lib/**'
-
- lf_packer: &lf_packer_common
name: lf-packer-common
# Provide configuration common in packer jobs.
branch: master
build-timeout: 90
+ cron: '@monthly'
stream: master
submodule-recursive: true
packer-cloud-settings: packer-cloud-env
- packer-version: 1.2.2
+ packer-version: 1.2.3
#####################
# Job Configuration #
submodule-recursive: true
gerrit_trigger_file_paths:
- - compare-type: ANT
- pattern: '**/*.sh'
- - compare-type: ANT
- pattern: '**/*.yaml'
- - compare-type: ANT
- pattern: '**/global-jjb'
+ - compare-type: REG_EXP
+ pattern: '.*\.groovy'
+ - compare-type: REG_EXP
+ pattern: '.*\.sh'
+ - compare-type: REG_EXP
+ pattern: '.*\.bat'
+ - compare-type: REG_EXP
+ pattern: '.*\.ps1'
+ - compare-type: REG_EXP
+ pattern: '.*\.(yaml|yml)'
+ - compare-type: REG_EXP
+ pattern: '.*\/global-jjb'
+
+ # github_included_regions MUST match gerrit_trigger_file_paths
+ github_included_regions:
+ - '.*\.groovy'
+ - '.*\.sh'
+ - '.*\.bat'
+ - '.*\.ps1'
+ - '.*\.(yaml|yml)'
+ - '.*\/global-jjb'
#####################
# Job Configuration #
comment-contains-value: remerge$
gerrit_trigger_file_paths:
- - compare-type: ANT
- pattern: 'jenkins-config/**'
+ - compare-type: REG_EXP
+ pattern: 'jenkins-config\/.*'
+
+ # github_included_regions MUST match gerrit_trigger_file_paths
+ github_included_regions:
+ - 'jenkins-config\/.*'
#####################
# Job Configuration #
- github
- pollscm:
cron: ''
- - lf-infra-github-pr-trigger:
+ - github-pull-request:
trigger-phrase: '^remerge$'
only-trigger-phrase: true
status-context: 'Jenkins CFG Merge'
permit-all: false
github-hooks: true
- github-org: '{github-org}'
- github_pr_whitelist: '{obj:github_pr_whitelist}'
- github_pr_admin_list: '{obj:github_pr_admin_list}'
-
+ org-list:
+ - '{github-org}'
+ white-list: '{obj:github_pr_whitelist}'
+ admin-list: '{obj:github_pr_admin_list}'
+ included-regions: '{obj:github_included_regions}'
##################
# JJB DEPLOY JOB #
######################
branch: master
- git-url: '$GIT_URL/$GERRIT_PROJECT'
+ # jjb-deploy should always clone $PROJECT regardless of what
+ # $GERRIT_PROJECT triggered the job.
+ git-url: '$GIT_URL/$PROJECT'
stream: master
gerrit_jjb_deploy_job_triggers:
- comment-added-contains-event:
trigger-on: '{obj:gerrit_jjb_deploy_job_triggers}'
projects:
- project-compare-type: ANT
- project-pattern: '{project}'
+ project-pattern: '**'
branches:
- branch-compare-type: ANT
- branch-pattern: '**/{branch}'
+ branch-pattern: '**'
skip-vote:
success: true
failed: true
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- - lf-infra-github-pr-trigger:
+ - github-pull-request:
trigger-phrase: '^jjb-deploy (?!\*+$).+$'
only-trigger-phrase: true
status-context: 'JJB Deploy Job'
permit-all: true
github-hooks: true
- github-org: ''
- github_pr_whitelist:
- - ''
- github_pr_admin_list:
- - ''
#############
# JJB Merge #
- shell: !include-raw-escape:
- ../shell/jjb-install.sh
- ../shell/jjb-merge-job.sh
+ - ../shell/jjb-cleanup.sh
- job-template:
name: '{project-name}-jjb-merge'
# yamllint disable-line rule:key-duplicates
<<: *lf_jjb_merge
+ #####################
+ # Job Configuration #
+ #####################
+
properties:
- github:
url: '{git-url}/{github-org}/{project}'
- github
- pollscm:
cron: ''
- - lf-infra-github-pr-trigger:
+ - github-pull-request:
trigger-phrase: '^remerge$'
only-trigger-phrase: true
status-context: 'JJB Merge'
permit-all: false
github-hooks: true
- github-org: '{github-org}'
- github_pr_whitelist: '{obj:github_pr_whitelist}'
- github_pr_admin_list: '{obj:github_pr_admin_list}'
+ org-list:
+ - '{github-org}'
+ white-list: '{obj:github_pr_whitelist}'
+ admin-list: '{obj:github_pr_admin_list}'
+ included-regions: '{obj:github_included_regions}'
##############
# JJB Verify #
exclude-no-code-change: false
- draft-published-event
- comment-added-contains-event:
- comment-contains-value: recheck$
+ comment-contains-value: '^Patch Set[ ]+[0-9]+:([ ]+|[\n]+)(recheck|reverify)$'
#####################
# Job Configuration #
- ../shell/jjb-install.sh
- ../shell/jjb-verify-job.sh
- ../shell/jjb-check-unicode.sh
+ - ../shell/jjb-cleanup.sh
- lf-infra-gpg-verify-git-signature
- job-template:
# yamllint disable-line rule:key-duplicates
<<: *lf_jjb_verify
+ ######################
+ # Default parameters #
+ ######################
+
git-url: '$GIT_URL/$GERRIT_PROJECT'
+ #####################
+ # Job Configuration #
+ #####################
+
scm:
- lf-infra-gerrit-scm:
git-url: '{git-url}'
# yamllint disable-line rule:key-duplicates
<<: *lf_jjb_verify
+ #####################
+ # Job Configuration #
+ #####################
+
properties:
- github:
url: '{git-url}/{github-org}/{project}'
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- - lf-infra-github-pr-trigger:
- trigger-phrase: '^recheck$'
+ - github-pull-request:
+ trigger-phrase: '^(recheck|reverify)$'
only-trigger-phrase: false
status-context: 'JJB Verify'
permit-all: true
github-hooks: true
- github-org: ''
- github_pr_whitelist:
- - ''
- github_pr_admin_list:
- - ''
+ included-regions: '{obj:github_included_regions}'
-################
-# Packer Merge #
-################
+#################
+# License Check #
+#################
-- lf_packer_merge: &lf_packer_merge
- name: lf-packer-merge
+- lf_license_check: &lf_license_check
+ name: lf-license-check
######################
# Default parameters #
######################
- gerrit_merge_triggers:
- - change-merged-event
+ branch: master
+ build-days-to-keep: 7
+ build-timeout: 15
+ file-patterns: >
+ *.go
+ *.groovy
+ *.java
+ *.py
+ *.sh
+ git-url: '$GIT_URL/$PROJECT'
+ lhc-version: 0.2.0
+ license-exclude-paths: ''
+ licenses-allowed: Apache-2.0,EPL-1.0,MIT
+ spdx-disable: false
+ stream: master
+
+ #####################
+ # Job Configuration #
+ #####################
+
+ project-type: freestyle
+ node: '{build-node}'
+ concurrent: true
+
+ properties:
+ - lf-infra-properties:
+ build-days-to-keep: '{build-days-to-keep}'
+
+ parameters:
+ - lf-infra-parameters:
+ project: '{project}'
+ branch: '{branch}'
+ stream: '{stream}'
+ lftools-version: '{lftools-version}'
+
+ wrappers:
+ - lf-infra-wrappers:
+ build-timeout: '{build-timeout}'
+ jenkins-ssh-credential: '{jenkins-ssh-credential}'
+
+ builders:
+ - lf-license-check:
+ file-patterns: '{file-patterns}'
+ spdx-disable: '{spdx-disable}'
+ lhc-version: '{lhc-version}'
+ license-exclude-paths: '{license-exclude-paths}'
+ licenses-allowed: '{licenses-allowed}'
+
+ publishers:
+ - lf-infra-publish
+
+- job-template:
+ name: '{project-name}-license-check'
+ id: gerrit-license-check
+ <<: *lf_license_check
+
+ git-url: '$GIT_URL/$GERRIT_PROJECT'
+
+ scm:
+ - lf-infra-gerrit-scm:
+ git-url: '{git-url}'
+ refspec: '$GERRIT_REFSPEC'
+ branch: '$GERRIT_BRANCH'
+ # Submodules are out of the project's control
+ submodule-recursive: false
+ choosing-strategy: gerrit
+ jenkins-ssh-credential: '{jenkins-ssh-credential}'
+
+ triggers:
+ - gerrit:
+ server-name: '{gerrit-server-name}'
+ # Trigger should not be overridable as we want to always run
+ trigger-on:
+ - patchset-created-event:
+ exclude-drafts: false
+ exclude-trivial-rebase: false
+ exclude-no-code-change: false
+ - draft-published-event
+ - comment-added-contains-event:
+ comment-contains-value: recheck$
+ projects:
+ - project-compare-type: ANT
+ project-pattern: '**'
+ branches:
+ - branch-compare-type: ANT
+ branch-pattern: '**'
+
+- job-template:
+ name: '{project-name}-license-check'
+ id: github-license-check
+ <<: *lf_license_check
+
+ properties:
+ - github:
+ url: '{git-url}/{github-org}/{project}'
+
+ scm:
+ - lf-infra-github-scm:
+ url: '{git-clone-url}{github-org}/{project}'
+ refspec: '+refs/pull/*:refs/remotes/origin/pr/*'
+ branch: '$sha1'
+ # Submodules are out of the project's control
+ submodule-recursive: false
+ choosing-strategy: default
+ jenkins-ssh-credential: '{jenkins-ssh-credential}'
+
+ triggers:
+ - github-pull-request:
+ trigger-phrase: '^recheck$'
+ only-trigger-phrase: false
+ status-context: 'License Check'
+ permit-all: true
+ github-hooks: true
+
+####################
+# Info YAML Verify #
+####################
+
+- lf_info_yaml_verify: &lf_info_yaml_verify
+ name: lf-info-yaml-verify
+
+ ######################
+ # Default parameters #
+ ######################
+
+ gerrit_verify_triggers:
+ - patchset-created-event:
+ exclude-drafts: true
+ exclude-trivial-rebase: false
+ exclude-no-code-change: false
+ - draft-published-event
- comment-added-contains-event:
- comment-contains-value: remerge$
+ comment-contains-value: '^Patch Set[ ]+[0-9]+:([ ]+|[\n]+)(recheck|reverify)$'
+
+ #####################
+ # Job Configuration #
+ #####################
+
+ concurrent: true
+
+ builders:
+ - shell: !include-raw-escape:
+ - ../shell/git-validate-info-yaml.sh
+ - ../shell/info-file-validate.sh
+
+- job-template:
+ name: '{project-name}-info-yaml-verify'
+ id: gerrit-info-yaml-verify
+ <<: *lf_jjb_common
+ # yamllint disable-line rule:key-duplicates
+ <<: *lf_info_yaml_verify
+
+ git-url: '$GIT_URL/$GERRIT_PROJECT'
+
+ scm:
+ - lf-infra-gerrit-scm:
+ git-url: '{git-url}'
+ refspec: '$GERRIT_REFSPEC'
+ branch: '$GERRIT_BRANCH'
+ submodule-recursive: '{submodule-recursive}'
+ choosing-strategy: gerrit
+ jenkins-ssh-credential: '{jenkins-ssh-credential}'
+
+ triggers:
+ - gerrit:
+ server-name: '{gerrit-server-name}'
+ trigger-on: '{obj:gerrit_verify_triggers}'
+ projects:
+ - project-compare-type: ANT
+ project-pattern: '{project}'
+ branches:
+ - branch-compare-type: ANT
+ branch-pattern: '**/{branch}'
+ file-paths:
+ - compare-type: REG_EXP
+ pattern: 'INFO.yaml'
+
+- job-template:
+ name: '{project-name}-info-yaml-verify'
+ id: github-info-yaml-verify
+ <<: *lf_jjb_common
+ # yamllint disable-line rule:key-duplicates
+ <<: *lf_info_yaml_verify
+
+ properties:
+ - github:
+ url: '{git-url}/{github-org}/{project}'
+
+ scm:
+ - lf-infra-github-scm:
+ url: '{git-clone-url}{github-org}/{project}'
+ refspec: '+refs/pull/*:refs/remotes/origin/pr/*'
+ branch: '$sha1'
+ submodule-recursive: '{submodule-recursive}'
+ choosing-strategy: default
+ jenkins-ssh-credential: '{jenkins-ssh-credential}'
+
+ triggers:
+ - github-pull-request:
+ trigger-phrase: '^(recheck|reverify)$'
+ only-trigger-phrase: false
+ status-context: 'INFO File Verify'
+ permit-all: true
+ github-hooks: true
+ # included-regions MUST match gerrit file-paths
+ included-regions: 'INFO.yaml'
+
+################
+# Packer Merge #
+################
+
+- lf_packer_merge: &lf_packer_merge
+ name: lf-packer-merge
#####################
# Job Configuration #
# yamllint disable-line rule:key-duplicates
<<: *lf_packer_merge
+ ######################
+ # Default parameters #
+ ######################
+
git-url: '$GIT_URL/$GERRIT_PROJECT'
+ gerrit_merge_triggers:
+ - change-merged-event
+ - comment-added-contains-event:
+ comment-contains-value: remerge$
+
+ #####################
+ # Job Configuration #
+ #####################
+
scm:
- lf-infra-gerrit-scm:
git-url: '{git-url}'
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- - timed: '00 10 1 * *'
+ - timed: '{cron}'
- gerrit:
server-name: '{gerrit-server-name}'
trigger-on: '{obj:gerrit_merge_triggers}'
branches:
- branch-compare-type: ANT
branch-pattern: '**/{branch}'
- <<: *lf_packer_file_paths
+ file-paths:
+ - compare-type: REG_EXP
+ pattern: 'packer\/provision\/.*\.sh'
+ - compare-type: REG_EXP
+ pattern: 'packer\/provision\/.*\.bat'
+ - compare-type: REG_EXP
+ pattern: 'packer\/provision\/.*\.ps1'
+ - compare-type: REG_EXP
+ pattern: 'packer\/provision\/{templates}\.yaml'
+ - compare-type: REG_EXP
+ pattern: 'packer\/provision\/local-{templates}\.yaml'
+ - compare-type: REG_EXP
+ pattern: 'packer\/templates\/{templates}\.json'
+ - compare-type: REG_EXP
+ pattern: 'packer\/vars\/{platforms}\.json'
- job-template:
name: '{project-name}-packer-merge-{platforms}-{templates}'
# yamllint disable-line rule:key-duplicates
<<: *lf_packer_merge
+ #####################
+ # Job Configuration #
+ #####################
+
properties:
- github:
url: '{git-url}/{github-org}/{project}'
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- - timed: '00 10 1 * *'
+ - timed: '{cron}'
- github
- pollscm:
cron: ''
- - lf-infra-github-pr-trigger:
+
+ - github-pull-request:
trigger-phrase: '^remerge$'
only-trigger-phrase: true
- status-context: 'Packer {platforms}-${templates} Merge'
+ status-context: 'Packer {platforms}-{templates} Merge'
permit-all: false
github-hooks: true
- github-org: '{github-org}'
- github_pr_whitelist: '{obj:github_pr_whitelist}'
- github_pr_admin_list: '{obj:github_pr_admin_list}'
+ org-list:
+ - '{github-org}'
+ white-list: '{obj:github_pr_whitelist}'
+ admin-list: '{obj:github_pr_admin_list}'
+ # included-regions MUST match Gerrit Trigger file-paths
+ included-regions:
+ - 'packer\/provision\/.*\.sh'
+ - 'packer\/provision\/.*\.bat'
+ - 'packer\/provision\/.*\.ps1'
+ - 'packer\/provision\/{templates}\.yaml'
+ - 'packer\/provision\/local-{templates}\.yaml'
+ - 'packer\/templates\/{templates}\.json'
+ - 'packer\/vars\/{platforms}\.json'
#################
# Packer Verify #
exclude-no-code-change: false
- draft-published-event
- comment-added-contains-event:
- comment-contains-value: recheck$
+ comment-contains-value: '^Patch Set[ ]+[0-9]+:([ ]+|[\n]+)(recheck|reverify)$'
gerrit_trigger_file_paths:
- - compare-type: ANT
- pattern: 'packer/vars/**.json'
- - compare-type: ANT
- pattern: 'packer/templates/**.json'
- - compare-type: ANT
- pattern: 'packer/provision/**.sh'
- - compare-type: ANT
- pattern: 'packer/provision/lib/**'
+ - compare-type: REG_EXP
+ pattern: 'packer\/.*'
+
+ # github_included_regions MUST match gerrit_trigger_file_paths
+ github_included_regions:
+ - 'packer\/.*'
#####################
# Job Configuration #
# yamllint disable-line rule:key-duplicates
<<: *lf_packer_verify
+ ######################
+ # Default parameters #
+ ######################
+
git-url: '$GIT_URL/$GERRIT_PROJECT'
+ #####################
+ # Job Configuration #
+ #####################
+
scm:
- lf-infra-gerrit-scm:
git-url: '{git-url}'
# yamllint disable-line rule:key-duplicates
<<: *lf_packer_verify
+ #####################
+ # Job Configuration #
+ #####################
+
properties:
- github:
url: '{git-url}/{github-org}/{project}'
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- - lf-infra-github-pr-trigger:
- trigger-phrase: '^recheck$'
+ - github-pull-request:
+ trigger-phrase: '^(recheck|reverify)$'
only-trigger-phrase: false
status-context: 'Packer Verify'
permit-all: true
github-hooks: true
- github-org: ''
- github_pr_whitelist:
- - ''
- github_pr_admin_list:
- - ''
+ included-regions: '{obj:github_included_regions}'
- inject:
properties-file: $WORKSPACE/.dependency.properties
+- builder:
+ name: lf-license-check
+ builders:
+ - inject:
+ properties-content: |
+ FILE_PATTERNS={file-patterns}
+ SPDX_DISABLE={spdx-disable}
+ LHC_VERSION={lhc-version}
+ LICENSE_EXCLUDE_PATHS={license-exclude-paths}
+ LICENSES_ALLOWED={licenses-allowed}
+ - shell: !include-raw-escape:
+ - ../shell/license-check.sh
+
- builder:
name: lf-infra-create-netrc
builders:
- description-setter:
regexp: '^Build logs: .*'
+- builder:
+ name: lf-infra-package-listing
+ builders:
+ - shell: !include-raw:
+ - ../shell/package-listing.sh
+
- builder:
name: lf-infra-packer-build
builders:
builders:
- inject:
properties-content: |
- PROJECT='{project}'
- GERRIT_COMMIT_MESSAGE='{gerrit-commit-message}'
- GERRIT_HOST='{gerrit-host}'
- GERRIT_TOPIC='{gerrit-topic}'
- GERRIT_USER='{gerrit-user}'
- REVIEWERS_EMAIL='{reviewers-email}'
+ PROJECT={project}
+ GERRIT_COMMIT_MESSAGE={gerrit-commit-message}
+ GERRIT_HOST={gerrit-host}
+ GERRIT_TOPIC={gerrit-topic}
+ GERRIT_USER={gerrit-user}
+ REVIEWERS_EMAIL={reviewers-email}
- shell: !include-raw-escape: ../shell/gerrit-push-patch.sh
- builder:
builder:
- shell: !include-raw-escape: ../shell/sigul-install.sh
+- builder:
+ name: lf-infra-provide-docker-cleanup
+ builders:
+ - shell: |
+ #!/bin/bash
+ set +e # DO NOT cause build failure if docker rmi fails
+ docker rmi -f $(docker images -a -q)
+ exit 0
+
+
##############
# PARAMETERS #
##############
name: PROJECT
default: '{project}'
description: |
- Parameter to identify a Gerrit project. This is typically the
- project repo path as exists in Gerrit.
- For example: ofextensions/circuitsw
+ Parameter to identify a SCM project to build. This is typically
+ the project repo path. For example: ofextensions/circuitsw
- string:
name: STREAM
default: '{stream}'
name: GERRIT_PROJECT
default: '{project}'
description: |
- Parameter to identify Gerrit project. This is typically the
- project repo path as exists in Gerrit.
- For example: ofextensions/circuitsw
+ Gerrit Trigger provided parameter to identify Gerrit project that
+ triggered the build. This is typically the project repo path as
+ exists in Gerrit. For example: ofextensions/circuitsw
- Note that Gerrit will override this parameter automatically if a
- job is triggered by Gerrit.
+ If using Gerrit, in a manual build this should match the PROJECT
+ parameter above.
- string:
name: GERRIT_BRANCH
default: '{branch}'
description: |
- Parameter to identify a Gerrit branch.
+ Gerrit Trigger provided parameter to identify a Gerrit branch.
- Note that Gerrit will override this parameter automatically if a
- job is triggered by Gerrit.
+ If using Gerrit, in a manual build override with the branch to
+ build against.
- string:
name: GERRIT_REFSPEC
default: 'refs/heads/{branch}'
description: |
- Parameter to identify a refspec when pulling from Gerrit.
+ Gerrit Trigger provided parameter to identify a refspec to fetch
+ from Gerrit.
+
+ If using Gerrit, in a manual build override with a refspec.
+ https://git-scm.com/book/en/v2/Git-Internals-The-Refspec
+ For example: 'refs/heads/master'
+ - string:
+ name: sha1
+ default: 'origin/{branch}'
+ description: |
+ GitHub PR Trigger provided parameter for specifying the commit
+ to checkout.
- Note that Gerrit will override this parameter automatically if a
- job is triggered by Gerrit.
+ If using GitHub, in a manual build override with a branch path or
+ sha1 hash to a specific commit. For example: 'origin/master'
+ # Tools
- string:
name: LFTOOLS_VERSION
default: '{lftools-version}'
- UNSTABLE
build-steps:
- lf-infra-sysstat
+ - lf-infra-package-listing
- lf-infra-ship-logs
mark-unstable-if-failed: true
- workspace-cleanup:
recursive: '{submodule-recursive}'
choosing-strategy: '{choosing-strategy}'
-############
-# TRIGGERS #
-############
-
-- trigger:
- name: lf-infra-github-pr-trigger
- triggers:
- - github-pull-request:
- trigger-phrase: '{trigger-phrase}'
- only-trigger-phrase: '{only-trigger-phrase}'
- status-context: '{status-context}'
- permit-all: '{permit-all}'
- github-hooks: '{github-hooks}'
- auto-close-on-fail: false
- org-list:
- - '{github-org}'
- white-list: '{obj:github_pr_whitelist}'
- admin-list: '{obj:github_pr_admin_list}'
-
############
# WRAPPERS #
############
# Job Configuration #
#####################
- triggers:
- # Build weekly on Saturdays
- - timed: 'H H * * 6'
- - gerrit:
- server-name: '{gerrit-server-name}'
- trigger-on: '{obj:gerrit_clm_triggers}'
- projects:
- - project-compare-type: ANT
- project-pattern: '{project}'
- branches:
- - branch-compare-type: ANT
- branch-pattern: '**/{branch}'
- skip-vote:
- successful: true
- failed: true
- unstable: true
- notbuilt: true
-
builders:
- lf-maven-install:
mvn-version: '{mvn-version}'
submodule-recursive: '{submodule-recursive}'
choosing-strategy: default
+ triggers:
+ # Build weekly on Saturdays
+ - timed: 'H H * * 6'
+ - gerrit:
+ server-name: '{gerrit-server-name}'
+ trigger-on: '{obj:gerrit_clm_triggers}'
+ projects:
+ - project-compare-type: ANT
+ project-pattern: '{project}'
+ branches:
+ - branch-compare-type: ANT
+ branch-pattern: '**/{branch}'
+ skip-vote:
+ successful: true
+ failed: true
+ unstable: true
+ notbuilt: true
+
- job-template:
name: '{project-name}-maven-clm-{stream}'
id: github-maven-clm
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- - lf-infra-github-pr-trigger:
+ # Build weekly on Saturdays
+ - timed: 'H H * * 6'
+ - github-pull-request:
trigger-phrase: '^run-clm$'
only-trigger-phrase: false
status-context: 'CLM'
permit-all: true
github-hooks: true
- github-org: ''
- github_pr_whitelist:
- - ''
- github_pr_admin_list:
- - ''
+ org-list:
+ - '{github-org}'
+ white-list: '{obj:github_pr_whitelist}'
+ admin-list: '{obj:github_pr_admin_list}'
#########################
# Maven Javadoc Publish #
- github
- pollscm:
cron: ''
- - lf-infra-github-pr-trigger:
+ - github-pull-request:
trigger-phrase: '^remerge$'
only-trigger-phrase: true
- status-context: 'JJB Merge'
+ status-context: 'Maven Javadoc Publish'
permit-all: false
github-hooks: true
- github-org: '{github-org}'
- github_pr_whitelist: '{obj:github_pr_whitelist}'
- github_pr_admin_list: '{obj:github_pr_admin_list}'
+ org-list:
+ - '{github-org}'
+ white-list: '{obj:github_pr_whitelist}'
+ admin-list: '{obj:github_pr_admin_list}'
########################
# Maven Javadoc Verify #
exclude-no-code-change: false
- draft-published-event
- comment-added-contains-event:
- comment-contains-value: recheck$
+ comment-contains-value: '^Patch Set[ ]+[0-9]+:([ ]+|[\n]+)(recheck|reverify)$'
#####################
# Job Configuration #
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- - lf-infra-github-pr-trigger:
- trigger-phrase: '^recheck$'
- only-trigger-phrase: true
- status-context: 'Maven Javadoc'
+ - github-pull-request:
+ trigger-phrase: ^(recheck|reverify)$
+ only-trigger-phrase: false
+ status-context: 'Maven Javadoc Verify'
permit-all: true
github-hooks: true
- github-org: ''
- github_pr_whitelist:
- - ''
- github_pr_admin_list:
- - ''
###############
# Maven Merge #
comment-contains-value: remerge$
gerrit_trigger_file_paths:
- - compare-type: ANT
- pattern: '**'
+ - compare-type: REG_EXP
+ pattern: '.*'
+
+ # github_included_regions MUST match gerrit_trigger_file_paths
+ github_included_regions:
+ - '.*'
post_build_trigger: ''
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- - lf-infra-github-pr-trigger:
+ - github
+ - pollscm:
+ cron: ''
+ - github-pull-request:
trigger-phrase: '^remerge$'
- only-trigger-phrase: false
+ only-trigger-phrase: true
status-context: 'Maven Merge'
- permit-all: true
+ permit-all: false
github-hooks: true
- github-org: ''
- github_pr_whitelist:
- - ''
- github_pr_admin_list:
- - ''
+ org-list:
+ - '{github-org}'
+ white-list: '{obj:github_pr_whitelist}'
+ admin-list: '{obj:github_pr_admin_list}'
+ included-regions: '{obj:github_included_regions}'
#################
# Maven Release #
triggers:
- timed: '{obj:cron}'
- - lf-infra-github-pr-trigger:
+ - github-pull-request:
trigger-phrase: '^build release$'
only-trigger-phrase: true
status-context: 'Maven Release'
permit-all: true
github-hooks: true
- github-org: ''
- github_pr_whitelist:
- - ''
- github_pr_admin_list:
- - ''
###############
# Maven Sonar #
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- - lf-infra-github-pr-trigger:
- trigger-phrase: '^recheck$'
+ - github-pull-request:
+ trigger-phrase: '^run-sonar$'
only-trigger-phrase: false
- status-context: 'Maven Verify'
+ status-context: 'Maven Sonar'
permit-all: true
github-hooks: true
- github-org: ''
- github_pr_whitelist:
- - ''
- github_pr_admin_list:
- - ''
################
# Maven Verify #
exclude-no-code-change: false
- draft-published-event
- comment-added-contains-event:
- comment-contains-value: recheck$
+ comment-contains-value: '^Patch Set[ ]+[0-9]+:([ ]+|[\n]+)(recheck|reverify)$'
+
gerrit_trigger_file_paths:
- - compare-type: ANT
- pattern: '**'
+ - compare-type: REG_EXP
+ pattern: '.*'
+
+ # github_included_regions MUST match gerrit_trigger_file_paths
+ github_included_regions:
+ - '.*'
#####################
# Job Configuration #
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- - lf-infra-github-pr-trigger:
- trigger-phrase: '^recheck$'
+ - github-pull-request:
+ trigger-phrase: '^(recheck|reverify)$'
only-trigger-phrase: false
status-context: 'Maven Verify'
permit-all: true
github-hooks: true
- github-org: ''
- github_pr_whitelist:
- - ''
- github_pr_admin_list:
- - ''
+ included-regions: '{obj:github_included_regions}'
#############################
# Maven Verify Dependencies #
submodule-recursive: true
gerrit_trigger_file_paths:
- - compare-type: ANT
- pattern: '{node-dir}/**'
+ - compare-type: REG_EXP
+ pattern: '.*'
- gerrit_verify_triggers:
- - patchset-created-event:
- exclude-drafts: false
- exclude-trivial-rebase: false
- exclude-no-code-change: false
- - draft-published-event
- - comment-added-contains-event:
- comment-contains-value: recheck$
+ # github_included_regions MUST match gerrit_trigger_file_paths
+ github_included_regions:
+ - '.*'
#####################
# Job Configuration #
id: gerrit-node-verify
<<: *lf_node_verify
+ ######################
+ # Default parameters #
+ ######################
+
+ gerrit_verify_triggers:
+ - patchset-created-event:
+ exclude-drafts: true
+ exclude-trivial-rebase: false
+ exclude-no-code-change: false
+ - draft-published-event
+ - comment-added-contains-event:
+ comment-contains-value: '^Patch Set[ ]+[0-9]+:([ ]+|[\n]+)(recheck|reverify)$'
+
+ #####################
+ # Job Configuration #
+ #####################
+
scm:
- lf-infra-gerrit-scm:
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- github-pull-request:
- trigger-phrase: '^recheck$'
+ trigger-phrase: '^(recheck|reverify)$'
only-trigger-phrase: false
status-context: 'Node Verify'
permit-all: true
github-hooks: true
- auto-close-on-fail: false
+ included-regions: '{obj:github_included_regions}'
archive-artifacts: >
**/*.log
- nexus-iq-cli-version: 1.44.0-01
#####################
# Job Configuration #
build-timeout: 60
git-url: '$GIT_URL/$PROJECT'
java-version: openjdk8
+ nexus-iq-cli-version: 1.44.0-01
staging-profile-id: '' # Unused in this job
stream: master
submodule-recursive: true
- gerrit_clm_triggers:
- - comment-added-contains-event:
- comment-contains-value: run-clm$
gerrit_trigger_file_paths:
- compare-type: ANT
- pattern: '**'
+ pattern: '.*'
+
+ # github_included_regions MUST match gerrit_trigger_file_paths
+ github_included_regions:
+ - '.*'
#####################
# Job Configuration #
#####################
parameters:
+ - lf-infra-parameters:
+ project: '{project}'
+ branch: '{branch}'
+ stream: '{stream}'
+ lftools-version: '{lftools-version}'
- string:
name: NEXUS_IQ_CLI_VERSION
default: '{nexus-iq-cli-version}'
description: Nexus IQ CLI package to download and use.
- triggers:
- # Build weekly on Saturdays
- - timed: 'H H * * 6'
- - gerrit:
- server-name: '{gerrit-server-name}'
- trigger-on: '{obj:gerrit_clm_triggers}'
- projects:
- - project-compare-type: ANT
- project-pattern: '{project}'
- branches:
- - branch-compare-type: ANT
- branch-pattern: '**/{branch}'
- file-paths: '{obj:gerrit_trigger_file_paths}'
- skip-vote:
- successful: true
- failed: true
- unstable: true
- notbuilt: true
wrappers:
- credentials-binding:
- username-password-separated:
# yamllint disable-line rule:key-duplicates
<<: *lf_python_xc_clm
+ ######################
+ # Default parameters #
+ ######################
+
+ gerrit_clm_triggers:
+ - comment-added-contains-event:
+ comment-contains-value: run-clm$
+
+ #####################
+ # Job Configuration #
+ #####################
+
scm:
- lf-infra-gerrit-scm:
jenkins-ssh-credential: '{jenkins-ssh-credential}'
submodule-recursive: '{submodule-recursive}'
choosing-strategy: default
+ triggers:
+ # Build weekly on Saturdays
+ - timed: 'H H * * 6'
+ - gerrit:
+ server-name: '{gerrit-server-name}'
+ trigger-on: '{obj:gerrit_clm_triggers}'
+ projects:
+ - project-compare-type: ANT
+ project-pattern: '{project}'
+ branches:
+ - branch-compare-type: ANT
+ branch-pattern: '**/{branch}'
+ file-paths: '{obj:gerrit_trigger_file_paths}'
+ skip-vote:
+ successful: true
+ failed: true
+ unstable: true
+ notbuilt: true
+
- job-template:
name: '{project-name}-python-clm-{stream}'
id: github-python-xc-clm
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- - lf-infra-github-pr-trigger:
+ # Build weekly on Saturdays
+ - timed: 'H H * * 6'
+ - github-pull-request:
trigger-phrase: '^run-clm$'
only-trigger-phrase: false
status-context: 'CLM'
permit-all: true
github-hooks: true
- github-org: ''
- github_pr_whitelist:
- - ''
- github_pr_admin_list:
- - ''
+ included-regions: '{obj:github_included_regions}'
########################
# Python Sonar with Tox #
stream: master
submodule-recursive: true
- gerrit_sonar_triggers:
- - comment-added-contains-event:
- comment-contains-value: run-sonar$
gerrit_trigger_file_paths:
- - compare-type: ANT
- pattern: '**'
+ - compare-type: REG_EXP
+ pattern: '.*'
+
+ # github_included_regions MUST match gerrit_trigger_file_paths
+ github_included_regions:
+ - '.*'
#####################
# Job Configuration #
however to use a specific version of the sonar-maven-plugin we
can call "org.codehaus.mojo:sonar-maven-plugin:3.3.0.603:sonar".
- triggers:
- - timed: '{obj:cron}'
- - gerrit:
- server-name: '{gerrit-server-name}'
- trigger-on: '{obj:gerrit_sonar_triggers}'
- projects:
- - project-compare-type: 'ANT'
- project-pattern: '{project}'
- branches:
- - branch-compare-type: 'ANT'
- branch-pattern: '**/master'
- file-paths: '{obj:gerrit_trigger_file_paths}'
- skip-vote:
- successful: true
- failed: true
- unstable: true
- notbuilt: true
-
builders:
- shell: !include-raw-escape:
# Workaround issue where the tox run later breaks the lftools virtualenv.
# yamllint disable-line rule:key-duplicates
<<: *lf_tox_sonar
+ ######################
+ # Default parameters #
+ ######################
+
+ gerrit_sonar_triggers:
+ - comment-added-contains-event:
+ comment-contains-value: run-sonar$
+
+ #####################
+ # Job Configuration #
+ #####################
+
scm:
- lf-infra-gerrit-scm:
jenkins-ssh-credential: '{jenkins-ssh-credential}'
submodule-recursive: '{submodule-recursive}'
choosing-strategy: default
+ triggers:
+ - timed: '{obj:cron}'
+ - gerrit:
+ server-name: '{gerrit-server-name}'
+ trigger-on: '{obj:gerrit_sonar_triggers}'
+ projects:
+ - project-compare-type: 'ANT'
+ project-pattern: '{project}'
+ branches:
+ - branch-compare-type: 'ANT'
+ branch-pattern: '**/{branch}'
+ file-paths: '{obj:gerrit_trigger_file_paths}'
+ skip-vote:
+ successful: true
+ failed: true
+ unstable: true
+ notbuilt: true
+
- job-template:
name: '{project-name}-tox-sonar'
id: github-tox-sonar
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- - lf-infra-github-pr-trigger:
+ - github-pull-request:
trigger-phrase: '^run-sonar$'
only-trigger-phrase: false
status-context: 'Python Sonar'
permit-all: true
github-hooks: true
- github-org: ''
- github_pr_whitelist:
- - ''
- github_pr_admin_list:
- - ''
+ included-regions: '{obj:github_included_regions}'
##############
# Tox Verify #
build-days-to-keep: 7
build-timeout: 15
git-url: '$GIT_URL/$GERRIT_PROJECT'
+ parallel: true
python-version: python2
stream: master
submodule-recursive: true
tox-dir: ''
tox-envs: ''
+ gerrit_trigger_file_paths:
+ - compare-type: REG_EXP
+ pattern: '.*'
+
+ # github_included_regions MUST match gerrit_trigger_file_paths
+ github_included_regions:
+ - '.*'
+
#####################
# Job Configuration #
#####################
- lf-infra-tox-parameters:
tox-dir: '{tox-dir}'
tox-envs: '{tox-envs}'
+ - bool:
+ name: PARALLEL
+ default: '{parallel}'
+ description: Tox test type used to configure serial or parallel testing.
wrappers:
- lf-infra-wrappers:
exclude-no-code-change: false
- draft-published-event
- comment-added-contains-event:
- comment-contains-value: recheck$
- gerrit_trigger_file_paths:
- - compare-type: ANT
- pattern: '**'
-
- parallel: true
+ comment-contains-value: '^Patch Set[ ]+[0-9]+:([ ]+|[\n]+)(recheck|reverify)$'
#####################
# Job Configuration #
submodule-recursive: '{submodule-recursive}'
choosing-strategy: gerrit
- parameters:
- - bool:
- name: PARALLEL
- default: '{parallel}'
- description: Tox test type used to configure serial or parallel testing.
-
-
triggers:
- gerrit:
server-name: '{gerrit-server-name}'
triggers:
- github-pull-request:
- trigger-phrase: '^recheck$'
+ trigger-phrase: '^(recheck|reverify)$'
only-trigger-phrase: false
- status-context: 'JJB Verify'
+ status-context: 'Tox Verify'
permit-all: true
github-hooks: true
- auto-close-on-fail: false
+ included-regions: '{obj:github_included_regions}'
######################
gerrit_trigger_file_paths:
- - compare-type: ANT
- pattern: '**/*.rst'
- - compare-type: ANT
- pattern: '**/conf.py'
+ - compare-type: REG_EXP
+ pattern: '.*\.css'
+ - compare-type: REG_EXP
+ pattern: '.*\.html'
+ - compare-type: REG_EXP
+ pattern: '.*\.rst'
+ - compare-type: REG_EXP
+ pattern: '.*\/conf.py'
+
+ # github_included_regions MUST match gerrit_trigger_file_paths
+ github_included_regions:
+ - '.*\.css'
+ - '.*\.html'
+ - '.*\.rst'
+ - '.*\/conf.py'
#####################
# Job Configuration #
url: '{git-url}/{github-org}/{project}'
triggers:
- - lf-infra-github-pr-trigger:
+ - timed: 'H H * * *'
+ - github
+ - pollscm:
+ cron: ''
+ - github-pull-request:
trigger-phrase: '^remerge$'
- only-trigger-phrase: false
+ only-trigger-phrase: true
status-context: 'RTD Merge'
- permit-all: true
+ permit-all: false
github-hooks: true
- github-org: ''
- github_pr_whitelist:
- - ''
- github_pr_admin_list:
- - ''
- - timed: 'H H * * *'
+ org-list:
+ - '{github-org}'
+ white-list: '{obj:github_pr_whitelist}'
+ admin-list: '{obj:github_pr_admin_list}'
+ included-regions: '{obj:github_included_regions}'
#############
# RTD VERIFY#
exclude-no-code-change: false
- draft-published-event
- comment-added-contains-event:
- comment-contains-value: recheck$
+ comment-contains-value: '^Patch Set[ ]+[0-9]+:([ ]+|[\n]+)(recheck|reverify)$'
#####################
# Job Configuration #
jenkins-ssh-credential: '{jenkins-ssh-credential}'
triggers:
- - lf-infra-github-pr-trigger:
- trigger-phrase: '^recheck$'
+ - github-pull-request:
+ trigger-phrase: '^(recheck|reverify)$'
only-trigger-phrase: false
status-context: 'RTD Verify'
permit-all: true
github-hooks: true
- github-org: ''
- github_pr_whitelist:
- - ''
- github_pr_admin_list:
- - ''
+ included-regions: '{obj:github_included_regions}'
--- /dev/null
+global-jjb v0.20.0 Release Notes
+================================
+
+Thanks to Anil, Dimitrios, Jeremy, Jessica, Thanh, Trevor and Zack
+for contributing to this release.
+
+
+Upgrade considerations
+----------------------
+
+Not required but we highly recommend switching to Maven 3.5.2 for any Maven
+Merge jobs.
+
+
+Notable changes since v0.19.0
+-----------------------------
+
+008d7f1 Thanh Ha Remove maven_metadata_validate function
+The maven_metadata_validate function is removed and we now recommend using
+Maven 3.5.2 or later for maven-merge jobs.
+
+dc91493 Jeremy Phelps Add docker clean macro
+New macro `lf-infra-provide-docker-cleanup` to cleanup docker images.
+
+1951f33 Zack Williams Add *.groovy as a filetype trigger to ci-verify
+aec2a0c Trevor Bramwell Support the .yml YAML File Suffix for CI Jobs
+jjb-verify jobs will now trigger on *.groovy and *.yml file changes.
+
+
+Full Change Log
+---------------
+
+ed23bb6 Thanh Ha Remove echo statement
+1951f33 Zack Williams Add *.groovy as a filetype trigger to ci-verify
+7a1a1bd Thanh Ha Simplify conditional statement for ALT_NEXUS_REPO
+9b24045 Trevor Bramwell Fix echo when removing venv in jjb-cleanup.sh
+1c748f6 Trevor Bramwell Use a temporary directory for jjb-installs
+aec2a0c Trevor Bramwell Support the .yml YAML File Suffix for CI Jobs
+4925983 Jeremy Phelps Prevent unbound variable error on create-netrc.sh
+7550665 Jessica Wagantall Add lf-infra-parameters XC CLM
+dc91493 Jeremy Phelps Add docker clean macro
+f90d002 Jeremy Phelps Add ALT_NEXUS_URL to create-netrc
+334d6e2 Anil Belur Use --jobs-only option with jjb-deploy
+d3cac52 Dimitrios Markou Bug Fix: Replace -type f with -xtype f
+008d7f1 Thanh Ha Remove maven_metadata_validate function
--- /dev/null
+global-jjb v0.21.0 Release Notes
+================================
+
+Thanks to Andrew, Anil, Jessica, Luis, Thanh, and Trevor
+for contributing to this release.
+
+
+Upgrade considerations
+----------------------
+
+efc1f45 Thanh Ha Consolidate keypair default names to jenkins-ssh
+This is a potential breaking change. If you are not explicitly declaring
+`jenkins-ssh-credential` in defaults.yaml or `KEY_PAIR_NAME` in clouds.cfg
+for the "jenkins-cfg" job. Then the default naming for these keypairs is now
+`jenkins-ssh`. Make sure to update either the Jenkins Credentials name or
+set the explicit configurations as mentioned previously.
+
+
+Notable changes since v0.20.0
+-----------------------------
+
+d1fb28b Luis Gomez Allow any project/branch to deploy jjb jobs
+The "jjb-deploy" job can now be run from any Gerrit project. When run against
+an "ci-management" repo then it will perform as it did before. When run against
+a non-ci-manamgent repo the job will fetch the master branch from ci-management
+and deploy the job from there.
+
+55ab265 Thanh Ha Setup tools in global-jjb tox.ini
+Creates a global tox.ini which can be used by projects to use common
+PyPi tools. This replaces the need to "source /path/to/venv"
+everywhere in our scripts to jump to different tools.
+Instead we replace it with one of:
+
+ tox -q -c jjb/global-jjb/tox.ini -e openstack -- {parameters}
+
+d9aeddd Thanh Ha Add jenkins-init-scripts to global-jjb
+These are the minion initialization scripts that start running
+when our cloud plugin spawns a new minion. They are responsible
+for prepping the minion for Jenkins to take over and run the job.
+
+efc1f45 Thanh Ha Consolidate keypair default names to jenkins-ssh
+This is an attempt to consolidate all our different instances
+of the ssh-keypair name all over the place. By default we will
+consolidate on `jenkins-ssh` as the default keypair name.
+
+a079f1b Andrew Grimberg Add package lists to job logs
+Jobs will now archive a package-listing of OS level package installations for
+reference in the log server.
+
+f6db870 Trevor Bramwell Configurable Timed Trigger for Packer Merge Jobs
+Packer jobs timed triggers can now be overrided by projects that want a
+different timer than default of "@monthly".
+
+b83660f Jessica Wagantall Add INFO.yaml verify jobs
+New job for verifying LF INFO.yaml files. These files will be used in the
+future to manage things like project permissions from inside the repo.
+
+
+Full Change Log
+---------------
+
+a8ff64a Thanh Ha Remove unnecessary github trigger macro
+22db112 Anil Belur Set minimal packer version to 1.2.3
+b83660f Jessica Wagantall Add INFO.yaml verify jobs
+f6db870 Trevor Bramwell Configurable Timed Trigger for Packer Merge Jobs
+3142525 Thanh Ha Set docs version from git
+d0b8cdb Andrew Grimberg Only create package diff if start and end exist
+a57c991 Andrew Grimberg Make package listing script smarter
+6eefde1 Thanh Ha Use include-regions to limit RTD build paths
+0f14d33 Thanh Ha Use include-regions to limit MVN build paths
+2bad748 Thanh Ha Use include-regions to limit node build paths
+3046ee6 Thanh Ha Use include-regions to limit Py build paths
+a0fac97 Thanh Ha Use include-regions to limit JenkinsCFG build path
+add7cdb Thanh Ha Use include-regions to limit JJB build paths
+0845375 Thanh Ha Use include-regions to limit packer build paths
+4d6ed41 Thanh Ha Add CSS and HTML to rtd build pattern
+2fea1a4 Trevor Bramwell Add documentation for CFP files
+a079f1b Andrew Grimberg Add package lists to job logs
+801f8ef Anil Belur Document lf-infra-push-gerrit-patch macro in RTD
+9dd5c89 Thanh Ha Update OpenStack plugin supported versions list
+c6c8d6b Thanh Ha DO NOT fail if global-vars-.sh is not configured
+ad6bd3d Thanh Ha Handle missing CREDENTIAL when creating .netrc
+b9e3254 Thanh Ha Disable -x to prevent printing passwords
+386d0e8 Thanh Ha Clear log credential files with force (-f)
+efa1504 Thanh Ha Add sha1 parameter for GitHub jobs
+efc1f45 Thanh Ha Consolidate keypair default names to jenkins-ssh
+d9aeddd Thanh Ha Add jenkins-init-scripts to global-jjb
+140ad4d Thanh Ha Add labels for linking in configuration.rst
+55ab265 Thanh Ha Setup tools in global-jjb tox.ini
+eeefc4d Thanh Ha Redirect README documentation to RTD
+d1fb28b Luis Gomez Allow any project/branch to deploy jjb jobs
+5aaa1c1 Thanh Ha Migrate README to RTD
+9f181f0 Thanh Ha REM security notice about PostBuildScript plugin
##############################################################################
echo "---> create-netrc.sh"
-# Ensure we fail the job if any steps fail.
-set -eu -o pipefail
-
-ALT_NEXUS_URL="${ALT_NEXUS_URL}"
-
-if [ -z "$ALT_NEXUS_URL" ]
-then
+if [ -z "$ALT_NEXUS_URL" ]; then
NEXUS_URL="${NEXUSPROXY:-$NEXUS_URL}"
else
NEXUS_URL="${ALT_NEXUS_URL}"
fi
-
CREDENTIAL=$(xmlstarlet sel -N "x=http://maven.apache.org/SETTINGS/1.0.0" \
-t -m "/x:settings/x:servers/x:server[x:id='${SERVER_ID}']" \
-v x:username -o ":" -v x:password \
"$SETTINGS_FILE")
+# Ensure we fail the job if any steps fail.
+set -eu -o pipefail
+
+# Handle when a project chooses to not archive logs to a log server
+# in other cases if CREDENTIAL is not found then fail the build.
+if [ -z "$CREDENTIAL" ] && [ "$SERVER_ID" == "logs" ]; then
+ echo "WARN: Log server credential not found."
+ exit 0
+elif [ -z "$CREDENTIAL" ]; then
+ echo "ERROR: Credential not found."
+ exit 1
+fi
+
machine=$(echo "$NEXUS_URL" | awk -F/ '{print $3}')
user=$(echo "$CREDENTIAL" | cut -f1 -d:)
pass=$(echo "$CREDENTIAL" | cut -f2 -d:)
+set +x # Disable `set -x` to prevent printing passwords
echo "machine ${machine%:*} login $user password $pass" > ~/.netrc
# Takes a list of Gerrit patches and fetches all projects and cherry-pick
# patches for projects. The trigger is
# 'recheck: SPACE_SEPERATED_LIST_OF_PATCHES'
+# or
+# 'reverify: SPACE_SEPERATED_LIST_OF_PATCHES'
#
# NOTE: This script assumes the user will provide the correct dependency order
# via the PATCHES list.
REPOS_DIR="$WORKSPACE/.repos"
-IFS=" " read -r -a PATCHES <<< "$(echo "$GERRIT_EVENT_COMMENT_TEXT" | grep 'recheck:' | awk -F: '{print $2}')"
+IFS=" " read -r -a PATCHES <<< "$(echo "$GERRIT_EVENT_COMMENT_TEXT" | grep -E '(recheck:|reverify:)' | awk -F: '{print $2}')"
# Workaround for git-review bug in v1.24
# https://storyboard.openstack.org/#!/story/2001081
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2018 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+echo '--> git-validate-info-yaml.sh'
+# This script will make sure that the INFO.yaml file changes are kept
+# isolated from other file changes.
+# INFO.yaml file creation or moddifications should be kept in its own separate
+# Gerrit.
+
+# This script will fail if other file changes are also included in the same
+# patch.
+
+# Ensure we fail the job if any steps fail.
+set -e -o pipefail
+set +u
+
+MODIFIED_FILES=$(git diff --name-only HEAD~1)
+
+if [ "$MODIFIED_FILES" != "INFO.yaml" ]; then
+ echo 'ERROR: Do not combine INFO.yaml file changes with other files. Please isolate INFO.yaml changes.'
+ exit 1
+fi
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2018 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+echo '--> info-file-validate.sh'
+set -e -o pipefail
+set -x # Enable trace
+
+virtualenv --quiet "/tmp/v/info"
+# shellcheck source=/tmp/v/info/bin/activate disable=SC1091
+source "/tmp/v/info/bin/activate"
+pip install PyYAML jsonschema rfc3987
+
+# Cloning global-jjb to get access to needed scripts
+git clone https://github.com/lfit/releng-global-jjb.git
+
+python releng-global-jjb/yaml-verify-schema.py \
+ -s releng-global-jjb/info-schema \
+ -y INFO.yaml
+
+rm -rf releng-global-jjb
set -eu -o pipefail
+version_ge() { test "$(echo "$@" | tr " " "\n" | sort -rV | head -n 1)" == "$1"; }
+
get_cfg() {
if [ -z ${3+x} ]; then
>&2 echo "Usage: get_cfg CFG_FILE SETTING DEFAULT"
exit 1
fi
- cfg=$(grep "${setting^^}" "$cfg_file" | tail -1 | awk -F'=' '{print $2}')
+ cfg=$(grep "^${setting^^}=" "$cfg_file" | tail -1 | awk -F'=' '{print $2}')
cfg=${cfg:-"$default"}
echo "$cfg"
}
echo ")"
}
+get_launcher_factory() {
+ if [ -z $1 ]; then
+ >&2 echo "Usage: get_launcher_factory JNLP|SSH"
+ exit 1
+ fi
+
+ local connection_type="$1"
+
+ if [ "$connection_type" == "JNLP" ]; then
+ echo "new LauncherFactory.JNLP()"
+ elif [ "$connection_type" == "SSH" ]; then
+ echo "new LauncherFactory.SSH(\"$key_pair_name\", \"\")"
+ else
+ >&2 echo "Unknown connection type $connection_type"
+ exit 1
+ fi
+}
+
get_minion_options() {
if [ -z $1 ]; then
>&2 echo "Usage: get_minion_options CFG_FILE"
volume_size=$(get_cfg "$cfg_file" VOLUME_SIZE "")
hardware_id=$(get_cfg "$cfg_file" HARDWARE_ID "")
network_id=$(get_cfg "$cfg_file" NETWORK_ID "")
- user_data_id=$(get_cfg "$cfg_file" USER_DATA_ID "jenkins-init-script")
+
+ udi_default="$(get_cfg "$(dirname $cfg_file)/cloud.cfg" USER_DATA_ID "jenkins-init-script")"
+ user_data_id=$(get_cfg "$cfg_file" USER_DATA_ID "$udi_default")
# Handle Sandbox systems that might have a different cap.
instance_cap=$(get_cfg "$cfg_file" INSTANCE_CAP "null")
if [ "$silo" == "sandbox" ]; then
instance_cap=$(get_cfg "$cfg_file" SANDBOX_CAP "null")
fi
+ min_instance_cap=$(get_cfg "$cfg_file" MIN_INSTANCE_CAP "null")
floating_ip_pool=$(get_cfg "$cfg_file" FLOATING_IP_POOL "")
security_groups=$(get_cfg "$cfg_file" SECURITY_GROUPS "default")
availability_zone=$(get_cfg "$cfg_file" AVAILABILITY_ZONE "")
start_timeout=$(get_cfg "$cfg_file" START_TIMEOUT "600000")
- key_pair_name=$(get_cfg "$cfg_file" KEY_PAIR_NAME "jenkins")
+
+ kpn_default="$(get_cfg "$(dirname $cfg_file)/cloud.cfg" KEY_PAIR_NAME "jenkins-ssh")"
+ key_pair_name=$(get_cfg "$cfg_file" KEY_PAIR_NAME "$kpn_default")
+
num_executors=$(get_cfg "$cfg_file" NUM_EXECUTORS "1")
jvm_options=$(get_cfg "$cfg_file" JVM_OPTIONS "")
fs_root=$(get_cfg "$cfg_file" FS_ROOT "/w")
retention_time=$(get_cfg "$cfg_file" RETENTION_TIME "0")
-
- if [ ! -z "$volume_size" ]; then
- echo " new BootSource.VolumeFromImage(\"$image_name\", $volume_size),"
- else
- echo " new BootSource.Image(\"$image_name\"),"
+ connection_type=$(get_cfg "$cfg_file" CONNECTION_TYPE "SSH")
+ launcher_factory=$(get_launcher_factory "$connection_type")
+
+ OS_PLUGIN_VER="$(lftools jenkins plugins list \
+ | grep -i 'OpenStack Cloud Plugin' | awk -F':' '{print $2}')"
+ if version_ge "$OS_PLUGIN_VER" "2.35"; then
+ if [ ! -z "$volume_size" ]; then
+ echo " new BootSource.VolumeFromImage(\"$image_name\", $volume_size),"
+ else
+ echo " new BootSource.Image(\"$image_name\"),"
+ fi
+ echo " \"${flavors[${hardware_id}]}\","
+ echo " \"$network_id\","
+ echo " \"$user_data_id\","
+ echo " $instance_cap,"
+ echo " $min_instance_cap,"
+ echo " \"$floating_ip_pool\","
+ echo " \"$security_groups\","
+ echo " \"$availability_zone\","
+ echo " $start_timeout,"
+ echo " \"$key_pair_name\","
+ echo " $num_executors,"
+ echo " \"$jvm_options\","
+ echo " \"$fs_root\","
+ echo " $launcher_factory,"
+ echo " $retention_time"
+
+ else # SlaveOptions() structure for versions <= 2.34
+ if [ ! -z "$volume_size" ]; then
+ echo " new BootSource.VolumeFromImage(\"$image_name\", $volume_size),"
+ else
+ echo " new BootSource.Image(\"$image_name\"),"
+ fi
+ echo " \"${flavors[${hardware_id}]}\","
+ echo " \"$network_id\","
+ echo " \"$user_data_id\","
+ echo " $instance_cap,"
+ echo " \"$floating_ip_pool\","
+ echo " \"$security_groups\","
+ echo " \"$availability_zone\","
+ echo " $start_timeout,"
+ echo " \"$key_pair_name\","
+ echo " $num_executors,"
+ echo " \"$jvm_options\","
+ echo " \"$fs_root\","
+ echo " $launcher_factory,"
+ echo " $retention_time"
fi
- echo " \"${flavors[${hardware_id}]}\","
- echo " \"$network_id\","
- echo " \"$user_data_id\","
- echo " $instance_cap,"
- echo " \"$floating_ip_pool\","
- echo " \"$security_groups\","
- echo " \"$availability_zone\","
- echo " $start_timeout,"
- echo " \"$key_pair_name\","
- echo " $num_executors,"
- echo " \"$jvm_options\","
- echo " \"$fs_root\","
- echo " new LauncherFactory.SSH(\"jenkins\", \"\"),"
- echo " $retention_time"
}
get_template_cfg() {
- if [ -z $1 ]; then
- >&2 echo "Usage: get_template_cfg CFG_FILE"
+ if [ -z $2 ]; then
+ >&2 echo "Usage: get_template_cfg CFG_FILE SILO [MINION_PREFIX]"
exit 1
fi
local cfg_file="$1"
- local minion_prefix="${2:-}"
+ local silo="${2}"
+ local minion_prefix="${3:-}"
+
template_name=$(basename $cfg_file .cfg)
labels=$(get_cfg "$cfg_file" LABELS "")
echo "minion_options = new SlaveOptions("
- get_minion_options "$cfg_file"
+ get_minion_options "$cfg_file" "$silo"
echo ")"
echo "template = new JCloudsSlaveTemplate("
node_prefix="${silo}-"
fi
+ set +x # Disable `set -x` to prevent printing passwords
+ echo "Configuring $silo"
+ JENKINS_URL=$(crudini --get "$HOME"/.config/jenkins_jobs/jenkins_jobs.ini "$silo" url)
+ JENKINS_USER=$(crudini --get "$HOME"/.config/jenkins_jobs/jenkins_jobs.ini "$silo" user)
+ JENKINS_PASSWORD=$(crudini --get "$HOME"/.config/jenkins_jobs/jenkins_jobs.ini "$silo" password)
+ export JENKINS_URL
+ export JENKINS_USER
+ export JENKINS_PASSWORD
+
echo "-----> Groovy script $script_file"
for cloud in "${clouds[@]}"; do
cfg_dir="${cloud}"
echo "templates = []" >> $insert_file
mapfile -t templates < <(find $cfg_dir -maxdepth 1 -not -type d -not -name "cloud.cfg")
for template in "${templates[@]}"; do
- get_template_cfg "$template" "$node_prefix" >> "$insert_file"
+ get_template_cfg "$template" "$silo" "$node_prefix" >> "$insert_file"
echo "templates.add(template)" >> "$insert_file"
done
cat "$insert_file" >> "$script_file"
done
- set +x # Disable `set -x` to prevent printing passwords
- echo "Configuring $silo"
- JENKINS_URL=$(crudini --get "$HOME"/.config/jenkins_jobs/jenkins_jobs.ini "$silo" url)
- JENKINS_USER=$(crudini --get "$HOME"/.config/jenkins_jobs/jenkins_jobs.ini "$silo" user)
- JENKINS_PASSWORD=$(crudini --get "$HOME"/.config/jenkins_jobs/jenkins_jobs.ini "$silo" password)
- export JENKINS_URL
- export JENKINS_USER
- export JENKINS_PASSWORD
lftools jenkins groovy "$script_file"
done
set -eu -o pipefail
for silo in $silos; do
+ if [ ! -f "$WORKSPACE/jenkins-config/global-vars-$silo.sh" ]; then
+ echo "WARN: jenkins-config/global-vars-$silo.sh does not exist. Skipping cloud management..."
+ echo "We highly recommend setting up global-vars-$silo.sh to manage the Jenkins global variables."
+ echo "Refer to https://docs.releng.linuxfoundation.org/projects/global-jjb/en/latest/jjb/lf-ci-jobs.html#global-environment-variables for details."
+ continue
+ fi
+
set +x # Ensure that no other scripts add `set -x` and print passwords
echo "Configuring $silo"
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2017 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+echo "---> jjb-cleanup.sh"
+# Cleans up the temporary directory created for the virtualenv but only if it
+# exists under /tmp. This is to ensure we never attempt to blow away '/'
+# through mis-set bash variables.
+
+# Ensure we fail the job if any steps fail.
+# DO NOT set -u as virtualenv's activate script has unbound variables
+set -e -o pipefail
+
+# shellcheck source="$WORKSPACE/.jjb.properties" disable=SC1091
+source "$WORKSPACE/.jjb.properties"
+if [[ -n "$JJB_VENV" && "$JJB_VENV" =~ /tmp/.* ]]; then
+ rm -r "$JJB_VENV" && echo "$JJB_VENV removed"
+ unset JJB_VENV
+fi
+rm "$WORKSPACE/.jjb.properties"
# shellcheck source=/tmp/v/jenkins-job-builder/bin/activate disable=SC1091
source "/tmp/v/jenkins-job-builder/bin/activate"
-echo "-----> Fetching project"
-git fetch origin "$GERRIT_REFSPEC" && git checkout FETCH_HEAD
+# Fetch patch if gerrit project matches the jjb-deploy project
+if [ "${GERRIT_PROJECT}" == "${PROJECT}" ]; then
+ echo "-----> Fetching ${PROJECT} patch"
+ git fetch origin "$GERRIT_REFSPEC" && git checkout FETCH_HEAD
+fi
# If not Gerrit Trigger than assume GitHub
COMMENT="${GERRIT_EVENT_COMMENT_TEXT:-$ghprbCommentBody}"
# DO NOT set -u as virtualenv's activate script has unbound variables
set -e -o pipefail
-virtualenv "/tmp/v/jjb"
-# shellcheck source=/tmp/v/jjb/bin/activate disable=SC1091
-source "/tmp/v/jjb/bin/activate"
+# Create a virtualenv in a temporary directoy and write it down to used
+# or cleaned up later; cleanup is done in the script jjb-cleanup.sh.
+JJB_VENV="$(mktemp -d)"
+export JJB_VENV
+virtualenv "$JJB_VENV"
+echo "JJB_VENV=$JJB_VENV" > "$WORKSPACE/.jjb.properties"
+# shellcheck source=$VENV_DIR/bin/activate disable=SC1091
+source "$JJB_VENV/bin/activate"
pip install --quiet --upgrade "pip==9.0.3" setuptools
pip install --quiet --upgrade "jenkins-job-builder==$JJB_VERSION"
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2017 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+# Scans files for license header violations
+#
+# This script is meant to be called by a Jenkins job and was not designed to
+# run standalone refer to the below Inputs for details on optional ENVVAR
+# inputs.
+
+echo "---> license-check.sh"
+
+# --- Inputs
+
+# Space separated list of file patterns to scan for license headers.
+file_patterns=("${FILE_PATTERNS:-*.go *.groovy *.java *.py *.sh}")
+# Version of the License Header Checker to install
+lhc_version="${LHC_VERSION:-0.2.0}"
+# Comma-separated list of paths to exclude from license checking
+license_exclude_paths="${LICENSE_EXCLUDE_PATHS:-}"
+# Comma-separated list of allowed licenses
+licenses_allowed="${LICENSES_ALLOWED:-Apache-2.0,EPL-1.0,MIT}"
+
+if [[ "${SPDX_DISABLE}" == "true" ]]; then
+ disable_spdx="--disable-spdx"
+else
+ disable_spdx=""
+fi
+
+# --- Script start
+
+# DO NOT enable -u because LICENSE_EXCLUDE_PATHS is unbound.
+# Ensure we fail the job if any steps fail.
+set -eux -o pipefail
+
+if hash lhc 2>/dev/null; then
+ echo "License Header Checker is installed."
+ lhc --version
+else
+ echo "License Header Checker is not installed. Installing..."
+ mkdir "$WORKSPACE/bin"
+ wget -nv -O "/tmp/lhc.tar.gz" "https://nexus.opendaylight.org/content/repositories/hosted_installers/org/linuxfoundation/lhc/${lhc_version}/lhc-${lhc_version}.tar.gz"
+ tar -zxvf /tmp/lhc.tar.gz -C "$WORKSPACE/bin"
+ chmod +x "$WORKSPACE/bin/lhc"
+ export PATH="$WORKSPACE/bin:$PATH"
+ lhc --version
+fi
+
+
+set -f # Disable globbing for $file_patterns to pass '*'
+# Purposely disable SC2068 for $file_patterns
+# shellcheck disable=SC2068
+lhc --license "$licenses_allowed" ${disable_spdx} \
+ --exclude "$license_exclude_paths" \
+ ${file_patterns[@]}
+set +f
##############################################################################
# Clear log credential files
-rm "$SETTINGS_FILE"
-rm ~/.netrc
+rm -f "$SETTINGS_FILE"
+rm -f ~/.netrc
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2018 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+
+echo "---> package-listing.sh"
+
+# Ensure we fail the job if any steps fail
+set -eu -o pipefail
+
+set -x # Trace commands for this script to make debugging easier
+
+OS_FAMILY=$(facter osfamily | tr '[:upper:]' '[:lower:]')
+
+# Capture the CI WORKSPACE safely in the case that it doesn't exist
+workspace="${WORKSPACE:-}"
+
+START_PACKAGES=/tmp/packages_start.txt
+END_PACKAGES=/tmp/packages_end.txt
+DIFF_PACKAGES=/tmp/packages_diff.txt
+
+# Swap to creating END_PACKAGES if we are running in a CI job (determined by if
+# we have a workspace env) or if the starting packages listing already exists.
+PACKAGES="${START_PACKAGES}"
+if ( [ "${workspace}" ] || [ -f "${START_PACKAGES}" ] )
+then
+ PACKAGES="${END_PACKAGES}"
+fi
+
+case "${OS_FAMILY}" in
+ redhat|suse)
+ # RedHat and Suse flavors all use rpm at the package level
+ rpm -qa | sort > "${PACKAGES}"
+ ;;
+ debian)
+ # Debian derived flavors all use dpkg at the package level
+ dpkg -l | grep '^ii' > "${PACKAGES}"
+ ;;
+ *)
+ # nothing to do
+ ;;
+esac
+
+if ( [ -f "${START_PACKAGES}" ] && [ -f "${END_PACKAGES}" ] )
+then
+ diff "${START_PACKAGES}" "${END_PACKAGES}" > "${DIFF_PACKAGES}"
+fi
+
+# If running in a Jenkins job, then copy the created files to the archives
+# location
+if [ "${workspace}" ]
+then
+ mkdir -p "${workspace}/archives/"
+ cp -f /tmp/packages_*.txt "${workspace}/archives/"
+fi
set -eu -o pipefail
# $PACKER_VERSION : Define a packer version passed as job paramter
-PACKER_VERSION="${PACKER_VERSION:-1.2.2}"
+PACKER_VERSION="${PACKER_VERSION:-1.2.3}"
export PATH="${WORKSPACE}/bin:$PATH"
packer_install() {
# Setup sigul RPM repo
echo "[fedora-infra-sigul]
name=Fedora builder packages for sigul
-baseurl=https://infrastructure.fedoraproject.org/repo/infra/epel\$releasever-infra/\$basearch/
+baseurl=https://kojipkgs.fedoraproject.org/repos-dist/epel\$releasever-infra/latest/\$basearch/
enabled=1
gpgcheck=1
gpgkey=https://infrastructure.fedoraproject.org/repo/infra/RPM-GPG-KEY-INFRA-TAGS
[tox]
minversion = 1.6
-envlist = coala,jjb,jjb-compare-xml,license
+envlist = coala,ensure-documented,jjb,jjb-compare-xml,license
skipsdist = true
+passenv = TOX_WORK_DIR
+toxworkdir = {env:TOX_WORK_DIR:/tmp/v}
[testenv:coala]
basepython = python3
deps = -rrequirements.txt
commands =
sphinx-build -W -b html -n -d {envtmpdir}/doctrees ./docs/ {toxinidir}/docs/_build/html
- echo "Generated docs available in {toxinidir}/docs/_build/html"
-whitelist_externals = echo
[testenv:ensure-documented]
commands =
[testenv:license]
deps = lftools
commands = lftools license check-dir -r '.+' shell
+
+#########
+# Tools #
+#########
+
+[testenv:jenkins-jobs]
+deps =
+ jenkins-job-builder{env:JJB_VERSION_OP:~=}{env:JJB_VERSION:2.0.9}
+commands =
+ jenkins-jobs {posargs:--help}
+
+[testenv:lftools]
+deps = lftools{env:LFTOOLS_VERSION_OP:~=}{env:LFTOOLS_VERSION:0.13.0}
+commands = lftools {posargs:--help}
+
+[testenv:openstack]
+deps = python-openstackclient{env:OPENSTACK_VERSION_OP:~=}{env:OPENSTACK_VERSION:3.15.0}
+commands = openstack {posargs:--help}
+
+[testenv:reno]
+deps = reno{env:RENO_VERSION_OP:~=}{env:RENO_VERSION:2.9.1}
+commands = reno {posargs:--help}
--- /dev/null
+#!/usr/bin/env python3
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2018 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+"""
+Verify YAML Schema
+"""
+import argparse
+import logging
+import jsonschema
+import yaml
+
+LOADER = yaml.CSafeLoader if yaml.__with_libyaml__ else yaml.SafeLoader
+
+
+def main():
+ """Parse arguments and verify YAML"""
+ logging.basicConfig(level=logging.INFO)
+
+ parser = argparse.ArgumentParser()
+ parser.add_argument('--yaml', '-y', type=str, required=True)
+ parser.add_argument('--schema', '-s', type=str, required=True)
+
+ args = parser.parse_args()
+
+ with open(args.yaml) as _:
+ yaml_file = yaml.load(_, Loader=LOADER)
+
+ with open(args.schema) as _:
+ schema_file = yaml.load(_, Loader=LOADER)
+
+ validation = jsonschema.Draft4Validator(
+ schema_file,
+ format_checker=jsonschema.FormatChecker()
+ )
+
+ errors = 0
+ for error in validation.iter_errors(yaml_file):
+ errors += 1
+ logging.error(error)
+ if errors > 0:
+ raise RuntimeError("%d issues invalidate the schema" % errors)
+
+
+if __name__ == "__main__":
+ main()
Code Review / releng / global-jjb.git / commitdiff