- gerrit-pypi-merge
- gerrit-pypi-release-merge
- gerrit-pypi-release-verify
- - gerrit-pypi-verify
- gerrit-tox-sonar
+ - gerrit-tox-sonarqube
+ - gerrit-pypi-verify
project-name: gerrit-python
- github-pypi-release-merge
- github-pypi-release-verify
- github-tox-sonar
+ - github-tox-sonarqube
- github-pypi-verify
project-name: github-python
:install-prefix: CMAKE_INSTALL_PREFIX to use for install.
(default: $BUILD_DIR/output)
:make-opts: Parameters to pass to make. (default: '')
+ :pre-build: Shell script to run before performing build. Useful for
+ setting up dependencies. (default: '')
:sonar-additional-args: Command line arguments. (default: '')
:sonar-java-opts: JVM options. For example, use option -Xmx
to increase the memory size limit. (default: '')
- :sonar-prescan-script: A shell script to run before the build and scan.
- Useful for setting up dependencies. (default: '')
:sonar-project-file: The file name with Sonar configuration properties
(default: sonar-project.properties)
:sonar-properties: Sonar configuration properties. (default: '')
:build-node: The node to run build on.
:jenkins-ssh-credential: Credential to use for SSH.
(Configure in defaults.yaml)
+ :mvn-settings: The name of settings file containing credentials for the project.
:nexus-group-id: The Maven style Group ID for the namespace of the project
in Nexus.
:staging-profile-id: The unique Nexus Staging Profile ID for the project.
:install-prefix: CMAKE_INSTALL_PREFIX to use for install.
(default: $BUILD_DIR/output)
:make-opts: Parameters to pass to make. (default: '')
+ :mvn-global-settings: The name of the Maven global settings to use for
+ Maven configuration. (default: global-settings)
:pre-build: Shell script to run before performing build. Useful for
setting up dependencies. (default: '')
:stream: Keyword that to represent a release code-name.
CMake Verify
------------
-Verify job which runs cmake && make && make install to test a project build..
+Verify job which runs cmake && make && make install to test a project build.
:Template Names:
and gather coverage statistics from the test results, then invokes
Maven to publish the results to either a Sonar server or SonarCloud.
+**Deprecated**, new projects should use Tox Sonarqube.
+
To get the Sonar coverage results, file tox.ini must exist and contain
coverage commands to run.
.. comment Stop ignoring
+Tox SonarQube
+-------------
+
+The SonarQube job invokes tox to run tests and generate code-coverage
+statistics, then runs the SonarQube Scanner Jenkins plug-in to analyze
+code, gather coverage data, and upload the results to a SonarQube server
+such as SonarCloud.io. Optionally runs a shell script before tox.
+
+Requires ``SonarQube Scanner for Jenkins``
+
+This job runs on the master branch because the basic Sonar configuration
+does not support multi-branch.
+
+Plug-in configurations
+ Manage Jenkins --> Configure System --> SonarQube servers
+ - Name: Sonar (fixed)
+ - Server URL: https://sonar.project.org/ or https://sonarcloud.io
+ - Server authentication token: none for local, API token (saved as
+ a "secret text" credential) for Sonarcloud
+
+ Manage Jenkins --> Global Tool Configuration --> SonarQube Scanner
+ - Name: SonarQube Scanner (fixed)
+ - Install automatically
+ - Select latest version
+
+:Template Names:
+
+ - {project-name}-tox-sonarqube
+ - gerrit-tox-sonarqube
+ - github-tox-sonarqube
+
+:Comment Trigger: ``run-sonar``
+
+:Required parameters:
+
+ :build-node: The node to run the build on.
+ (Commonly in defaults.yaml)
+ :jenkins-ssh-credential: Credential to use for SSH.
+ (Commonly in defaults.yaml)
+ :project: The git repository name.
+ :project-name: Prefix used to name jobs.
+
+.. comment Start ignoring WriteGoodLintBear
+
+:Optional Parameters:
+
+ :archive-artifacts: Pattern for files to archive to the logs server
+ (default: '\*\*/\*.log')
+ :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
+ :build-timeout: Timeout in minutes before aborting build. (default: 15)
+ :cron: Cron schedule when to trigger the job. This parameter also
+ supports multiline input via YAML pipe | character in cases where
+ one may want to provide more than 1 cron timer. (default: @weekly)
+ :disable-job: Whether to disable the job (default: false)
+ :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+ :github-url: URL for Github. (default: https://github.com)
+ :parallel: Boolean indicator for tox to run tests in parallel or series.
+ (default: false, in series)
+ :pre-build-script: Shell script to run before tox. Useful for setting up
+ dependencies. (default: a string with a shell comment)
+ :python-version: Python version to invoke pip install of tox-pyenv
+ (default: python3)
+ :sonar-additional-args: Command line arguments. (default: '')
+ :sonar-java-opts: JVM options. For example, use option -Xmx
+ to increase the memory size limit. (default: '')
+ :sonar-project-file: The file name with Sonar configuration properties
+ (default: sonar-project.properties)
+ :sonar-properties: Sonar configuration properties. (default: '')
+ :sonar-task: Sonar task to run. (default: '')
+ :tox-dir: Directory containing the project's tox.ini relative to
+ the workspace. The default uses tox.ini at the project root.
+ (default: '.')
+ :tox-envs: Tox environments to run. If blank run everything described
+ in tox.ini. (default: '')
+
+.. comment Stop ignoring
+
+.. note:: A job definition must provide one of the optional parameters
+ ``sonar-project-file`` and ``sonar-properties``; they cannot both be
+ empty. Set Sonar properties directly in the job definition by setting
+ the ``sonar-project-file`` property to ``""`` and adding all properties
+ under ``sonar-properties``.
+
+:Required Sonar Properties:
+
+ - sonar.login: The API token for authentication at SonarCloud.
+ Commonly defined as key "sonarcloud_api_token" in defaults.yaml.
+ - sonar.organization: The umbrella project name; e.g., "opendaylight".
+ Commonly defined as key "sonarcloud_project_organization" in defaults.yaml.
+ - sonar.projectName: The git repository name without slashes; e.g., "infrautils".
+ - sonar.projectKey: The globally unique key for the report in SonarCloud. Most
+ teams use the catenation of sonar.organization, an underscore, and
+ sonar.projectName; e.g., "opendaylight_infrautils".
+
+:Optional Sonar Properties:
+
+ - sonar.cfamily.gcov.reportsPath: directory with GCOV output files
+ - Documentation of SonarQube properties is here:
+ https://docs.sonarqube.org/latest/analysis/overview/
+
+
+Example job definition
+^^^^^^^^^^^^^^^^^^^^^^
+
+The following example defines a job for a basic Python project. This definition
+uses configuration parameters in the umbrella project's defaults.yaml file.
+
+.. code-block:: yaml
+
+ - project:
+ name: my-package-sonar
+ project: my/package
+ project-name: my-package
+ sonar-project-file: ""
+ sonar-properties: |
+ sonar.login={sonarcloud_api_token}
+ sonar.projectKey={sonarcloud_project_organization}_{project-name}
+ sonar.projectName={project-name}
+ sonar.organization={sonarcloud_project_organization}
+ sonar.sourceEncoding=UTF-8
+ sonar.sources=mypackage
+ sonar.exclusions=tests/*,setup.py
+ sonar.python.coverage.reportPaths=coverage.xml
+ jobs:
+ - gerrit-tox-sonarqube
+
Tox Verify
----------
---
+####################
+# COMMON FUNCTIONS #
+####################
+
+- lf_cmake_common: &lf_cmake_common
+ name: lf-cmake-common
+
+ #####################
+ # Job Configuration #
+ #####################
+
+ project-type: freestyle
+ node: "{build-node}"
+ disabled: "{disable-job}"
+
+ properties:
+ - lf-infra-properties:
+ build-days-to-keep: "{build-days-to-keep}"
+
+ parameters:
+ - lf-infra-parameters:
+ project: "{project}"
+ branch: "{branch}"
+ stream: "{stream}"
+ - lf-cmake-parameters:
+ build-dir: "{build-dir}"
+ cmake-opts: "{cmake-opts}"
+ install-prefix: "{install-prefix}"
+ make-opts: "{make-opts}"
+
+ wrappers:
+ - lf-infra-wrappers:
+ build-timeout: "{build-timeout}"
+ jenkins-ssh-credential: "{jenkins-ssh-credential}"
+
+ publishers:
+ - lf-infra-publish
+
###############
# CMAKE SONAR #
###############
submodule-timeout: 10
submodule-disable: false
- #####################
- # Job Configuration #
- #####################
-
- project-type: freestyle
- node: "{build-node}"
- disabled: "{disable-job}"
-
- properties:
- - lf-infra-properties:
- build-days-to-keep: "{build-days-to-keep}"
-
- parameters:
- - lf-infra-parameters:
- project: "{project}"
- branch: "{branch}"
- stream: "{stream}"
- lftools-version: "{lftools-version}"
- - lf-cmake-parameters:
- build-dir: "{build-dir}"
- cmake-opts: "{cmake-opts}"
- install-prefix: "{install-prefix}"
- make-opts: "{make-opts}"
-
- wrappers:
- - lf-infra-wrappers:
- build-timeout: "{build-timeout}"
- jenkins-ssh-credential: "{jenkins-ssh-credential}"
-
builders:
- lf-infra-pre-build
- shell: "{pre-build}"
- shell: !include-raw-escape: ../shell/cmake-sonar.sh
- lf-provide-maven-settings-cleanup
- publishers:
- - lf-infra-publish
-
- job-template:
name: "{project-name}-cmake-sonar"
id: gerrit-cmake-sonar
concurrent: false
+ <<: *lf_cmake_common
<<: *lf_cmake_sonar
scm:
name: "{project-name}-cmake-sonar"
id: github-cmake-sonar
concurrent: false
+ <<: *lf_cmake_common
<<: *lf_cmake_sonar
properties:
github-url: "https://github.com"
install-prefix: "$BUILD_DIR/output"
make-opts: ""
+ pre-build: ""
stream: master
submodule-recursive: true
submodule-timeout: 10
# Sonar properties
sonar-additional-args: ""
sonar-java-opts: ""
- sonar-prescan-script: ""
sonar-project-file: "sonar-project.properties"
sonar-properties: ""
sonar-task: ""
- #####################
- # Job Configuration #
- #####################
-
- project-type: freestyle
- node: "{build-node}"
- disabled: "{disable-job}"
-
- properties:
- - lf-infra-properties:
- build-days-to-keep: "{build-days-to-keep}"
-
- wrappers:
- - lf-infra-wrappers:
- build-timeout: "{build-timeout}"
- jenkins-ssh-credential: "{jenkins-ssh-credential}"
-
parameters:
- lf-infra-parameters:
project: "{project}"
builders:
- lf-infra-pre-build
- - shell: "{sonar-prescan-script}"
+ - shell: "{pre-build}"
- shell: !include-raw-escape: ../shell/cmake-sonarqube.sh
- lf-infra-sonar:
sonar-task: "{sonar-task}"
sonar-java-opts: "{sonar-java-opts}"
sonar-additional-args: "{sonar-additional-args}"
- publishers:
- - lf-infra-publish
-
- job-template:
name: "{project-name}-cmake-sonarqube"
id: gerrit-cmake-sonarqube
concurrent: false
+ <<: *lf_cmake_common
<<: *lf_cmake_sonarqube
+ gerrit_sonar_triggers:
+ - comment-added-contains-event:
+ comment-contains-value: '^Patch Set\s+\d+:\s+run-sonar\s*$'
+
scm:
- lf-infra-gerrit-scm:
branch: "$GERRIT_BRANCH"
- timed: "{obj:cron}"
- gerrit:
server-name: "{gerrit-server-name}"
- trigger-on:
- - comment-added-contains-event:
- comment-contains-value: '^Patch Set\s+\d+:\s+run-sonar\s*$'
+ trigger-on: "{obj:gerrit_sonar_triggers}"
projects:
- project-compare-type: "ANT"
project-pattern: "{project}"
name: "{project-name}-cmake-sonarqube"
id: github-cmake-sonarqube
concurrent: false
+ <<: *lf_cmake_common
<<: *lf_cmake_sonarqube
scm:
submodule-disable: false
version: ""
- #####################
- # Job Configuration #
- #####################
-
- project-type: freestyle
- node: "{build-node}"
- disabled: "{disable-job}"
-
- properties:
- - lf-infra-properties:
- build-days-to-keep: "{build-days-to-keep}"
-
parameters:
- lf-infra-parameters:
project: "{project}"
| awk '{{print $NF}}' | awk -F')' '{{print $1}}')"
echo "${{MAJOR_VERSION}}.${{MINOR_VERSION}}.${{PATCH_VERSION}}" > /tmp/artifact_version
- wrappers:
- - lf-infra-wrappers:
- build-timeout: "{build-timeout}"
- jenkins-ssh-credential: "{jenkins-ssh-credential}"
-
builders:
- lf-infra-pre-build
- lf-provide-maven-settings:
- shell: !include-raw-escape: ../shell/cmake-stage.sh
- lf-provide-maven-settings-cleanup
- publishers:
- - lf-infra-publish
-
- job-template:
name: "{project-name}-cmake-stage-{stream}"
id: gerrit-cmake-stage
concurrent: true
+ <<: *lf_cmake_common
<<: *lf_cmake_stage
scm:
name: "{project-name}-cmake-stage-{stream}"
id: github-cmake-stage
concurrent: true
+ <<: *lf_cmake_common
<<: *lf_cmake_stage
properties:
github_included_regions:
- ".*"
- #####################
- # Job Configuration #
- #####################
-
- project-type: freestyle
- node: "{build-node}"
- disabled: "{disable-job}"
-
- properties:
- - lf-infra-properties:
- build-days-to-keep: "{build-days-to-keep}"
-
- parameters:
- - lf-infra-parameters:
- project: "{project}"
- branch: "{branch}"
- stream: "{stream}"
- - lf-cmake-parameters:
- build-dir: "{build-dir}"
- cmake-opts: "{cmake-opts}"
- install-prefix: "{install-prefix}"
- make-opts: "{make-opts}"
-
- wrappers:
- - lf-infra-wrappers:
- build-timeout: "{build-timeout}"
- jenkins-ssh-credential: "{jenkins-ssh-credential}"
-
builders:
- shell: "{pre-build}"
- shell: !include-raw-escape: ../shell/cmake-build.sh
- publishers:
- - lf-infra-publish
-
- job-template:
name: "{project-name}-cmake-verify-{stream}"
id: gerrit-cmake-verify
concurrent: true
+ <<: *lf_cmake_common
<<: *lf_cmake_verify
scm:
name: "{project-name}-cmake-verify-{stream}"
id: github-cmake-verify
concurrent: true
+ <<: *lf_cmake_common
<<: *lf_cmake_verify
properties:
- "{branch}"
included-regions: "{obj:github_included_regions}"
+#################
+# Tox SonarQube #
+#################
+
+- lf_tox_sonarqube: &lf_tox_sonarqube
+ name: lf-tox-sonarqube
+
+ ######################
+ # Default parameters #
+ ######################
+
+ archive-artifacts: >
+ **/*.log
+ branch: master # Sonar should always be run on master branch
+ build-days-to-keep: 7
+ build-timeout: 15
+ cron: "@weekly"
+ disable-job: false
+ git-url: "$GIT_URL/$PROJECT"
+ github-url: "https://github.com"
+ parallel: false
+ pre-build-script: "# pre-build script goes here"
+ python-version: python3
+ stream: master
+ submodule-recursive: true
+ submodule-timeout: 10
+ submodule-disable: false
+ tox-dir: "."
+ tox-envs: ""
+ # Sonar properties
+ sonar-additional-args: ""
+ sonar-java-opts: ""
+ sonar-project-file: "sonar-project.properties"
+ sonar-properties: ""
+ sonar-task: ""
+
+ #####################
+ # Job Configuration #
+ #####################
+
+ project-type: freestyle
+ node: "{build-node}"
+ disabled: "{disable-job}"
+
+ properties:
+ - lf-infra-properties:
+ build-days-to-keep: "{build-days-to-keep}"
+
+ wrappers:
+ - lf-infra-wrappers:
+ build-timeout: "{build-timeout}"
+ jenkins-ssh-credential: "{jenkins-ssh-credential}"
+
+ parameters:
+ - lf-infra-parameters:
+ project: "{project}"
+ branch: "{branch}"
+ stream: "{stream}"
+ - lf-infra-tox-parameters:
+ tox-dir: "{tox-dir}"
+ tox-envs: "{tox-envs}"
+ - string:
+ name: ARCHIVE_ARTIFACTS
+ default: "{archive-artifacts}"
+ description: Artifacts to archive to the logs server.
+
+ builders:
+ - lf-infra-pre-build
+ - lf-infra-tox-install:
+ python-version: "{python-version}"
+ - shell: "{pre-build-script}"
+ - lf-infra-tox-run:
+ parallel: "{parallel}"
+ - lf-infra-sonar:
+ sonar-task: "{sonar-task}"
+ sonar-project-file: "{sonar-project-file}"
+ sonar-properties: "{sonar-properties}"
+ sonar-java-opts: "{sonar-java-opts}"
+ sonar-additional-args: "{sonar-additional-args}"
+
+ publishers:
+ - lf-infra-publish
+
+- job-template:
+ name: "{project-name}-tox-sonarqube"
+ id: gerrit-tox-sonarqube
+ concurrent: false
+ <<: *lf_tox_sonarqube
+
+ ######################
+ # Default parameters #
+ ######################
+
+ gerrit_sonar_triggers:
+ - comment-added-contains-event:
+ comment-contains-value: '^Patch Set\s+\d+:\s+run-sonar\s*$'
+
+ scm:
+ - lf-infra-gerrit-scm:
+ branch: "$GERRIT_BRANCH"
+ jenkins-ssh-credential: "{jenkins-ssh-credential}"
+ git-url: "{git-url}"
+ refspec: "$GERRIT_REFSPEC"
+ submodule-recursive: "{submodule-recursive}"
+ submodule-timeout: "{submodule-timeout}"
+ submodule-disable: "{submodule-disable}"
+ choosing-strategy: default
+
+ triggers:
+ - timed: "{obj:cron}"
+ - gerrit:
+ server-name: "{gerrit-server-name}"
+ trigger-on: "{obj:gerrit_sonar_triggers}"
+ projects:
+ - project-compare-type: "ANT"
+ project-pattern: "{project}"
+ branches:
+ - branch-compare-type: "ANT"
+ branch-pattern: "**/{branch}"
+
+- job-template:
+ name: "{project-name}-tox-sonarqube"
+ id: github-tox-sonarqube
+ concurrent: false
+ <<: *lf_tox_sonarqube
+
+ scm:
+ - lf-infra-github-scm:
+ branch: "$sha1"
+ jenkins-ssh-credential: "{jenkins-ssh-credential}"
+ url: "{git-clone-url}{github-org}/{project}"
+ refspec: "+refs/pull/*:refs/remotes/origin/pr/*"
+ submodule-recursive: "{submodule-recursive}"
+ submodule-timeout: "{submodule-timeout}"
+ submodule-disable: "{submodule-disable}"
+ choosing-strategy: default
+
+ triggers:
+ - github-pull-request:
+ trigger-phrase: "^run-sonar$"
+ only-trigger-phrase: false
+ status-context: "Tox Sonarqube"
+ permit-all: true
+ github-hooks: true
+ white-list-target-branches:
+ - "{branch}"
+
##############
# Tox Common #
##############
--- /dev/null
+---
+fixes:
+ - |
+ Update jq validation of returned json blocks to work properly with jq v1.5.
--- /dev/null
+---
+fixes:
+ - |
+ Refactor templates in lf-c-cpp-jobs.yaml with common anchor/alias
+ lf_cmake_common to reduce redundant configuration. This includes
+ gerrit-cmake-sonar, github-cmake-sonar, gerrit-cmake-sonarqube,
+ github-cmake-sonarqube, gerrit-cmake-stage, github-cmake-stage,
+ gerrit-cmake-verify and github-cmake-verify. No functional change.
+
+ Document maven settings parameters in g*t-cmake-stage templates.
--- /dev/null
+---
+fixes:
+ - |
+ Revise templates gerrit-cmake-sonarqube and gerrit-tox-sonarqube
+ to move the triggering comment string into a parameter that can be
+ overridden in a job definition. Github comment remains hardcoded.
+ The default is still "run-sonar".
+
+ Rename gerrit-cmake-sonarqube tempplate configuration parameter
+ from sonar-prescan-script to pre-build to be consistent with
+ existing CMake stage and verify templates. This is a breaking
+ change for any early adopters of this new template.
--- /dev/null
+---
+features:
+ - |
+ New templates gerrit-tox-sonarqube and github-tox-sonarqube
+ use the SonarQube Jenkins plug-in to analyze Python code and
+ publish the results. Modeled after the generic gerrit/github-sonar
+ templates. The new templates eliminate the need for mvn-settings in
+ the job configuration and the need for a pom.xml file in the repo.
json_block=$(curl -s "$url")
# check if JSON returned and can be parsed
-if jq <<< "$json_block" > /dev/null 2>&1; then
+if jq . <<< "$json_block" > /dev/null 2>&1; then
cost=$(jq .cost <<< "$json_block")
resource=$(jq .resource <<< "$json_block" | tr -d '"')
else
Code Review / releng / global-jjb.git / commitdiff