Add templates for CXX projects using autotools 77/63977/6
authorLott, Christopher (cl778h) <cl778h@att.com>
Wed, 13 May 2020 14:26:41 +0000 (10:26 -0400)
committerLott, Christopher (cl778h) <cl778h@att.com>
Wed, 13 May 2020 15:09:04 +0000 (11:09 -0400)
New jobs for the "configure && make && make install" pattern used
by many open-source projects. Supports generation of the configure
script for projects that do not store generated files in version
control. Includes gerrit and github versions of templates
autotools-packagecloud-stage, autotools-sonarqube and
autotools-verify.

Change-Id: I4e35db15c73f99b38486f31f137b224e8d6acde0
Signed-off-by: Lott, Christopher (cl778h) <cl778h@att.com>
.jjb-test/lf-c-cpp-jobs.yaml
docs/jjb/lf-c-cpp-jobs.rst
docs/jjb/lf-macros.rst
jjb/lf-c-cpp-jobs.yaml
jjb/lf-macros.yaml
releasenotes/notes/autotools-cxx-jobs-3cda2e30d6e2d4de.yaml [new file with mode: 0644]
shell/autotools-build.sh [new file with mode: 0644]
shell/autotools-sonarqube.sh [new file with mode: 0644]

index 6099d69..f3d4f49 100644 (file)
@@ -1,4 +1,32 @@
 ---
+- project:
+    name: gerrit-autotools-jobs
+    jobs:
+      - gerrit-autotools-packagecloud-stage
+      - gerrit-autotools-sonarqube
+      - gerrit-autotools-verify
+
+    project-name: gerrit-autotools
+    mvn-settings: gerrit-autotools-project-settings
+    debian-distribution-versions: ubuntu/bionic
+    packagecloud-account: example
+    packagecloud-repo: master
+    rpm-distribution-versions: el/5
+
+- project:
+    name: github-autotools-jobs
+    jobs:
+      - github-autotools-packagecloud-stage
+      - github-autotools-sonarqube
+      - github-autotools-verify
+
+    project-name: github-autotools
+    mvn-settings: github-autotools-project-settings
+    debian-distribution-versions: ubuntu/bionic
+    packagecloud-account: example
+    packagecloud-repo: master
+    rpm-distribution-versions: el/5
+
 - project:
     name: gerrit-cmake-jobs
     jobs:
index 8b6fe24..51e1bac 100644 (file)
@@ -5,6 +5,259 @@ C/C++ Jobs
 Job Templates
 =============
 
+Autotools PackageCloud Stage
+----------------------------
+
+Stage job which runs configure && make, then uploads all DEB/RPM files
+in the build directory to PackageCloud.io. Triggered by comment.
+
+The default configuration supplies a pre-build script that runs GNU
+Autotools to generate the configure shell script. Must be overridden
+if that script is in the version-control system.
+
+The Jenkins system must have a configuration file provider that
+installs files ".packagecloud" and "packagecloud_api" to the Jenkins
+home directory with appropriate credentials.
+
+The Jenkins build minion must have the Ruby gem "package_cloud"
+installed.
+
+:Template Names:
+
+    - {project-name}-autotools-packagecloud-stage-{stream}
+    - gerrit-autotools-packagecloud-stage
+    - github-autotools-packagecloud-stage
+
+:Comment Trigger: stage-release
+
+:Required parameters:
+
+    :build-node: The node to run build on.
+    :debian-distribution-versions: list of DEB package distro/version strings
+        separated by space; example: "ubuntu/bionic debian/stretch"
+    :jenkins-ssh-credential: Credential to use for SSH.
+        (Configure in defaults.yaml)
+    :packagecloud-account: PackageCloud account ID; example: oran
+    :packagecloud-repo: PackageCloud repository; example: master, staging
+    :project: The git repository name.
+    :project-name: Prefix used to name jobs.
+    :rpm-distribution-versions: list of RPM package distro/version strings
+        separated by space; example: "el/4 el/5"
+
+:Optional parameters:
+
+    :branch: Git branch to fetch for the build. (default: master)
+    :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
+    :build-dir: Directory with package files (default: $WORKSPACE)
+    :build-timeout: Timeout in minutes before aborting build. (default: 15)
+    :configure-opts: Parameters to pass to configure. (default: '')
+    :disable-job: Whether to disable the job (default: false)
+    :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+    :install: Install build products to /usr/local. (default: false)
+    :install-prefix: Path to use for install. (default: $BUILD_DIR/output)
+    :make-opts: Parameters and targets for make. (default: '')
+    :pre-build: Shell script to generate the configure file and install
+        dependencies. (default: 'autoreconf --install')
+    :stream: Keyword that to represent a release code-name.
+        Often the same as the branch. (default: master)
+    :submodule-recursive: Whether to checkout submodules recursively.
+        (default: true)
+    :submodule-timeout: Timeout (in minutes) for checkout operation.
+        (default: 10)
+    :submodule-disable: Disable submodule checkout operation.
+        (default: false)
+
+Autotools SonarQube
+-------------------
+
+The SonarQube job installs the SonarQube CXX build wrapper, runs
+configure, uses the build wrapper to invoke make, then runs the
+SonarQube Scanner Jenkins plug-in to analyze code for bugs, code
+smells and security vulnerabilities, and to upload the result
+(possibly including code-coverage statistics) to a SonarQube server or
+to SonarCloud.io. Optionally runs a shell script before the build to
+install prerequisites.
+
+The default configuration supplies a pre-build script that runs GNU
+Autotools to generate the configure shell script. Must be overridden
+if that script is in the version-control system.
+
+Requires ``SonarQube Scanner for Jenkins``
+
+This job runs on the master branch because the basic Sonar
+configuration does not support multi-branch.
+
+Plug-in configurations
+    Manage Jenkins --> Configure System --> SonarQube servers
+        - Name: Sonar (fixed)
+        - Server URL: https://sonar.project.org/ or https://sonarcloud.io
+        - Server authentication token: none for local, API token (saved as
+          a "secret text" credential) for Sonarcloud
+
+    Manage Jenkins --> Global Tool Configuration --> SonarQube Scanner
+        - Name: SonarQube Scanner (fixed)
+        - Install automatically
+        - Select latest version
+
+:Template Names:
+
+    - {project-name}-autotools-sonarqube
+    - gerrit-autotools-sonarqube
+    - github-autotools-sonarqube
+
+:Comment Trigger: ``run-sonar``
+
+:Required parameters:
+
+    :build-node: The node to run the build on.
+        (Commonly in defaults.yaml)
+    :jenkins-ssh-credential: Credential to use for SSH.
+        (Commonly in defaults.yaml)
+    :project: The git repository name.
+    :project-name: Prefix used to name jobs.
+
+.. comment Start ignoring WriteGoodLintBear
+
+:Optional Parameters:
+
+    :branch: Git branch to fetch for the build. (default: master)
+    :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
+    :build-timeout: Timeout in minutes before aborting build. (default: 15)
+    :build-wrap-dir: Build wrapper output subdirectory name.
+        (default: $WORKSPACE/bw-output)
+    :configure-opts: Parameters to pass to configure. (default: '')
+    :cron: Cron schedule when to trigger the job. This parameter also
+        supports multiline input via YAML pipe | character in cases where
+        one may want to provide more than 1 cron timer.  (default: @weekly)
+    :disable-job: Whether to disable the job (default: false)
+    :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+    :install-prefix: Path to use for install. (default: $BUILD_DIR/output)
+    :make-opts: Parameters and targets for make. (default: '')
+    :pre-build: Shell script to generate the configure file and install
+        dependencies. (default: 'autoreconf --install')
+    :sonar-additional-args: Command line arguments. (default: '')
+    :sonar-java-opts: JVM options. For example, use option -Xmx
+        to increase the memory size limit.  (default: '')
+    :sonar-project-file: The file name with Sonar configuration properties
+        (default: sonar-project.properties)
+    :sonar-properties: Sonar configuration properties. (default: '')
+    :sonar-task: Sonar task to run. (default: '')
+    :stream: Keyword that to represent a release code-name.
+        Often the same as the branch. (default: master)
+    :submodule-recursive: Whether to checkout submodules recursively.
+        (default: true)
+    :submodule-timeout: Timeout (in minutes) for checkout operation.
+        (default: 10)
+    :submodule-disable: Disable submodule checkout operation.
+        (default: false)
+
+.. comment Stop ignoring
+
+.. note:: A job definition must provide one of the optional parameters
+    ``sonar-project-file`` and ``sonar-properties``; they cannot both be
+    empty.  Set Sonar properties directly in the job definition by setting
+    the ``sonar-project-file`` property to ``""`` and adding all properties
+    under ``sonar-properties``.
+
+:Required Sonar Properties:
+
+    - sonar.login: The API token for authentication at SonarCloud.
+      Commonly defined as key "sonarcloud_api_token" in defaults.yaml.
+    - sonar.organization: The umbrella project name; e.g., "opendaylight".
+      Commonly defined as key "sonarcloud_project_organization" in defaults.yaml.
+    - sonar.projectName: The git repository name without slashes; e.g., "infrautils".
+    - sonar.projectKey: The globally unique key for the report in SonarCloud. Most
+      teams use the catenation of sonar.organization, an underscore, and
+      sonar.projectName; e.g., "opendaylight_infrautils".
+
+:Optional Sonar Properties:
+
+    - sonar.cfamily.gcov.reportsPath: directory with GCOV output files
+    - Documentation of SonarQube properties is here:
+      https://docs.sonarqube.org/latest/analysis/overview/
+
+
+Example job definition
+^^^^^^^^^^^^^^^^^^^^^^
+
+The following example defines a job for a project with CXX source files
+in the "src" directory, and unit tests that write coverage files in
+GCOV format to the "test" directory. This definition uses configuration
+parameters in the umbrella project's defaults.yaml file.
+
+.. code-block:: yaml
+
+    - project:
+        name: my-project-sonar
+        project: my/project
+        project-name: my-project
+        sonar-project-file: ""
+        sonar-properties: |
+            sonar.login={sonarcloud_api_token}
+            sonar.projectKey={sonarcloud_project_organization}_{project-name}
+            sonar.projectName={project-name}
+            sonar.organization={sonarcloud_project_organization}
+            sonar.sourceEncoding=UTF-8
+            sonar.sources=src
+            sonar.cfamily.build-wrapper-output=$WORKSPACE/bw-output
+            sonar.cfamily.gcov.reportsPath=test
+        jobs:
+          - gerrit-autotools-sonarqube
+
+Autotools Verify
+----------------
+
+Verify job which runs configure && make to test a project build, then
+optionally runs make install, copies the build products to /usr/local
+and runs ldconfig to make the shared lib(s) available. The install steps
+run by default, see parameter "install".
+
+The default configuration supplies a pre-build script that runs GNU
+Autotools to generate the configure shell script. Must be overridden
+if the script is in the version-control system.
+
+:Template Names:
+
+    - {project-name}-autotools-verify-{stream}
+    - gerrit-autotools-verify
+    - github-autotools-verify
+
+:Comment Trigger: recheck|reverify
+
+:Required parameters:
+
+    :build-node: The node to run build on.
+    :jenkins-ssh-credential: Credential to use for SSH.
+        (Configure in defaults.yaml)
+    :project: The git repository name.
+    :project-name: Prefix used to name jobs.
+
+:Optional parameters:
+
+    :branch: Git branch to fetch for the build. (default: master)
+    :build-days-to-keep: Days to keep build logs in Jenkins. (default: 7)
+    :build-timeout: Timeout in minutes before aborting build. (default: 15)
+    :configure-opts: Parameters to pass to configure. (default: '')
+    :disable-job: Whether to disable the job (default: false)
+    :git-url: URL clone project from. (default: $GIT_URL/$PROJECT)
+    :install: Install build products to /usr/local. (default: true)
+    :install-prefix: Path to use for install. (default: $BUILD_DIR/output)
+    :make-opts: Parameters and targets for make. (default: 'test')
+    :pre-build: Shell script to generate the configure file and install
+        dependencies. (default: 'autoreconf --install')
+    :stream: Keyword that to represent a release code-name.
+        Often the same as the branch. (default: master)
+    :submodule-recursive: Whether to checkout submodules recursively.
+        (default: true)
+    :submodule-timeout: Timeout (in minutes) for checkout operation.
+        (default: 10)
+    :submodule-disable: Disable submodule checkout operation.
+        (default: false)
+
+    :gerrit_verify_triggers: Override Gerrit Triggers.
+    :gerrit_trigger_file_paths: Override file paths which to filter which file
+        modifications will trigger a build.
+
 CMake Sonar
 -----------
 
index 357ecb6..f216912 100644 (file)
@@ -372,8 +372,14 @@ Requires ``SonarQube Scanner for Jenkins``
 Parameters
 ==========
 
+lf-autotools-parameters
+-----------------------
+
+Provides parameters needed by configure and make. Should be used by any jobs
+that need to call the ``configure && make`` pattern.
+
 lf-clm-parameters
--------------------
+-----------------
 
 Provides the policy evaluation stage to run against Nexus IQ Server.
 Valid values include: 'build', 'stage-release', 'operate'.
index 115eac7..d1ba86f 100644 (file)
@@ -1,8 +1,437 @@
 ---
 ####################
-# COMMON FUNCTIONS #
+# AUTOTOOLS COMMON #
 ####################
 
+- lf_autotools_common: &lf_autotools_common
+    name: lf-autotools-common
+
+    #####################
+    # Job Configuration #
+    #####################
+
+    project-type: freestyle
+    node: "{build-node}"
+    disabled: "{disable-job}"
+
+    properties:
+      - lf-infra-properties:
+          build-days-to-keep: "{build-days-to-keep}"
+
+    parameters:
+      - lf-infra-parameters:
+          project: "{project}"
+          branch: "{branch}"
+          stream: "{stream}"
+      - lf-autotools-parameters:
+          configure-opts: "{configure-opts}"
+          install-prefix: "{install-prefix}"
+          make-opts: "{make-opts}"
+
+    wrappers:
+      - lf-infra-wrappers:
+          build-timeout: "{build-timeout}"
+          jenkins-ssh-credential: "{jenkins-ssh-credential}"
+
+    publishers:
+      - lf-infra-publish
+
+################################
+# AUTOTOOLS PACKAGECLOUD STAGE #
+################################
+
+- lf_autotools_packagecloud_stage: &lf_autotools_packagecloud_stage
+    name: lf-autotools-packagecloud-stage
+
+    ######################
+    # Default parameters #
+    ######################
+
+    branch: master
+    build-days-to-keep: 7
+    build-dir: "$WORKSPACE" # directory with deb/rpm files
+    build-timeout: 15
+    configure-opts: ""
+    disable-job: false
+    git-url: "$GIT_URL/$PROJECT"
+    github-url: "https://github.com"
+    install: false
+    install-prefix: $WORKSPACE/output
+    make-opts: "" # target that creates deb/rpm files
+    pre-build: | # default script to generate configure file
+      autoreconf --install
+    stream: master
+    submodule-recursive: true
+    submodule-timeout: 10
+    submodule-disable: false
+
+    gerrit_trigger_file_paths:
+      - compare-type: REG_EXP
+        pattern: ".*"
+
+    # github_included_regions MUST match gerrit_trigger_file_paths
+    github_included_regions:
+      - ".*"
+
+    parameters:
+      - lf-infra-parameters:
+          project: "{project}"
+          branch: "{branch}"
+          stream: "{stream}"
+      - lf-autotools-parameters:
+          configure-opts: "{configure-opts}"
+          install-prefix: "{install-prefix}"
+          make-opts: "{make-opts}"
+      - bool:
+          name: INSTALL
+          default: "{install}"
+          description: "Install build products to /usr/local (true/false)."
+
+    builders:
+      - config-file-provider:
+          files:
+            - file-id: ".packagecloud"
+              target: "$HOME/.packagecloud"
+      - shell: "{pre-build}"
+      - shell: !include-raw-escape: ../shell/autotools-build.sh
+      - lf-packagecloud-push:
+          build-dir: "{build-dir}"
+          packagecloud-account: "{packagecloud-account}"
+          packagecloud-repo: "{packagecloud-repo}"
+          debian-distribution-versions: "{debian-distribution-versions}"
+          rpm-distribution-versions: "{rpm-distribution-versions}"
+
+- job-template:
+    # build with configure/make and publish to PackageCloud on command
+    name: "{project-name}-autotools-packagecloud-stage-{stream}"
+    id: gerrit-autotools-packagecloud-stage
+    <<: *lf_autotools_common
+    <<: *lf_autotools_packagecloud_stage
+
+    ######################
+    # Default parameters #
+    ######################
+
+    gerrit_stage_triggers:
+      - comment-added-contains-event:
+          comment-contains-value: '^Patch Set\s+\d+:\s+stage-release\s*$'
+
+    #####################
+    # Job Configuration #
+    #####################
+
+    scm:
+      - lf-infra-gerrit-scm:
+          jenkins-ssh-credential: "{jenkins-ssh-credential}"
+          git-url: "{git-url}"
+          refspec: "$GERRIT_REFSPEC"
+          branch: "$GERRIT_BRANCH"
+          submodule-recursive: "{submodule-recursive}"
+          submodule-timeout: "{submodule-timeout}"
+          submodule-disable: "{submodule-disable}"
+          # stage jobs always build from tip
+          choosing-strategy: default
+
+    triggers:
+      - gerrit:
+          server-name: "{gerrit-server-name}"
+          trigger-on: "{obj:gerrit_stage_triggers}"
+          projects:
+            - project-compare-type: ANT
+              project-pattern: "{project}"
+              branches:
+                - branch-compare-type: ANT
+                  branch-pattern: "**/{branch}"
+              file-paths: "{obj:gerrit_trigger_file_paths}"
+
+- job-template:
+    name: "{project-name}-autotools-packagecloud-stage-{stream}"
+    id: github-autotools-packagecloud-stage
+    concurrent: true
+    <<: *lf_autotools_common
+    <<: *lf_autotools_packagecloud_stage
+
+    properties:
+      - lf-infra-properties:
+          build-days-to-keep: "{build-days-to-keep}"
+      - github:
+          url: "{github-url}/{github-org}/{project}"
+
+    scm:
+      - lf-infra-github-scm:
+          url: "{git-clone-url}{github-org}/{project}"
+          refspec: "+refs/pull/*:refs/remotes/origin/pr/*"
+          branch: "$sha1"
+          submodule-recursive: "{submodule-recursive}"
+          submodule-timeout: "{submodule-timeout}"
+          submodule-disable: "{submodule-disable}"
+          choosing-strategy: default
+          jenkins-ssh-credential: "{jenkins-ssh-credential}"
+
+    triggers:
+      - github-pull-request:
+          trigger-phrase: "^(stage-release)$"
+          only-trigger-phrase: true
+          status-context: "Autotools PackageCloud Stage"
+          permit-all: true
+          github-hooks: true
+          white-list-target-branches:
+            - "{branch}"
+
+#######################
+# AUTOTOOLS SONARQUBE #
+#######################
+
+- lf_autotools_sonarqube: &lf_autotools_sonarqube
+    name: lf-autotools-sonarqube
+
+    ######################
+    # Default parameters #
+    ######################
+
+    branch: master # Sonar should always be run on master branch
+    build-days-to-keep: 7
+    build-timeout: 15
+    build-wrap-dir: $WORKSPACE/bw-output
+    configure-opts: CXXFLAGS="-O0 --coverage" LDFLAGS=--coverage # gcc coverage options
+    cron: "@weekly"
+    disable-job: false
+    git-url: "$GIT_URL/$PROJECT"
+    github-url: "https://github.com"
+    install-prefix: $WORKSPACE/output
+    make-opts: "" # target that tests and generates gcov files
+    pre-build: | # default script to generate configure file
+      autoreconf --install
+    stream: master
+    submodule-recursive: true
+    submodule-timeout: 10
+    submodule-disable: false
+
+    # Sonar properties
+    sonar-additional-args: ""
+    sonar-java-opts: ""
+    sonar-project-file: sonar-project.properties
+    sonar-properties: ""
+    sonar-task: ""
+
+    parameters:
+      - lf-infra-parameters:
+          project: "{project}"
+          branch: "{branch}"
+          stream: "{stream}"
+      - lf-autotools-parameters:
+          configure-opts: "{configure-opts}"
+          install-prefix: "{install-prefix}"
+          make-opts: "{make-opts}"
+      - string:
+          name: BUILD_WRAP_DIR
+          default: "{build-wrap-dir}"
+          description: SonarQube build wrapper output directory
+
+    builders:
+      - shell: "{pre-build}"
+      - shell: !include-raw-escape: ../shell/autotools-sonarqube.sh
+      - lf-infra-sonar:
+          sonar-task: "{sonar-task}"
+          sonar-project-file: "{sonar-project-file}"
+          sonar-properties: "{sonar-properties}"
+          sonar-java-opts: "{sonar-java-opts}"
+          sonar-additional-args: "{sonar-additional-args}"
+
+- job-template:
+    name: "{project-name}-autotools-sonarqube"
+    id: gerrit-autotools-sonarqube
+    concurrent: false
+    <<: *lf_autotools_common
+    <<: *lf_autotools_sonarqube
+
+    gerrit_sonar_triggers:
+      - comment-added-contains-event:
+          comment-contains-value: '^Patch Set\s+\d+:\s+run-sonar\s*$'
+
+    scm:
+      - lf-infra-gerrit-scm:
+          branch: "$GERRIT_BRANCH"
+          jenkins-ssh-credential: "{jenkins-ssh-credential}"
+          git-url: "{git-url}"
+          refspec: "$GERRIT_REFSPEC"
+          submodule-recursive: "{submodule-recursive}"
+          submodule-timeout: "{submodule-timeout}"
+          submodule-disable: "{submodule-disable}"
+          choosing-strategy: default
+
+    triggers:
+      - timed: "{obj:cron}"
+      - gerrit:
+          server-name: "{gerrit-server-name}"
+          trigger-on: "{obj:gerrit_sonar_triggers}"
+          projects:
+            - project-compare-type: "ANT"
+              project-pattern: "{project}"
+              branches:
+                - branch-compare-type: "ANT"
+                  branch-pattern: "**/{branch}"
+
+- job-template:
+    name: "{project-name}-autotools-sonarqube"
+    id: github-autotools-sonarqube
+    concurrent: false
+    <<: *lf_autotools_common
+    <<: *lf_autotools_sonarqube
+
+    scm:
+      - lf-infra-github-scm:
+          branch: "$sha1"
+          jenkins-ssh-credential: "{jenkins-ssh-credential}"
+          url: "{git-clone-url}{github-org}/{project}"
+          refspec: "+refs/pull/*:refs/remotes/origin/pr/*"
+          submodule-recursive: "{submodule-recursive}"
+          submodule-timeout: "{submodule-timeout}"
+          submodule-disable: "{submodule-disable}"
+          choosing-strategy: default
+
+    triggers:
+      - github-pull-request:
+          trigger-phrase: "^run-sonar$"
+          only-trigger-phrase: false
+          status-context: "Autotools Sonarqube"
+          permit-all: true
+          github-hooks: true
+          white-list-target-branches:
+            - "{branch}"
+
+####################
+# AUTOTOOLS VERIFY #
+####################
+
+- lf_autotools_verify: &lf_autotools_verify
+    name: lf-autotools-verify
+
+    ######################
+    # Default parameters #
+    ######################
+
+    branch: master
+    build-days-to-keep: 7
+    build-timeout: 15
+    configure-opts: ""
+    disable-job: false
+    git-url: "$GIT_URL/$PROJECT"
+    github-url: "https://github.com"
+    install: false
+    install-prefix: $WORKSPACE/output
+    make-opts: "test" # target that compiles and tests
+    pre-build: | # default script to generate configure file
+      autoreconf --install
+    stream: master
+    submodule-recursive: true
+    submodule-timeout: 10
+    submodule-disable: false
+
+    gerrit_verify_triggers:
+      - patchset-created-event:
+          exclude-drafts: true
+          exclude-trivial-rebase: false
+          exclude-no-code-change: false
+      - draft-published-event
+      - comment-added-contains-event:
+          comment-contains-value: '^Patch Set\s+\d+:\s+(recheck|reverify)\s*$'
+
+    gerrit_trigger_file_paths:
+      - compare-type: REG_EXP
+        pattern: ".*"
+
+    # github_included_regions MUST match gerrit_trigger_file_paths
+    github_included_regions:
+      - ".*"
+
+    parameters:
+      - lf-infra-parameters:
+          project: "{project}"
+          branch: "{branch}"
+          stream: "{stream}"
+      - lf-autotools-parameters:
+          configure-opts: "{configure-opts}"
+          install-prefix: "{install-prefix}"
+          make-opts: "{make-opts}"
+      - bool:
+          name: INSTALL
+          default: "{install}"
+          description: "Install build products to /usr/local (true/false)."
+
+    builders:
+      - shell: "{pre-build}"
+      - shell: !include-raw-escape: ../shell/autotools-build.sh
+
+- job-template:
+    name: "{project-name}-autotools-verify-{stream}"
+    id: gerrit-autotools-verify
+    concurrent: true
+    <<: *lf_autotools_common
+    <<: *lf_autotools_verify
+
+    scm:
+      - lf-infra-gerrit-scm:
+          branch: "$GERRIT_BRANCH"
+          jenkins-ssh-credential: "{jenkins-ssh-credential}"
+          git-url: "{git-url}"
+          refspec: "$GERRIT_REFSPEC"
+          submodule-recursive: "{submodule-recursive}"
+          submodule-timeout: "{submodule-timeout}"
+          submodule-disable: "{submodule-disable}"
+          choosing-strategy: gerrit
+
+    triggers:
+      - gerrit:
+          server-name: "{gerrit-server-name}"
+          trigger-on: "{obj:gerrit_verify_triggers}"
+          projects:
+            - project-compare-type: "ANT"
+              project-pattern: "{project}"
+              branches:
+                - branch-compare-type: "ANT"
+                  branch-pattern: "**/{branch}"
+              file-paths: "{obj:gerrit_trigger_file_paths}"
+
+- job-template:
+    name: "{project-name}-autotools-verify-{stream}"
+    id: github-autotools-verify
+    concurrent: true
+    <<: *lf_autotools_common
+    <<: *lf_autotools_verify
+
+    properties:
+      - lf-infra-properties:
+          build-days-to-keep: "{build-days-to-keep}"
+      - github:
+          url: "{github-url}/{github-org}/{project}"
+
+    scm:
+      - lf-infra-github-scm:
+          url: "{git-clone-url}{github-org}/{project}"
+          refspec: "+refs/pull/*:refs/remotes/origin/pr/*"
+          branch: "$sha1"
+          submodule-recursive: "{submodule-recursive}"
+          submodule-timeout: "{submodule-timeout}"
+          submodule-disable: "{submodule-disable}"
+          choosing-strategy: default
+          jenkins-ssh-credential: "{jenkins-ssh-credential}"
+
+    triggers:
+      - github-pull-request:
+          trigger-phrase: "^(recheck|reverify)$"
+          only-trigger-phrase: false
+          status-context: "Autotools Verify"
+          permit-all: true
+          github-hooks: true
+          included-regions: "{obj:github_included_regions}"
+          white-list-target-branches:
+            - "{branch}"
+
+################
+# CMAKE COMMON #
+################
+
 - lf_cmake_common: &lf_cmake_common
     name: lf-cmake-common
 
           white-list-target-branches:
             - "{branch}"
 
-##################
-# CMAKE+PC STAGE #
-##################
+############################
+# CMAKE PACKAGECLOUD STAGE #
+############################
 
 - lf_cmake_pc_stage: &lf_cmake_pc_stage
     name: lf-cmake-packagecloud-stage
index ddc9f01..0db15aa 100644 (file)
 # PARAMETERS #
 ##############
 
+- parameter:
+    name: lf-autotools-parameters
+    # Parameters for configure-make builds.
+    parameters:
+      - string:
+          name: CONFIGURE_OPTS
+          default: "{configure-opts}"
+          description: |
+            Options to pass to configure.
+            Example: --build=i686-pc-linux-gnu
+      - string:
+          name: INSTALL_PREFIX
+          default: "{install-prefix}"
+          description: |
+            Install prefix for configure.
+            Usually a workspace subdirectory.
+      - string:
+          name: MAKE_OPTS
+          default: "{make-opts}"
+          description: |
+            Options to pass to make.
+            Example: -j8 all
+
 - parameter:
     name: lf-clm-parameters
     parameters:
diff --git a/releasenotes/notes/autotools-cxx-jobs-3cda2e30d6e2d4de.yaml b/releasenotes/notes/autotools-cxx-jobs-3cda2e30d6e2d4de.yaml
new file mode 100644 (file)
index 0000000..9fef198
--- /dev/null
@@ -0,0 +1,10 @@
+---
+features:
+  - |
+    Add new templates to build CXX projects with GNU autotools.
+    These support the "configure && make && make install"
+    pattern used by many open-source projects. Supports
+    generation of the configure script for projects that do
+    not store generated files in version control. Includes
+    gerrit and github versions of autotools-packagecloud-stage,
+    autotools-sonarqube and autotools-verify.
diff --git a/shell/autotools-build.sh b/shell/autotools-build.sh
new file mode 100644 (file)
index 0000000..4101380
--- /dev/null
@@ -0,0 +1,67 @@
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2020 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+
+# Invokes configure and make with the specified options.
+# Optionally runs make install and tars up all files from the install prefix,
+# then uses sudo to extract those files to /usr/local and runs ldconfig,
+# leaving shared lib(s) ready for use.
+# Prereqs:
+# The build minion has make, gcc etc.
+# The project repo has an executable shell script "configure"
+# Environment variables:
+# WORKSPACE is a non-empty path (required)
+# INSTALL_PREFIX is a non-empty path (required)
+# PROJECT is a non-empty name (required)
+# CONFIGURE_OPTS has options for configure (optional, empty default)
+# MAKE_OPTS has options for make (optional, empty default)
+# INSTALL is "true" or "false" (optional, default false)
+
+echo "---> autotools-build.sh"
+
+# be careful and verbose
+set -eux -o pipefail
+
+c="$WORKSPACE/configure"
+if [[ ! -f $c || ! -x $c ]]; then
+    echo "ERROR: failed to find executable file $c"
+    exit 1
+fi
+
+configure_opts="${CONFIGURE_OPTS:-}"
+make_opts="${MAKE_OPTS:-}"
+install="${INSTALL:-false}"
+# Not a misspelling as shellcheck reports.
+# shellcheck disable=SC2153
+project="${PROJECT//\//\-}"
+
+# use eval to disable bash quoting behavior;
+# e.g., if configure-opts=CXXFLAGS="-O0 --coverage"
+# configure needs to wordsplit to pass options
+# shellcheck disable=SC2086
+eval $c --prefix="$INSTALL_PREFIX" $configure_opts
+
+# show make version to assist debugging
+make -version
+# use eval to disable bash quoting behavior
+# $make_opts may be empty
+# make needs to wordsplit to pass options
+# shellcheck disable=SC2086
+eval make $make_opts
+
+if [[ $install == true ]]; then
+    make install
+    mkdir -p "$WORKSPACE/dist"
+    tar -cJvf "$WORKSPACE/dist/$project.tar.xz" -C "$INSTALL_PREFIX" .
+    sudo tar -xvf "$WORKSPACE/dist/$project.tar.xz" -C "/usr/local"
+    sudo ldconfig
+fi
+
+echo "---> autotools-build.sh ends"
diff --git a/shell/autotools-sonarqube.sh b/shell/autotools-sonarqube.sh
new file mode 100644 (file)
index 0000000..c52b9bb
--- /dev/null
@@ -0,0 +1,61 @@
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2020 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+
+# Invokes configure and make with the specified options,
+# using a wrapper for the SonarQube Jenkins plug-in
+# https://docs.sonarqube.org/latest/analysis/languages/cfamily/
+# Prereqs:
+# The build minion has make, gcc etc.
+# The project repo has an executable shell script "configure"
+# Environment variables:
+# WORKSPACE is a non-empty path (required)
+# INSTALL_PREFIX is a non-empty path (required)
+# CONFIGURE_OPTS has options for configure (optional, empty default)
+# MAKE_OPTS has options for make (optional, empty default)
+# BUILD_WRAP_DIR is a path (optional, this provides a usable default)
+
+echo "---> autotools-sonarqube.sh"
+
+# be careful and verbose
+set -eux -o pipefail
+
+c="$WORKSPACE/configure"
+if [[ ! -f $c || ! -x $c ]]; then
+    echo "ERROR: failed to find executable file $c"
+    exit 1
+fi
+
+configure_opts="${CONFIGURE_OPTS:-}"
+make_opts="${MAKE_OPTS:-}"
+build_wrap_dir="${BUILD_WRAP_DIR:-$WORKSPACE/bw-output}"
+
+# download and install the Sonar build wrapper
+bw=bw.zip
+wget -q -O "$bw" https://sonarcloud.io/static/cpp/build-wrapper-linux-x86.zip
+unzip -q "$bw"
+sudo mv build-wrapper-* /opt/build-wrapper
+rm -f "$bw"
+
+# use eval to disable bash quoting behavior;
+# e.g., if configure-opts=CXXFLAGS="-O0 --coverage"
+# configure needs to wordsplit to pass options
+# shellcheck disable=SC2086
+eval $c --prefix="$INSTALL_PREFIX" $configure_opts
+
+# to analyze coverage, make must run tests and run gcov
+# to process .gcno/.gcda files into .gcov files.
+# use eval to disable bash quoting behavior
+# $make_opts may be empty
+# make needs to wordsplit to pass options
+# shellcheck disable=SC2086
+eval /opt/build-wrapper/build-wrapper-linux-x86-64 --out-dir "$build_wrap_dir" make $make_opts
+
+echo "---> autotools-sonarqube.sh ends"