From 5aaa1c1de410b0489ff7ae8d8b26ebf9f40afbee Mon Sep 17 00:00:00 2001 From: Thanh Ha Date: Wed, 30 May 2018 21:02:57 -0400 Subject: [PATCH] Migrate README to RTD Migrate the contents of the README (legacy) into RTD so that we can properly link to the data there through Sphinx. Issue: RELENG-981 Change-Id: I22b21eddb008a6f0c98c3f1ac3d3543372b72752 Signed-off-by: Thanh Ha --- docs/appendix.rst | 11 +++ docs/configuration.rst | 94 +++++++++++++++++++ docs/glossary.rst | 18 ++++ docs/index.rst | 6 +- docs/install.rst | 236 ++++++++++++++++++++++++++++++++++++++++++++++++ docs/jjb/lf-ci-jobs.rst | 2 + 6 files changed, 366 insertions(+), 1 deletion(-) create mode 100644 docs/appendix.rst create mode 100644 docs/glossary.rst create mode 100644 docs/install.rst diff --git a/docs/appendix.rst b/docs/appendix.rst new file mode 100644 index 00000000..185acf2e --- /dev/null +++ b/docs/appendix.rst @@ -0,0 +1,11 @@ +######## +Appendix +######## + +ShellCheck +========== + +When using ShellCheck to lint global-jjb or any projects that include +global-jjb as part of their project (common with ci-management repos) then +we require version 0.4.x of ShellCheck installed on the build vms. This version +introduces annotations used by shell scripts in this repo. diff --git a/docs/configuration.rst b/docs/configuration.rst index e212d9ce..7fc6cede 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -4,6 +4,93 @@ Configuration ############# +.. _defaults-yaml: + +defaults.yaml +============= + +This file lives in the ci-management repo typically under the path +``jjb/defaults.yaml``. The purpose of this file is to store default variable +values used by global-jjb templates. + +**Required** + +:jenkins-ssh-credential: The name of the Jenkins Credential to + use for ssh connections. (ex: jenkins-ssh) + +:lftools-version: Version of lftools to install. Can be a specific version + like '0.6.1' or a `PEP-440 definition `_ + For example `<1.0.0` or `>=1.0.0,<2.0.0`. + +:mvn-site-id: Maven Server ID from settings.xml containing the credentials + to push to a Maven site repository. + +:mvn-staging-id: Maven Server ID from settings.xml containing the credentials + to push to a Maven staging repository. + +**Gerrit required parameters**: + +:gerrit-server-name: The name of the Gerrit Server as defined in Gerrit + Trigger global configuration. (ex: Primary) + +**GitHub required parameters**: + +:git-url: Set this to the base URL of your GitHub repo. In + general this should be . If you are using + GitHub Enterprise, or some other GitHub-style system, then it + should be whatever your installation base URL is. + +:git-clone-url: This is the clone prefix used by GitHub jobs. + Set this to either the same thing as **git-url** or the + 'git@github.com:' including the trailing ':' + +:github-org: The name of the GitHub organization interpolated + into the scm config. + +:github_pr_org: The name of the GitHub organization. All members + of this organization will be able to trigger any job using the + `lf-infra-github-pr` macro. + +:github_pr_whitelist: List of GitHub members you wish to be able to + trigger any job that uses the `lf-infra-github-pr-trigger` macro. + +:github_pr_admin_list: List of GitHub members that will have admin + privileges on any job using the `lf-infra-github-pr-trigger` macro. + +Example Gerrit Infra: + +.. code-block:: yaml + + - defaults: + name: global + + # lf-infra defaults + jenkins-ssh-credential: opendaylight-jenkins-ssh + gerrit-server-name: OpenDaylight + lftools-version: '<1.0.0' + mvn-site-id: opendaylight-site + mvn-staging-id: opendaylight-staging + +Example GitHub Infra: + +.. code-block:: yaml + + - defaults: + name: global + + # lf-infra defaults + jenkins-ssh-credential: jenkins-ssh + github-org: lfit + github_pr_whitelist: + - jpwku + - tykeal + - zxiiro + github_pr_admin_list: + - tykeal + lftools-version: '<1.0.0' + mvn-site-id: opendaylight-site + mvn-staging-id: opendaylight-staging + Jenkins Files ============= @@ -57,3 +144,10 @@ jenkins-cfg-merge This job manages Jenkins Global configuration. Refer to the :ref:`CI Documentation ` for job configuration details. + +Log Archiving +============= + +The logs account requires a Maven Settings file created called +**jenkins-log-archives-settings** with a server ID of **logs** containing the +credentials for the logs user in Nexus. diff --git a/docs/glossary.rst b/docs/glossary.rst new file mode 100644 index 00000000..a0e001dc --- /dev/null +++ b/docs/glossary.rst @@ -0,0 +1,18 @@ +######## +Glossary +######## + +.. glossary:: + + ciman + Short for :term:`ci-management`. + + ci-management + Refers to the SCM repository containing the :term:`JJB` configuration + files. In most LF Projects this is the repository named + ``ci-management``, but is ``releng/builder`` in the OpenDaylight project + and ``releng`` in the OPNFV project. + + JJB + Short for Jenkins Job Builder (JJB) a tool used to convert YAML + definitions into XML as a way to define Jenkins job configuration. diff --git a/docs/index.rst b/docs/index.rst index 0eec76f5..c9b27a58 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -26,9 +26,13 @@ Guides .. toctree:: :maxdepth: 2 - best-practices + install configuration + best-practices + glossary + appendix + Global JJB Templates -------------------- diff --git a/docs/install.rst b/docs/install.rst new file mode 100644 index 00000000..fb7a649e --- /dev/null +++ b/docs/install.rst @@ -0,0 +1,236 @@ +####### +Install +####### + +global-jjb requires configuration in 2 places; ``Jenkins`` and the +:term:`ci-management` repository. + +.. _jenkins-config: + +Jenkins configuration +===================== + +On the Jenkins side, we need to prep ``environment variables`` and +``plugins`` required by the jobs in global-jjb before we can start our first +jobs. + +.. _jenkins-install-plugins: + +Install Jenkins plugins +----------------------- + +Install the following required Jenkins plugins and any optional ones as +necessary by the project. + +**Required** + +- `Config File Provider `_ +- `Description Setter `_ +- `Environment Injector Plugin `_ +- `Git plugin `_ +- `Post Build Script `_ +- `SSH Agent `_ +- `Workspace Cleanup `_ + +**Required for Gerrit connected systems** + +- `Gerrit Trigger `_ + +**Required for GitHub connected systems** + +- `GitHub plugin `_ +- `GitHub Pull Request Builder `_ + +**Optional** + +- `Mask Passwords `_ +- `MsgInject `_ +- `OpenStack Cloud `_ +- `Timestamper `_ + +.. _jenkins-envvars: + +Environment Variables +--------------------- + +The :ref:`lf-global-jjb-jenkins-cfg-merge` job can manage environment variables +job but we must first bootstrap them in Jenkins so that the job can run and +take over. + +**Required**:: + + GIT_URL=ssh://jenkins-$SILO@git.opendaylight.org:29418 + JENKINS_HOSTNAME=jenkins092 + NEXUS_URL=https://nexus.opendaylight.org + SILO=production + SONAR_URL=https://sonar.opendaylight.org + +**Gerrit**:: + + GERRIT_URL=https://git.opendaylight.org/gerrit + +**GitHub**:: + + GIT_URL=https://github.com + GIT_CLONE_URL=git@github.com: + +.. note:: + + Use ``GIT_CLONE_URL`` for GitHub projects as this will be different from the + URL used in the properties configuration. + +**Optional**:: + + LOGS_SERVER=https://logs.opendaylight.org + +Steps + +#. Navigate to https://jenkins.example.org/configure +#. Configure the environment variables as described above +#. Configure the same environment variables in the :term:`ci-management` repo + +.. _jenkins-ci-management: + +ci-management +============= + +:term:`ci-management` is a git repository containing :term:`JJB` configuration +files for Jenkins Jobs. Deploying Global JJB here as a submodule allows us easy +management to install, upgrade, and rollback changes via git tags. Install +Global JJB as follows: + +#. Install Global JJB + + .. code-block:: bash + + GLOBAL_JJB_VERSION=v0.1.0 + git submodule add https://github.com/lfit/releng-global-jjb.git jjb/global-jjb + cd jjb/global-jjb + git checkout $GLOBAL_JJB_VERSION + cd ../.. + git add jjb/global-jjb + git commit -sm "Install global-jjb $GLOBAL_JJB_VERSION" + + .. note:: + + We are purposely using github for production deploys of global-jjb so that + uptime of LF Gerrit does not affect projects using global-jjb. In a test + environment we can use + https://gerrit.linuxfoundation.org/infra/releng/global-jjb if desired. + +#. Setup ``jjb/defaults.yaml`` + + Create and configure the following parameters in the + ``jjb/defaults.yaml`` file as described in the + `defaults.yaml configuration docs `. + + Once configured commit the modifications: + + .. code-block:: bash + + git add jjb/defaults.yaml + git commit -sm "Setup defaults.yaml" + +#. Push patches to Gerrit / GitHub using your favourite push method + +At this point global-jjb installation is complete in the :term:`ci-management` +repo and is ready for use. + +.. _deploy-ci-jobs: + +Deploy ci-jobs +============== + +The CI job group contains jobs that should deploy in all LF +Jenkins infra. The minimal configuration to deploy the +**{project-name}-ci-jobs** job group as defined in **lf-ci-jobs.yaml** is as +follows: + +jjb/ci-management/ci-management.yaml: + +.. code-block:: yaml + + - project: + name: ci-jobs + + jobs: + - '{project-name}-ci-jobs' + + project: ci-management + project-name: ci-management + build-node: centos7-builder-2c-1g + +**Required parameters**: + +:project: The project repo as defined in source control. +:project-name: A custom name to call the job in Jenkins. +:build-node: The name of the builder to use when building (Jenkins label). + +**Optional parameters**: + +:branch: The git branch to build from. (default: master) +:jjb-version: The version of JJB to install in the build minion. (default: + ) + +.. _deploy-packer-jobs: + +Deploy packer-jobs +================== + +The packer job group contains jobs to build custom minion images. The minimal +configuration needed to deploy the packer jobs is as follows which deploys the +**{project-name}-packer-jobs** job group as defined in **lf-ci-jobs.yaml**. + +jjb/ci-management/packer.yaml: + +.. code-block:: yaml + + - project: + name: packer-builder-jobs + + jobs: + - '{project-name}-packer-jobs' + + project: ci-management + project-name: ci-management + branch: master + build-node: centos7-builder-2c-1g + + platforms: + - centos + - ubuntu-16.04 + + templates: builder + + - project: + name: packer-docker-jobs + + jobs: + - '{project-name}-packer-jobs' + + project: ci-management + project-name: ci-management + branch: master + build-node: centos7-builder-2c-1g + + templates: docker + + platforms: + - centos + - ubuntu-16.04 + +**Required parameters**: + +:project: The project repo as defined in source control. +:project-name: A custom name to call the job in Jenkins. +:build-node: The name of the builder to use when building (Jenkins label). +:platforms: A list of supported platforms. +:templates: A list of templates to build. We recommend setting one template per + ``project`` section so that we can control which platforms to build for + specific templates. + +**Optional parameters**: + +:branch: The git branch to build from. (default: master) +:packer-version: The version of packer to install in the build minion, + when packer is not available. (default: ) diff --git a/docs/jjb/lf-ci-jobs.rst b/docs/jjb/lf-ci-jobs.rst index f6895f14..de81269f 100644 --- a/docs/jjb/lf-ci-jobs.rst +++ b/docs/jjb/lf-ci-jobs.rst @@ -160,6 +160,8 @@ Full Example: .. literalinclude:: ../../.jjb-test/lf-ci-jobs/jenkins-cfg-merge-full.yaml :language: yaml +.. _jenkins-cfg-envvar: + Global Environment Variables ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -- 2.16.6