From: Jessica Wagantall Date: Thu, 26 Oct 2017 22:23:51 +0000 (-0700) Subject: Add Jenkins guide documentation X-Git-Url: https://gerrit.linuxfoundation.org/infra/gitweb?a=commitdiff_plain;h=refs%2Fchanges%2F01%2F7101%2F11;p=releng%2Fdocs.git Add Jenkins guide documentation Sections include: - Sandbox Overview - How to get access to the Sandbox - Pushing jobs to the Sandbox - Executing jobs in the Sandbox Change-Id: Iabadb6fc6fd3090af6b7070fb71dd4fed9ca8542 Issue-ID: RELENG-510 Signed-off-by: Jessica Wagantall --- diff --git a/docs/index.rst b/docs/index.rst index e378491..dbb6df4 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -14,6 +14,7 @@ Guides: best-practices gerrit + jenkins .. TODO Gerrit (RELENG-509) .. TODO Jenkins (RELENG-510) diff --git a/docs/jenkins.rst b/docs/jenkins.rst new file mode 100644 index 0000000..67e2c06 --- /dev/null +++ b/docs/jenkins.rst @@ -0,0 +1,249 @@ +.. _lfreleng-docs-jenkins: + +############# +Jenkins Guide +############# + +Sandbox Overview +================ + +The Jenkins Sandbox has similar configuration to the production instance. +It cannot publish artifacts or vote in Gerrit which makes it a safe environment +to test the jobs. The Sandbox has limited amount of Virtual Machine nodes instances +to test compared to production Jenkns. + +Facts to keep in mind before working on the Sandbox: + +- Jobs are automatically deleted every Saturday at 08:00 UTC +- Committers can login and configure Jenkins jobs in the Sandbox directly +- Sandbox jobs cannot perform any upload/deploy tasks +- There are no project configuration files and project credentials loaded into the system +- Sandbox jobs cannot vote on Gerrit +- Jenkins nodes have OpenStack configuration similarly to the production instance + +How to get access to the Sandbox +============================================== + +The Sandbox provides a testing/experimentation environment used before +pushing job templates to the production instance. + +To access the Sandbox use: jenkins./sandbox + +The access to the Sandbox uses the same LFID used in the production Jenkins +instance, but in this case a new `LF Helpdesk `_ ticket +(for the related project) needs creation to request the sanbox access. + +.. todo:: Link to Opening Helpdesk ticket docs + +The LF helpdesk team can add users to the appropriate group to grant permissions +to access the Sandbox via https://identity.linuxfoundation.org/. +The group that controls this access is -jenkins-sandbox-access +For example: +``https://identity.linuxfoundation.org/content/-jenkins-sandbox-access`` + +The requester will receive an invitation to join this group. +Once accepted, the user can now access the Sandbox same way as the production +Jenkins. + +Pushing jobs to the Sandbox +=========================== + +There are 2 supported ways to push jobs to the Sandbox the first one is easier and +does not require you to install anything to your local system and uses a jjb-deploy +job in production to push jobs from a Gerrit patch to the Sandbox. + +Using Gerrit comment +-------------------- + +This is the easiest and fastest way to start using the Sandbox. This is the recommended +default way to use the Sandbox since this does not require the user to install JJB or +configure it at all. + +This is the recommended way to push jobs to the Sandbox system and does not require +installation of Jenkins Job Builder locally. + +To push jobs to the Sandbox with jjb-deploy, add a comment on the Gerrit patch from ci-management: + +.. code-block:: bash + + jjb-deploy + +The resultant job's configuration reflects the same code the patch's code base in the Gerrit. +The job pushed into the Sandbox will reflect the changes made in the patch. + +.. note:: + + You can use * wildcard for job names. This is not a good practice. + We highly recommended to use specific Jenkins job names instead. + +Using JJB CLI +------------- + +JJB CLI needs configuration first. + +.. note:: + + Use this configuration if you prefer to use the JJB tool locally on your system. + +After getting access to the Sanbox group, configure the following. + +Create a jenkins.ini with the following contents modifying the relevant data: + +Example:: + + ; + + [job_builder] + ignore_cache=True + keep_descriptions=False + recursive=True + + [jenkins] + user-id= + password= + url=https://jenkins./sandbox + ignore_cache=True + +How to retrieve API token? +Login to the Jenkins Sandbox using your LFID, go to the user page by clicking on +your username. Click Configure and then click Show API Token. + +To start using the Sandbox, we must do a clone of ci-management or releng/builder +(in case of ODL) repo for the project. +For example: + +.. code-block:: bash + + git clone ssh://@gerrit.:29418/ci-management + +Make sure you sync global-jjb also using: + +.. code-block:: bash + + git submodule update --init + +Install JJB (Jenkins Job Builder). + +Execute the following commands to install JJB on your machine: + +.. code-block:: bash + + cd ci-management (or cd builder) + sudo pip install virtualenvwrapper + mkvirtualenv jjb + pip install jenkins-job-builder + jenkins-jobs --version + jenkins-jobs test --recursive jjb/ + +.. note:: + + More information on `Python Virtual Environments `_ + +To work on existing jobs or create new jobs, navigate to the `/jjb` directory +where you will find all job templates for the project. Follow the below commands +to test, push or delete jobs in your Sandbox environment. + +To Test a Job +------------- + +After you edit or create new job templates, test the job in the Sandbox +environment before you submit this job to production CI environment. + +.. code-block:: bash + + jenkins-jobs --conf jenkins.ini test jjb/ + +For Example: + +.. code-block:: bash + + jenkins-jobs --conf jenkins.ini test jjb/ ci-management-jjb-merge + +If the job you would like to test is a template with variables in its name, it +must be manually expanded before use. For example, the commonly used template +`{project-name}-jjb-merge` might expand to `ci-management-jjb-merge`. + +A successful test will output the XML description of the Jenkins job described +by the specified JJB job name. + +Execute the following command to pipe-out to a directory: + +.. code-block:: bash + + jenkins-jobs --conf jenkins.ini test jjb/ -o target + +The output directory will contain files with the XML configurations. + +To Push a Job +--------------- + +Ensure you have configured your jenkins.ini and verified it by outputting valid +XML descriptions of Jenkins jobs. Upon successful verification, execute the +following command to push the job to the Sandbox: + +.. code-block:: bash + + jenkins-jobs --conf jenkins.ini update jjb/ + +For Example: + +.. code-block:: bash + + jenkins-jobs --conf jenkins.ini update jjb/ ci-management-jjb-merge + +To Delete a Job +--------------- + +Execute the following command to Delete a job from Sandbox: + +.. code-block:: bash + + jenkins-jobs --conf jenkins.ini delete jjb/ + +For Example: + +.. code-block:: bash + + jenkins-jobs --conf jenkins.ini delete jjb/ ci-management-jjb-merge + +You can also delete the job from the UI options in Jenkins Sandbox. + +**Edit an Existing Job** + +In the Sandbox, you can directly edit the job configuration by selecting +the job name and clicking on the Configure button. +Click the Apply or Save (to save and exit the configuration) buttons to save the job. + +This is useful in the case where you might want to test quick tweaks to a job before +modifying the YAML. + +Edit the job in your terminal and follow the described steps in `To Test a Job`_ +and `To Push a Job`_ to push any changes and have them ready to push to Gerrit. + +.. important:: + + When pushing to the Sandbox with `jenkins-jobs`, do not forget the + parameter. Otherwise, JJB will push all job templates into the Sandbox and + will flood the system. + + If that happens, use **`ctrl+c` to cancel the upload**. + +A successful run of the desired job will look like this: + +.. code-block:: bash + + INFO:jenkins_jobs.builder:Number of jobs generated: 1 + +Executing jobs in the Sandbox +============================= + +Once you push the Jenkins job configuration to the Sandbox environment, run the +job from the Sandbox WebUI. Follow the below process to trigger the build: + +1. Login into the Jenkins Sandbox WebUI +2. Click on the job which you want to trigger +3. Click "Build with parameters" +4. Click Build +5. Verify the Build Executor Status bar to check on progress. + +You can click on the build number to view the job details and console output.