--- /dev/null
+Linux Foundation Release Engineering Workshop
+=============================================
+
+This is a container environment containing services used by the Linux
+Foundation Release Engineering team. It is loosely influenced by the work
+done by openfrontier_ (author of the Gerrit container).
+
+The primary services are:
+
+ * Gerrit
+ * Jenkins
+ * Nexus [TODO]
+
+And the secondary services that support these:
+
+ * OpenLDAP
+ * NGINX
+ * MariaDB [TODO]
+ * HAProxy [TODO]
+
+Most of the documentation for individual services and how they are
+configured can be found under the `config/` directory, and
+`$SERVICE.env` files contain environment_ variables used by the docker
+containers.
+
+Quick Reference
+---------------
+
+Jenkins
+ localhost:8001
+
+Gerrit
+ localhost:8000
+ localhost:29418
+
+Default user account: sandbox/sandbox
+
+Getting Started
+---------------
+
+.. code-block::
+
+ docker-compose up -d
+
+Will bring up an environment containing all the services with
+authentication backed by LDAP, a simple ci-management repo in
+Gerrit[TODO], and a basic job in Jenkins that verifies commits to the
+ci-management repo[TODO].
+
+To bring up a single service in the foreground you can use[TODO - Nginx]:
+
+.. code-block::
+
+ docker-compose up $SERVICE
+
+Note: dependent services will still be launched but in the background.
+Hitting '^C' will stop this service, but not the others.
+
+If a service is backed by a Dockerfile (most will be eventually), then
+changes to the Dockerfile or files under '$SERVICE/' will require
+rebuilding the container:
+
+.. code-block::
+
+ docker-compose build
+
+To tear down the environment removing all the volumes, and start from
+scratch, run:
+
+.. code-block::
+
+ docker-compose down -v
+
+For other useful docker-compose commands such as logs, see::
+
+ docker-compose -h
+
+Default User
+------------
+
+The default username and password in LDAP are:
+
+ user: sandbox
+ pass: sandbox
+
+This user is part of the 'sandbox-admins' group and should have full
+admin rights on all services.
+
+Goals
+-----
+
+The goal of this project is to have an easily created workshop where
+releng work can be tested or proof-of-concepts created.
+
+Some examples:
+
+ * Jenkins Plugin upgrades
+ * Gerrit upgrades
+ * Jenkins and Gerrit-Trigger testing
+ * Nexus configuration
+ * Gerrit/LDAP group integration
+
+TODO
+----
+
+The following is a list of automation tasks still needed before the
+environment can be considered usable:
+
+Nginx:
+- [ ] Configure NGINX container to use environment variables_
+ Note: If both Jenkins and Gerrit aren't both started, the NGINX
+ container will continuously restart. This can be worked around by
+ disabling the other service you don't want to use:
+ `mv nginx/config/gerrit.conf gerrit.conf.disabled`
+
+Nexus:
+- [ ] Setup and configure Nexus
+
+Gerrit:
+- [ ] Remove postgres container configuration and replace with MariaDB
+ (or make optional)
+
+General:
+- [ ] Setup OpenLDAP over SSL by default
+- [ ] Create a basic ci-management repository in Gerrit
+- [ ] Connect and configure Gerrit and Jenkins automatically
+ - [ ] Have the Gerrit configuration setup in Jenkins
+ - [ ] Create Gerrit user with ssh pubkey from Jenkins
+
+Jenkins:
+- [ ] Fix (on Jenkins restart)::
+ WARNING: Caught exception evaluating:
+ instance.hasExplicitPermission(attrs.sid,p) in /configureSecurity/.
+ Reason: java.lang.NullPointerException
+- [ ] Create a basic SSH Jenkins agent that jobs can be ran on
+- [ ] Disable CLI remoting
+- [ ] Enable Agent -> Master Access Controls
+
+.. _environment: https://docs.docker.com/compose/environment-variables/#configuring-compose-using-environment-variables
+.. _variables: https://docs.docker.com/samples/nginx/#using-environment-variables-in-nginx-configuration
+.. _openfrontier: https://github.com/openfrontier/ci-compose
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2017 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+
+#GERRIT_INIT_ARGS=" \
+# --install-plugin=commit-message-length-validator \
+# --install-plugin=replication \
+#"
+# --install-plugin=delete-project \
+
+WEBURL=http://localhost:8080
+#DATABASE_TYPE=postgresql
+AUTH_TYPE=LDAP
+LDAP_SERVER=ldap://ldap
+LDAP_USERNAME=cn=readonly,dc=example,dc=org
+LDAP_PASSWORD=readonly
+LDAP_ACCOUNTBASE=ou=Users,dc=example,dc=org
+LDAP_ACCOUNTPATTERN=(&(objectClass=inetOrgPerson)(uid=${username}))
+LDAP_ACCOUNTFULLNAME=${cn}
+LDAP_EMAILADDRESS=${email}
+LDAP_GROUPBASE=ou=Groups,dc=example,dc=org
+LDAP_GROUPMEMBERPATTERN=(member=${dn})
+LDAP_LOCALUSERNAMETOLOWERCASE=true
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2017 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+JAVA_OPTS="-Djenkins.install.runSetupWizard=false"
+JENKINS_OPTS=""
+
+JENKINS_ADMIN_USER=admin
+JENKINS_ADMIN_PASSWORD=password
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2017 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+LDAP_ORGANISATION=Test Org
+LDAP_DOMAIN=example.org
+# Only set this if you want it different from the LDAP_DOMAIN
+LDAP_BASE_DN=dc=example,dc=org
+
+LDAP_READONLY_USER=true
+LDAP_READONLY_USER_USERNAME=readonly
+LDAP_READONLY_USER_PASSWORD=readonly
+
+LDAP_ADMIN_PASSWORD=admin
+
+LDAP_TLS=false
+LDAP_TLS_ENFORCE=false
+
+LDAP_REMOVE_CONFIG_AFTER_SETUP=false
--- /dev/null
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2017 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+dn: ou=Users,dc=example,dc=org
+changetype: add
+objectClass: organizationalUnit
+ou: Users
+
+dn: ou=Groups,dc=example,dc=org
+changetype: add
+objectClass: organizationalUnit
+ou: Groups
+
+# Password: sandbox
+dn: uid=sandbox,ou=Users,dc=example,dc=org
+changetype: add
+objectClass: top
+objectClass: person
+objectClass: inetOrgPerson
+cn: Sandbox
+sn: User
+displayname: Sandbox User
+mail: sandbox@localhost
+userpassword: {SSHA}WyQT/fg9FHJ/zcXsdvs51tk5vSlyL0fM
+
+dn: cn=sandbox-admins,ou=Groups,dc=example,dc=org
+changetype: add
+objectClass: top
+objectClass: groupOfNames
+member: uid=sandbox,ou=Users,dc=example,dc=org
--- /dev/null
+server {
+ listen 8000;
+ server_name localhost;
+
+ location / {
+ proxy_set_header Host $host:8000;
+ proxy_set_header X-Real-IP $remote_addr;
+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+ proxy_set_header X-Forwarded-Proto $scheme;
+ proxy_pass http://gerrit:8080;
+ proxy_read_timeout 90;
+ proxy_redirect http://gerrit:8080 http://localhost:8000;
+ }
+}
--- /dev/null
+server {
+ listen 8001;
+ server_name localhost;
+
+ location / {
+ proxy_set_header Host $host:8001;
+ proxy_set_header X-Real-IP $remote_addr;
+ proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+ proxy_set_header X-Forwarded-Proto $scheme;
+ proxy_pass http://jenkins:8080;
+ proxy_read_timeout 90;
+ proxy_redirect http://jenkins:8080 http://localhost:8001;
+ }
+}
--- /dev/null
+#!/bin/bash
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2017 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+POSTGRES_USER=gerrit2
+POSTGRES_PASSWORD=gerrit
+POSTGRES_DB=reviewdb
--- /dev/null
+---
+version: '3'
+services:
+ ldap:
+ image: osixia/openldap:latest
+ container_name: releng-ldap
+ env_file: config/ldap.env
+ command: "--loglevel debug --copy-service"
+ volumes:
+ - ./config/ldap/bootstrap/groups.ldif:/container/service/slapd/assets/config/bootstrap/ldif/custom/50-groups.ldif
+ ports:
+ - "636:636"
+ jenkins:
+ build: ./jenkins
+ container_name: releng-jenkins
+ env_file: config/jenkins.env
+ volumes:
+ - jenkins:/var/jenkins_home
+ expose:
+ - "8080"
+ depends_on:
+ - ldap
+ - nginx
+ links:
+ - ldap
+ - nginx
+ gerrit:
+ image: openfrontier/gerrit:latest
+ container_name: releng-gerrit
+ env_file: config/gerrit.env
+ volumes:
+ - gerrit:/var/gerrit/review_site
+ ports:
+ - "8081:8080"
+ - "29418:29418"
+ depends_on:
+ - ldap
+ - nginx
+ links:
+ - ldap
+ - nginx
+ postgres:
+ image: postgres:latest
+ env_file: config/postgres.env
+ expose:
+ - "5432"
+ nginx:
+ image: nginx:latest
+ restart: always
+ container_name: releng-ingress
+ ports:
+ - "8000:8000"
+ - "8001:8001"
+ volumes:
+ - ./config/nginx:/etc/nginx/conf.d
+volumes:
+ jenkins:
+ gerrit:
--- /dev/null
+# SPDX-License-Identifier: EPL-1.0
+##############################################################################
+# Copyright (c) 2017 The Linux Foundation and others.
+#
+# All rights reserved. This program and the accompanying materials
+# are made available under the terms of the Eclipse Public License v1.0
+# which accompanies this distribution, and is available at
+# http://www.eclipse.org/legal/epl-v10.html
+##############################################################################
+FROM jenkins/jenkins:lts
+
+# Disable upgrade wizard
+RUN echo 2.0 > /usr/share/jenkins/ref/jenkins.install.UpgradeWizard.state
+
+# Install desired plugins
+COPY plugins.txt /usr/share/jenkins/ref/plugins.txt
+RUN /usr/local/bin/install-plugins.sh < /usr/share/jenkins/ref/plugins.txt
+
+# Set the instance hostname
+COPY hostname.groovy /usr/share/jenkins/ref/init.groovy.d/hostname.groovy
+
+# Setup basic security
+# TODO: Fix matrix permission causing NullPointerException ?
+# COPY authorization.groovy /usr/share/jenkins/ref/init.groovy.d/authorization.groovy
+
+COPY ldap.groovy /usr/share/jenkins/ref/init.groovy.d/ldap.groovy
+
+# TODO: Prevent Cross Site Script Forgery
+# TODO: Disable CLI over remoting
+# TODO: Enable Agent/Master Access Control
+# TODO: Disable JNLP Port
--- /dev/null
+/*
+ * SPDX-License-Identifier: EPL-1.0
+ *
+ * Copyright (c) 2017 The Linux Foundation and others.
+ *
+ * All rights reserved. This program and the accompanying materials
+ * are made available under the terms of the Eclipse Public License v1.0
+ * which accompanies this distribution, and is available at
+ * http://www.eclipse.org/legal/epl-v10.html
+ *
+ */
+import jenkins.*
+import hudson.*
+import hudson.model.*
+import jenkins.model.*
+import hudson.security.*
+
+// Gets Admin username and password from environment
+// and sets authentication to Jenkins' own user database,
+// and the authorization to Matrix-based security.
+def hudsonRealm = new HudsonPrivateSecurityRealm(false)
+def admin_user = System.getenv('JENKINS_ADMIN_USER')
+def admin_password = System.getenv('JENKINS_ADMIN_PASSWORD')
+hudsonRealm.createAccount(admin_user, admin_password)
+
+def instance = Jenkins.getInstance()
+instance.setSecurityRealm(hudsonRealm)
+instance.save()
+
+def strategy = new GlobalMatrixAuthorizationStrategy()
+
+// Set Anonymous Permissions
+strategy.add(Jenkins.READ,'anonymous')
+strategy.add(Item.READ,'anonymous')
+strategy.add(Item.WORKSPACE,'anonymous')
+strategy.add(View.READ,'anonymous')
+
+// Set Admin Permissions
+strategy.add(Jenkins.ADMINISTER, "admin")
+
+instance.setAuthorizationStrategy(strategy)
+instance.save()
--- /dev/null
+/*
+ * SPDX-License-Identifier: EPL-1.0
+ *
+ * Copyright (c) 2017 The Linux Foundation and others.
+ *
+ * All rights reserved. This program and the accompanying materials
+ * are made available under the terms of the Eclipse Public License v1.0
+ * which accompanies this distribution, and is available at
+ * http://www.eclipse.org/legal/epl-v10.html
+ *
+ */
+import jenkins.model.JenkinsLocationConfiguration
+
+def jlc = JenkinsLocationConfiguration.get()
+jlc.setUrl("http://localhost:8001/")
+jlc.save()
--- /dev/null
+/*
+ * SPDX-License-Identifier: EPL-1.0
+ *
+ * Copyright (c) 2017 The Linux Foundation and others.
+ *
+ * All rights reserved. This program and the accompanying materials
+ * are made available under the terms of the Eclipse Public License v1.0
+ * which accompanies this distribution, and is available at
+ * http://www.eclipse.org/legal/epl-v10.html
+ *
+ */
+import jenkins.*
+import jenkins.model.*
+import hudson.*
+import hudson.model.*
+import hudson.security.*
+
+def instance = Jenkins.getInstance()
+
+def LDAPRealm = new LDAPSecurityRealm("ldap://ldap",
+"dc=example,dc=org",
+"ou=Users",
+"uid={0}",
+"ou=Groups",
+"cn=readonly,dc=example,dc=org",
+"readonly",
+false,
+false
+)
+instance.setSecurityRealm(LDAPRealm)
+
+def strategy = new GlobalMatrixAuthorizationStrategy()
+
+strategy.add(Jenkins.ADMINISTER, 'sandbox-admins')
+
+strategy.add(hudson.model.Hudson.READ,'anonymous')
+strategy.add(hudson.model.Item.READ,'anonymous')
+strategy.add(hudson.model.Item.WORKSPACE,'anonymous')
+strategy.add(hudson.model.View.READ,'anonymous')
+
+instance.setAuthorizationStrategy(strategy)
+
+instance.save()
--- /dev/null
+plain-credentials:1.4
+token-macro:2.1
+junit:1.20
+cloudbees-folder:6.1.0
+github:1.27.0
+handlebars:1.1.1
+pam-auth:1.3
+ssh-credentials:1.13
+workflow-scm-step:2.6
+github-api:1.86
+build-timeout:1.18
+branch-api:2.0.11
+ant:1.5
+pipeline-input-step:2.7
+pipeline-milestone-step:1.3.1
+pipeline-model-definition:1.1.9
+email-ext:2.58
+git:3.4.1
+docker-commons:1.8
+workflow-support:2.14
+scm-api:2.2.0
+pipeline-build-step:2.5.1
+workflow-durable-task-step:2.13
+pipeline-stage-view:2.8
+momentjs:1.1.1
+gradle:1.27.1
+authentication-tokens:1.3
+git-server:1.7
+workflow-multibranch:2.16
+pipeline-stage-tags-metadata:1.1.9
+workflow-basic-steps:2.6
+jquery-detached:1.2.1
+structs:1.9
+docker-workflow:1.12
+workflow-job:2.12.1
+mailer:1.20
+workflow-cps:2.37
+pipeline-model-extensions:1.1.9
+jackson2-api:2.7.3
+resource-disposer:0.6
+external-monitor-job:1.7
+pipeline-model-api:1.1.9
+workflow-step-api:2.12
+timestamper:1.8.8
+credentials-binding:1.12
+workflow-api:2.19
+credentials:2.1.14
+matrix-project:1.11
+script-security:1.30
+subversion:2.9
+pipeline-github-lib:1.0
+durable-task:1.14
+antisamy-markup-formatter:1.5
+github-branch-source:2.2.2
+pipeline-rest-api:2.8
+bouncycastle-api:2.16.2
+git-client:2.4.6
+pipeline-graph-analysis:1.4
+matrix-auth:1.7
+display-url-api:2.0
+ws-cleanup:0.34
+icon-shim:2.0.3
+pipeline-model-declarative-agent:1.1.1
+workflow-cps-global-lib:2.8
+ssh-slaves:1.20
+workflow-aggregator:2.5
+ace-editor:1.1
+windows-slaves:1.3.1
+ldap:1.16
+pipeline-stage-step:2.2
+mapdb-api:1.0.9.0
Code Review / releng / ci-workshop.git / commitdiff