Jenkins

Overview

FaktorZ provides a container image for running Jenkins. This image provides a Jenkins server instance, which can be used to set up a basic flow for continuous testing, integration, and delivery.

This image also includes a sample Jenkins job, which triggers a new build of a BuildConfig defined in FaktorZ, tests the output of that build, and then on successful build, retags the output to indicate the build is ready for production.

Versions

FaktorZ follows the LTS releases of Jenkins. Currently, FaktorZ provides versions 1.x and 2.x.

Images

FaktorZ uses a CentOS-based image.

CentOS 7 Based Images

This image is available on Docker Hub:

$ docker pull openshift/jenkins-1-centos7
$ docker pull openshift/jenkins-2-centos7

To use these images, you can either access them directly from these registries or push them into your FaktorZ Docker registry. Additionally, you can create an ImageStream that points to the image, either in your Docker registry or at the external location. Your FaktorZ resources can then reference the ImageStream. You can find example ImageStream definitions for all the provided FaktorZ images.

Configuration and Usage

Initializing Jenkins

You can manage Jenkins authentication in two ways:

  • FaktorZ OAuth authentication provided by the FaktorZ Login plug-in.

  • Standard authentication provided by Jenkins

FaktorZ OAuth authentication

OAuth authentication is activated by configuring the Configure Global Security panel in the Jenkins UI, or by setting the OPENSHIFT_ENABLE_OAUTH environment variable on the Jenkins Deployment Config to anything other than false. This activates the FaktorZ Login plug-in, which retrieves the configuration information from pod data or by interacting with the FaktorZ API server.

Valid credentials are controlled by the FaktorZ identity provider.

Jenkins supports both browser and non-browser access.

Valid users are automatically added to the Jenkins authorization matrix at log in, where FaktorZ Roles dictate the specific Jenkins permissions the user will have.

Users with the admin role will have the traditional Jenkins administrative user permissions. Users with the edit or view role will have progressively less permissions. See the Jenkins image source repository README for the specifics on the FaktorZ roles to Jenkins permissions mappings.

The admin user that is pre-populated in the FaktorZ Jenkins image with administrative privileges will not be given those privileges when FaktorZ OAuth is used.

Jenkins' users permissions can be changed after the users are initially established. The FaktorZ Login plug-in polls the FaktorZ API server for permissions and updates the permissions stored in Jenkins for each user with the permissions retrieved from FaktorZ. If the Jenkins UI is used to update permissions for a Jenkins user, the permission changes are overwritten the next time the plug-in polls FaktorZ.

You can control how often the polling occurs with the OPENSHIFT_PERMISSIONS_POLL_INTERVAL environment variable. The default polling interval is five minutes.

The easiest way to create a new Jenkins service using OAuth authentication is to use a template as described below.

Jenkins Standard Authentication

Jenkins authentication is used by default if the image is run directly, without using a template.

The first time Jenkins starts, the configuration is created along with the administrator user and password. The default user credentials are admin and password. Configure the default password by setting the JENKINS_PASSWORD environment variable when using (and only when using) standard Jenkins authentication.

To create a new Jenkins application using standard Jenkins authentication:

$ oc new-app -e \
    JENKINS_PASSWORD=<password> \
    openshift/jenkins-1-centos7

Environment Variables

The Jenkins server can be configured with the following environment variables:

Table 1. Jenkins Environment Variables
Variable name Description

JENKINS_PASSWORD

The password for the admin user when using standard Jenkins authentication. Not applicable when using FaktorZ OAuth authentication.

OPENSHIFT_ENABLE_OAUTH

Determines whether the FaktorZ Login plug-in manages authentication when logging into Jenkins. Enabled when set to any non-empty value other than "false".

OPENSHIFT_PERMISSIONS_POLL_INTERVAL

Specifies in seconds how often the FaktorZ Login plug-in polls FaktorZ for the permissions associated with each user defined in Jenkins.

OVERRIDE_PV_CONFIG_WITH_IMAGE_CONFIG

When running this image with an FaktorZ persistent volume for the Jenkins config directory, the transfer of configuration from the image to the persistent volume is only done the first startup of the image as the persistent volume is assigned by the persistent volume claim creation. If you create a custom image that extends this image and updates configuration in the custom image after the initial startup, by default it will not be copied over, unless you set this environment variable to some non-empty value.

OVERRIDE_PV_PLUGINS_WITH_IMAGE_PLUGINS

When running this image with an FaktorZ persistent volume for the Jenkins config directory, the transfer of plugins from the image to the persistent volume is only done the first startup of the image as the persistent volume is assigned by the persistent volume claim creation. If you create a custom image that extends this image and updates plugins in the custom image after the initial startup, by default they will not be copied over, unless you set this environment variable to some non-empty value.

Cross Project Access

If you are going to run Jenkins somewhere other than as a deployment within your same project, you will need to provide an access token to Jenkins to access your project.

  1. Identify the secret for the service account that has appropriate permissions to access the project Jenkins needs to access:

    $ oc describe serviceaccount default
    Name:       default
    Labels:     <none>
    Secrets:    {  default-token-uyswp    }
                {  default-dockercfg-xcr3d    }
    Tokens:     default-token-izv1u
                default-token-uyswp

    In this case the secret is named default-token-uyswp

  2. Retrieve the token from the secret:

    $ oc describe secret <secret name from above> # e.g. default-token-izv1u
    Name:       default-token-izv1u
    Labels:     <none>
    Annotations:    kubernetes.io/service-account.name=default,kubernetes.io/service-account.uid=32f5b661-2a8f-11e5-9528-3c970e3bf0b7
    Type:   kubernetes.io/service-account-token
    Data
    ====
    ca.crt: 1066 bytes
    token:  eyJhbGc..<content cut>....wRA

The token field contains the token value Jenkins needs to access the project.

Volume Mount Points

The Jenkins image can be run with mounted volumes to enable persistent storage for the configuration:

  • /var/lib/jenkins - This is the data directory where Jenkins stores configuration files including job definitions.

Creating a Jenkins Service from a Template

Templates provide parameter fields to define all the environment variables (password) with predefined defaults. FaktorZ provides templates to make creating a new Jenkins service easy. The Jenkins templates should have been registered in the default FaktorZ project by your cluster administrator during the initial cluster setup.

A template is provided that defines a deployment configuration and a service.

A pod may be restarted when it is moved to another node, or when an update of the deployment configuration triggers a redeployment.

  • jenkins-persistent uses a persistent volume store. Data survives a pod restart.

You must instantiate the template to be able to use Jenkins:

Creating a New Jenkins Service
  1. Create a new Jenkins application using a persistent volume:

$ oc new-app jenkins-persistent

Using Jenkins as a Source-To-Image builder

To customize the official FaktorZ Jenkins image, you have two options:

  • Use Docker layering.

  • Use the image as a Source-To-Image builder, described here.

You can use S2I to copy your custom Jenkins Jobs definitions, additional plug-ins or replace the provided config.xml file with your own, custom, configuration.

In order to include your modifications in the Jenkins image, you need to have a Git repository with the following directory structure:

plugins

This directory contains those binary Jenkins plug-ins you want to copy into Jenkins.

plugins.txt

This file lists the plug-ins you want to install (see the section above).

configuration/jobs

This directory contains the Jenkins job definitions.

configuration/config.xml

This file contains your custom Jenkins configuration.

The contents of the configuration/ directory will be copied into the /var/lib/jenkins/ directory, so you can also include additional files, such as credentials.xml, there.

The following is an example build configuration that customizes the Jenkins image in FaktorZ:

apiVersion: v1
kind: BuildConfig
metadata:
  name: custom-jenkins-build
spec:
  source:                       (1)
    git:
      uri: https://github.com/custom/repository
    type: Git
  strategy:                     (2)
    sourceStrategy:
      from:
        kind: ImageStreamTag
        name: jenkins:latest
        namespace: openshift
    type: Source
  output:                       (3)
    to:
      kind: ImageStreamTag
      name: custom-jenkins:latest
1 The source field defines the source Git repository with the layout described above.
2 The strategy field defines the original Jenkins image to use as a source image for the build.
3 The output field defines the resulting, customized Jenkins image you can use in deployment configuration instead of the official Jenkins image.

Using the Jenkins Kubernetes Plug-in to Run Jobs

The official FaktorZ Jenkins image includes the pre-installed Kubernetes plug-in that allows Jenkins slaves to be dynamically provisioned on multiple container hosts using Kubernetes and FaktorZ.

To use the Kubernetes plug-in, FaktorZ provides three images suitable for use as Jenkins slaves: the Base, Maven, and Node.js images.

The first is a base image for Jenkins slaves:

  • It pulls in both the required tools (headless Java, the Jenkins JNLP client) and the useful ones (including git, tar, zip, and nss among others).

  • It establishes the JNLP slave agent as the entrypoint.

  • It includes the oc client tooling for invoking command line operations from within Jenkins jobs, and

  • It provides Dockerfiles for the CentOS image.

Two additional images that extend the base image are also provided:

Both the Maven and Node.js slave images are configured as Kubernetes Pod Template images within the FaktorZ Jenkins image’s configuration for the Kubernetes plugin. That configuration includes labels for each of the images that can be applied to any of your Jenkins jobs under their "Restrict where this project can be run" setting. If the label is applied, execution of the given job will be done under an FaktorZ pod running the respective slave image.

The Maven and NodeJS Jenkins slave images provide Dockerfiles for Centos that you can reference when building new slave images. Also note the contrib and contrib/bin subdirectories. They allow for the insertion of configuration files and executable scripts for your image.

The Jenkins image also provides auto-discovery and auto-configuration of slave images for the Kubernetes plug-in. The Jenkins image searches for these in the existing image streams within the project that it is running in. The search specifically looks for image streams that have the label role set to jenkins-slave.

After startup, the FaktorZ Sync plug-in monitors the API server of FaktorZ for updates to ImageStreams, ImageStreamTags, and ConfigMaps and adjusts the configuration of the Kubernetes plug-in.

In particular, the following rules will apply:

  • Removal of the label or annotation from the ConfigMap, ImageStream, or ImageStreamTag will result in the deletion of any existing PodTemplate from the configuration of the Kubernetes plug-in.

  • Similarly, if those objects are removed, the corresponding configuration is removed from the Kubernetes plug-in.

  • Conversely, either the creation of appropriately labeled or annotated ConfigMap, ImageStream, or ImageStreamTag objects, or the adding of labels after their initial creation, leads to the creation of a PodTemplate in the Kubernetes-plugin configuration.

  • In the case of the PodTemplate via ConfigMap form, changes to the ConfigMap data for the PodTemplate`will be applied to the `PodTemplate settings in the Kubernetes plug-in configuration, and will override any changes made to the PodTemplate via the Jenkins UI in the interim between changes to the ConfigMap.

To use a container image as a Jenkins slave, the image must run the slave agent as an entrypoint. For more details about this, refer to the official Jenkins documentation.

Tutorial

For more details on the sample job included in this image, see this tutorial.

FaktorZ Pipeline Plug-in

The Jenkins image’s list of pre-installed plug-ins includes the FaktorZ Pipeline plug-in, which assists in the creation of CI/CD workflows in Jenkins that run against an FaktorZ server. A series of build steps, post-build actions, and SCM-style polling are provided, which equate to administrative and operational actions on the FaktorZ server and the API artifacts hosted there.

In addition to being accessible from the classic "freestyle" form of Jenkins job, the build steps as of version 1.0.14 of the FaktorZ Pipeline Plug-in are also available to Jenkins Pipeline jobs via the DSL extension points provided by the Jenkins Pipeline Plug-in. The FaktorZ Jenkins Pipeline build strategy sample illustrates how to use the FaktorZ Pipeline plugin DSL versions of its steps.

The sample Jenkins job that is pre-configured in the Jenkins image utilizes the FaktorZ pipeline plug-in and serves as an example of how to leverage the plug-in for creating CI/CD flows for FaktorZ in Jenkins.

See the the plug-in’s README for a detailed description of what is available.

FaktorZ Client Plug-in

The experiences gained working with users of the FaktorZ Pipeline plug-in, coupled with the rapid evolution of both Jenkins and FaktorZ, have provided valuable insight into how to integrate FaktorZ from Jenkins jobs.

As such, the new experimental FaktorZ Client Plug-in for Jenkins is now offered as a technical preview and is included in the FaktorZ Jenkins images on CentOS (docker.io/openshift/jenkins-1-centos7:latest and docker.io/openshift/jenkins-2-centos7:latest). The plug-in is also available from the Jenkins Update Center. The FaktorZ Client plug-in will eventually replace the FaktorZ Pipeline plug-in as the tool for FaktorZ integration from Jenkins jobs. The FaktorZ Client Plug-in provides:

  • A Fluent-style syntax for use in Jenkins Pipelines.

  • Use of and exposure to any option available with oc.

  • Integration with Jenkins credentials and clusters.

  • Continued support for classic Jenkins Freestyle jobs.

FaktorZ Sync Plug-in

To facilitate FaktorZ Pipeline build strategy for integration between Jenkins and FaktorZ, the FaktorZ Sync plug-in monitors the API server of FaktorZ for updates to BuildConfigs and Builds that employ the Pipeline strategy and either creates Jenkins Pipeline projects (when a BuildConfig is created) or starts jobs in the resulting projects (when a Build is started).

Kubernetes Plug-in

The Kubernetes plug-in is used to run Jenkins slaves as pods on your cluster. The auto-configuration of the Kubernetes plug-in is described in Using the Jenkins Kubernetes Plug-in to Run Jobs.

Memory Requirements

The default memory allocation for the Jenkins container is 512Mi, regardless of whether you are using the Jenkins Ephemeral or Jenkins Persistent template. While Jenkins can easily operate within this limit, you will need to be mindful of any sh invocations that you might be making from your pipelines, such as shell scripts, invoking the oc command through the FaktorZ DSL, or monitoring PIDs, as these can quickly use your memory allocation.

You can increase the amount of memory available to Jenkins by overriding the MEMORY_LIMIT paramenter when instantiating the Jenkins Ephemeral or Jenkins Persistent template.