Kubernetes

Prerequisites

Docker: Refer to the Docker Engine installation instructions for your platform.
Kubernetes: If you don’t have a running Kubernetes cluster, see the Create a minikube cluster section.

Create a minikube cluster

Minikube is a tool that creates a single-node Kubernetes cluster on your computer. Follow these steps if you don’t have a running Kubernetes cluster:

Install minikube by following the steps from the Install minikube site.
Install kubectl. See the instructions from the Install and Set Up kubectl page.
You can now create a minikube cluster by entering the following command:
```
$ minikube start
```
Once the cluster has been created, you can verify its status by entering:
```
$ minikube status
```

Create a namespace

A distinct namespace provides an additional layer of isolation and more control over the continuous integration environment. Create a namespace for the Jenkins deployment by typing the following command on your terminal:

$ kubectl create namespace jenkins

Use the following command to list existing namespaces:

$ kubectl get namespaces

The output confirms that the jenkins namespace was created successfully.

$ kubectl get namespaces
NAME              STATUS   AGE
default           Active   50m
jenkins           Active   30s
kube-node-lease   Active   50m
kube-public       Active   50m
kube-system       Active   50m

Install Jenkins with Helm v3

A typical Jenkins deployment consists of a controller node and, optionally, one or more agents. To simplify the deployment of Jenkins, we’ll use Helm to deploy Jenkins. Helm is a package manager for Kubernetes and its package format is called a chart. Many community-developed charts are available on GitHub.

Helm Charts provide “push button” deployment and deletion of apps, making adoption and development of Kubernetes apps easier for those with little container or microservices experience.

Prerequisites

Helm command line interface: If you don’t have Helm command line interface installed and configured locally, see the sections below to Install Helm and Configure Helm.

Install Helm

To install Helm CLI, follow the instructions from the Installing Helm page.

Configure Helm

Once Helm is installed and set up properly, add the Jenkins repo as follows:

$ helm repo add jenkinsci https://charts.jenkins.io
$ helm repo update

The helm charts in the Jenkins repo can be listed with the command:

$ helm search repo jenkinsci

Create a persistent volume

We want to create a persistent volume for our Jenkins controller pod. This will prevent us from losing our whole configuration of the Jenkins controller and our jobs when we reboot our minikube. This official minikube doc explains which directories we can use to mount or data. In a multi-node Kubernetes cluster, you’ll need some solution like NFS to make the mount directory available in the whole cluster. But because we use minikube which is a one-node cluster we don’t have to bother about it.

We choose to use the /data directory. This directory will contain our Jenkins controller configuration.

We will create a volume which is called jenkins-pv:

Paste the content from https://raw.githubusercontent.com/installing-jenkins-on-kubernetes/jenkins-volume.yaml into a YAML formatted file called jenkins-volume.yaml.
Run the following command to apply the spec:
```
$ kubectl apply -f jenkins-volume.yaml
```

It’s worth noting that, in the above spec, hostPath uses the /data/jenkins-volume/ of your node to emulate network-attached storage. This approach is only suited for development and testing purposes. For production, you should provide a network resource like a Google Compute Engine persistent disk, or an Amazon Elastic Block Store volume.

Minikube configured for hostPath sets the permissions on /data to the root account only. Once the volume is created you will need to manually change the permissions to allow the jenkins account to write its data.

minikube ssh
sudo chown -R 1000:1000 /data/jenkins-volume

Create a service account

In Kubernetes, service accounts are used to provide an identity for pods. Pods that want to interact with the API server will authenticate with a particular service account. By default, applications will authenticate as the default service account in the namespace they are running in. This means, for example, that an application running in the test namespace will use the default service account of the test namespace.

We will create a service account called jenkins:

A ClusterRole is a set of permissions that can be assigned to resources within a given cluster. Kubernetes APIs are categorized into API groups, based on the API objects that they relate to. While creating a ClusterRole, you can specify the operations that can be performed by the ClusterRole on one or more API objects in one or more API groups, just as we have done above. ClusterRoles have several uses. You can use a ClusterRole to:

define permissions on namespaced resources and be granted within individual namespace(s)
define permissions on namespaced resources and be granted across all namespaces
define permissions on cluster-scoped resources

If you want to define a role cluster-wide, use a ClusterRole; if you want to define a role within a namespace, use a Role.

A role binding grants the permissions defined in a role to a user or set of users. It holds a list of subjects (users, groups, or service accounts), and a reference to the role being granted.

A RoleBinding may reference any Role in the same namespace. Alternatively, a RoleBinding can reference a ClusterRole and bind that ClusterRole to the namespace of the RoleBinding. To bind a ClusterRole to all the namespaces in our cluster, we use a ClusterRoleBinding.

Paste the content from https://raw.githubusercontent.com/installing-jenkins-on-kubernetes/jenkins-sa.yaml into a YAML formatted file called jenkins-sa.yaml.
Run the following command to apply the spec:
```
$ kubectl apply -f jenkins-sa.yaml
```

Install Jenkins

We will deploy Jenkins including the Jenkins Kubernetes plugin. See the official chart for more details.

To enable persistence, we will create an override file and pass it as an argument to the Helm CLI. Paste the content from raw.githubusercontent.com/jenkinsci/helm-charts/main/charts/jenkins/values.yaml into a YAML formatted file called jenkins-values.yaml.

The jenkins-values.yaml is used as a template to provide values that are necessary for setup.
Open the jenkins-values.yaml file in your favorite text editor and modify the following:
- nodePort: Because we are using minikube we need to use NodePort as service type. Only cloud providers offer load balancers. We define port 32000 as port.
- storageClass:
  storageClass: jenkins-pv
- serviceAccount: the serviceAccount section of the jenkins-values.yaml file should look like this:
  serviceAccount: create: false # Service account name is autogenerated by default name: jenkins annotations: {}
  Where `name: jenkins` refers to the serviceAccount created for jenkins.
- We can also define which plugins we want to install on our Jenkins. We use some default plugins like git and the pipeline plugin.
Now you can install Jenkins by running the helm install command and passing it the following arguments:
- The name of the release jenkins
- The -f flag with the YAML file with overrides jenkins-values.yaml
- The name of the chart jenkinsci/jenkins
- The -n flag with the name of your namespace jenkins
  $ chart=jenkinsci/jenkins $ helm install jenkins -n jenkins -f jenkins-values.yaml $chart
  This outputs something similar to the following:
  NAME: jenkins LAST DEPLOYED: Wed Sep 16 11:13:10 2020 NAMESPACE: jenkins STATUS: deployed REVISION: 1

Get your 'admin' user password by running:

$ jsonpath="{.data.jenkins-admin-password}"
$ secret=$(kubectl get secret -n jenkins jenkins -o jsonpath=$jsonpath)
$ echo $(echo $secret | base64 --decode)

Get the Jenkins URL to visit by running these commands in the same shell:

$ jsonpath="{.spec.ports[0].nodePort}"
$ NODE_PORT=$(kubectl get -n jenkins -o jsonpath=$jsonpath services jenkins)
$ jsonpath="{.items[0].status.addresses[0].address}"
$ NODE_IP=$(kubectl get nodes -n jenkins -o jsonpath=$jsonpath)
$ echo http://$NODE_IP:$NODE_PORT/login

Login with the password from step 1 and the username: admin
Use Jenkins Configuration as Code by specifying configScripts in your values.yaml file. See the configuration as code documentation and examples.

Visit the Jenkins on Kubernetes solutions page for more information on running Jenkins on Kubernetes. Visit the Jenkins Configuration as Code project for more information on configuration as code. . Depending on your environment, it can take a bit of time for Jenkins to start up. Enter the following command to inspect the status of your Pod:

$ kubectl get pods -n jenkins

Once Jenkins is installed, the status should be set to Running as in the following output:

$ kubectl get pods -n jenkins
NAME                       READY   STATUS    RESTARTS   AGE
jenkins-645fbf58d6-6xfvj   1/1     Running   0          2m

To access your Jenkins server, you must retrieve the password. You can retrieve your password using either of the two options below.

Option 1

Run the following command:
```
$ jsonpath="{.data.jenkins-admin-password}"
$ secret=$(kubectl get secret -n jenkins jenkins -o jsonpath=$jsonpath)
$ echo $(echo $secret | base64 --decode)
```
The output should look like this:
```
Um1kJLOWQY
```
👆🏻Note that your password will be different.

Option 2

Run the following command:
```
$ jsonpath="{.data.jenkins-admin-password}"
$ kubectl get secret -n jenkins jenkins -o jsonpath=$jsonpath
```
The output should be a base64 encoded string like this:
```
WkIwRkdnbDZYZg==
```
Decode the base64 string and you have your password. You can use this website to decode your output.
Get the name of the Pod running that is running Jenkins using the following command:
```
$ kubectl get pods -n jenkins
```

Use the kubectl command to set up port forwarding:

$ kubectl -n jenkins port-forward <pod_name> 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

Visit 127.0.0.1:8080/ and log in using admin as the username and the password you retrieved earlier.

Install Jenkins with YAML files

This section describes how to use a set of YAML (Yet Another Markup Language) files to install Jenkins on a Kubernetes cluster. The YAML files are easily tracked, edited, and can be reused indefinitely.

Create Jenkins deployment file

Copy the contents here into your preferred text editor and create a jenkins-deployment.yaml file in the “jenkins” namespace we created in this section above.

This deployment file is defining a Deployment as indicated by the kind field.
The Deployment specifies a single replica. This ensures one and only one instance will be maintained by the Replication Controller in the event of failure.
The container image name is jenkins and version is 2.32.2
The list of ports specified within the spec are a list of ports to expose from the container on the Pods IP address.
- Jenkins running on (http) port 8080.
- The Pod exposes the port 8080 of the jenkins container.
The volumeMounts section of the file creates a Persistent Volume. This volume is mounted within the container at the path /var/jenkins_home and so modifications to data within /var/jenkins_home are written to the volume. The role of a persistent volume is to store basic Jenkins data and preserve it beyond the lifetime of a pod.

Exit and save the changes once you add the content to the Jenkins deployment file.

Deploy Jenkins

To create the deployment execute:

$ kubectl create -f jenkins-deployment.yaml -n jenkins

The command also instructs the system to install Jenkins within the jenkins namespace.

To validate that creating the deployment was successful you can invoke:

$ kubectl get deployments -n jenkins

Grant access to Jenkins service

We have a Jenkins instance deployed but it is still not accessible. The Jenkins Pod has been assigned an IP address that is internal to the Kubernetes cluster. It’s possible to log into the Kubernetes Node and access Jenkins from there but that’s not a very useful way to access the service.

To make Jenkins accessible outside the Kubernetes cluster the Pod needs to be exposed as a Service. A Service is an abstraction that exposes Jenkins to the wider network. It allows us to maintain a persistent connection to the pod regardless of the changes in the cluster. With a local deployment, this means creating a NodePort service type. A NodePort service type exposes a service on a port on each node in the cluster. The service is accessed through the Node IP address and the service nodePort. A simple service is defined here:

This service file is defining a Service as indicated by the kind field.
The Service is of type NodePort. Other options are ClusterIP (only accessible within the cluster) and LoadBalancer (IP address assigned by a cloud provider e.g. AWS Elastic IP).
The list of ports specified within the spec is a list of ports exposed by this service.
- The port is the port that will be exposed by the service.
- The target port is the port to access the Pods targeted by this service. A port name may also be specified.
The selector specifies the selection criteria for the Pods targeted by this service.

To create the service execute:

$ kubectl create -f jenkins-service.yaml -n jenkins

To validate that creating the service was successful you can run:

$ kubectl get services -n jenkins
NAME       TYPE        CLUSTER-IP       EXTERNAL-IP    PORT(S)           AGE
jenkins    NodePort    10.103.31.217    <none>         8080:32664/TCP    59s

Access Jenkins dashboard

So now we have created a deployment and service, how do we access Jenkins?

From the output above we can see that the service has been exposed on port 32664. We also know that because the service is of type NodeType the service will route requests made to any node on this port to the Jenkins pod. All that’s left for us is to determine the IP address of the minikube VM. Minikube have made this really simple by including a specific command that outputs the IP address of the running cluster:

$ minikube ip
192.168.99.100

Now we can access the Jenkins instance at 192.168.99.100:30104/

To access Jenkins, you initially need to enter your credentials. The default username for new installations is admin. The password can be obtained in several ways. This example uses the Jenkins deployment pod name.

To find the name of the pod, enter the following command:

$ kubectl get pods -n jenkins

Once you locate the name of the pod, use it to access the pod’s logs.

$ kubectl logs <pod_name> -n jenkins

The password is at the end of the log formatted as a long alphanumeric string:

*************************************************************
*************************************************************
*************************************************************

Jenkins initial setup is required.
An admin user has been created and a password generated.
Please use the following password to proceed to installation:

94b73ef6578c4b4692a157f768b2cfef

This may also be found at:
/var/jenkins_home/secrets/initialAdminPassword

*************************************************************
*************************************************************
*************************************************************

You have successfully installed Jenkins on your Kubernetes cluster and can use it to create new and efficient development pipelines.

Install Jenkins with Jenkins Operator

The Jenkins Operator is a Kubernetes native Operator which manages operations for Jenkins on Kubernetes. It was built with immutability and declarative configuration as code in mind. The Jenkins Operator is easy to install with just a few manifest and allows users to configure and manage Jenkins on Kubernetes.

Prerequisites

Jenkins Operator: If you don’t have Jenkins Operator installed and configured locally, see the sections below to Install Jenkins Operator.

Install Jenkins Operator

Requirements

To run Jenkins Operator, you will need:

Access to a Kubernetes cluster. If you don’t have a running Kubernetes cluster, see the Create a Kubernetes cluster with minikube section above.
kubectl version 1.11+

Configure Custom Resource Definition

The Custom Resource Definition (CRD) API has been introduced to Kubernetes in v1.7 and it enables users to add custom APIs to their Kubernetes cluster which can be used like any other native Kubernetes objects. Defining a CRD object creates a new custom resource with a name and schema that you specify. The Kubernetes API serves and handles the storage of your custom resource.

Install Jenkins Custom Resource Definition

Kindly note that links to sample yaml files such as the one below are subject to change based on maintenance and will be best to verify from the official documentation here before use.

$ CRD_FILE=kubernetes-operator/master/deploy/crds/jenkins_v1alpha2_jenkins_crd.yaml
$ kubectl apply -f https://raw.githubusercontent.com/jenkinsci/$CRD_FILE

Deploy Jenkins Operator

There are two ways to deploy the Jenkins Operator.

Using YAML’s: Apply Service Account and RBAC roles:

$ DEPLOY_FILE=kubernetes-operator/master/deploy/all-in-one-v1alpha2.yaml
$ kubectl apply -f https://raw.githubusercontent.com/jenkinsci/$DEPLOY_FILE

Watch Jenkins Operator instance being created:

$ kubectl get pods

Now Jenkins Operator should be up and running in the default namespace.

Using Helm Chart: There is an option to use Helm to install the operator. It requires the Helm 3+ for deployment. If you don’t have Helm command line interface installed and configured locally, see the sections above to Install Helm
Create a namespace for the operator: See the Create a namespace section above.
Configure Helm: Once Helm is installed and set up properly, add the Jenkins-Operator repo as follows:

$ CHART_FILE=kubernetes-operator/master/chart
$ helm repo add jenkins https://raw.githubusercontent.com/jenkinsci/$CHART_FILE

Install Jenkins Operator

$ helm install <name> jenkins/jenkins-operator -n jenkins

To add custom labels and annotations, you can use values.yaml file as explained in the Install Jenkins section above or pass them into helm install command, e.g.:

$ LABEL="jenkins.labels.LabelKey=LabelValue"
$ ANNOTATION="jenkins.annotations.AnnotationKey=AnnotationValue"
$ NAME=my-jenkins-operator-install
$ helm install $NAME jenkins/jenkins-operator -n jenkins --set $LABEL,$ANNOTATION

Deploy Jenkins

Once Jenkins Operator is up and running let’s deploy the actual Jenkins instance. Create a manifest e.g. jenkins_instance.yaml with the following data and save it locally.

apiVersion: jenkins.io/v1alpha2
kind: Jenkins
metadata:
  name: example
spec:
  master:
    containers:
    - name: jenkins-master
      image: jenkins/jenkins:lts-jdk11
      imagePullPolicy: Always
      livenessProbe:
        failureThreshold: 12
        httpGet:
          path: /login
          port: http
          scheme: HTTP
        initialDelaySeconds: 80
        periodSeconds: 10
        successThreshold: 1
        timeoutSeconds: 5
      readinessProbe:
        failureThreshold: 3
        httpGet:
          path: /login
          port: http
          scheme: HTTP
        initialDelaySeconds: 30
        periodSeconds: 10
        successThreshold: 1
        timeoutSeconds: 1
      resources:
        limits:
          cpu: 1500m
          memory: 3Gi
        requests:
          cpu: "1"
          memory: 500Mi
  seedJobs:
  - id: jenkins-operator
    targets: "cicd/jobs/*.jenkins"
    description: "Jenkins Operator repository"
    repositoryBranch: master
    repositoryUrl: https://github.com/jenkinsci/kubernetes-operator.git

Deploy Jenkins to Kubernetes:

$ kubectl create -f jenkins_instance.yaml -n jenkins

Watch the Jenkins instance being created:

$ kubectl get pods -n jenkins

Get Jenkins credentials

$ SECRET_NAME=jenkins-operator-credentials-<cr_name>
$ kubectl -n jenkins get secret $SECRET_NAME -o 'jsonpath={.data.user}' | base64 -d
$ kubectl -n jenkins get secret $SECRET_NAME -o 'jsonpath={.data.password}' | base64 -d

Connect to Jenkins (minikube)

$ minikube service jenkins-operator-http-<cr_name> --url -n jenkins

Connect to Jenkins (actual Kubernetes cluster)

$ kubectl port-forward jenkins-<cr_name> 8080:8080

Then open a browser with the address localhost:8080 to view your Jenkins Dashboard.

Post-installation setup wizard

After downloading, installing and running Jenkins using one of the procedures above, the post-installation setup wizard begins.

This setup wizard takes you through a few quick "one-off" steps to unlock Jenkins, customize it with plugins and create the first administrator user through which you can continue accessing Jenkins.

Unlocking Jenkins

When you first access a new Jenkins instance, you are asked to unlock it using an automatically-generated password.

Browse to http://localhost:8080 (or whichever port you configured for Jenkins when installing it) and wait until the Unlock Jenkins page appears.
From the Jenkins console log output, copy the automatically-generated alphanumeric password (between the 2 sets of asterisks).

Note:
- The command: sudo cat /var/lib/jenkins/secrets/initialAdminPassword will print the password at console.
- If you are running Jenkins in Docker using the official jenkins/jenkins image you can use sudo docker exec ${CONTAINER_ID or CONTAINER_NAME} cat /var/jenkins_home/secrets/initialAdminPassword to print the password in the console without having to exec into the container.
On the Unlock Jenkins page, paste this password into the Administrator password field and click Continue.
Notes:
- You can always access the Jenkins console log from the Docker logs (above).
- The Jenkins console log indicates the location (in the Jenkins home directory) where this password can also be obtained. This password must be entered in the setup wizard on new Jenkins installations before you can access Jenkins’s main UI. This password also serves as the default admininstrator account’s password (with username "admin") if you happen to skip the subsequent user-creation step in the setup wizard.

Customizing Jenkins with plugins

After unlocking Jenkins, the Customize Jenkins page appears. Here you can install any number of useful plugins as part of your initial setup.

Click one of the two options shown:

Install suggested plugins - to install the recommended set of plugins, which are based on most common use cases.
Select plugins to install - to choose which set of plugins to initially install. When you first access the plugin selection page, the suggested plugins are selected by default.

If you are not sure what plugins you need, choose Install suggested plugins. You can install (or remove) additional Jenkins plugins at a later point in time via the Manage Jenkins > Manage Plugins page in Jenkins.

The setup wizard shows the progression of Jenkins being configured and your chosen set of Jenkins plugins being installed. This process may take a few minutes.

Creating the first administrator user

Finally, after customizing Jenkins with plugins, Jenkins asks you to create your first administrator user.

When the Create First Admin User page appears, specify the details for your administrator user in the respective fields and click Save and Finish.
When the Jenkins is ready page appears, click Start using Jenkins.
Notes:
- This page may indicate Jenkins is almost ready! instead and if so, click Restart.
- If the page does not automatically refresh after a minute, use your web browser to refresh the page manually.
If required, log in to Jenkins with the credentials of the user you just created and you are ready to start using Jenkins!

User Handbook

Tutorials

Resources

Kubernetes

Prerequisites

Create a minikube cluster

Create a namespace

Install Jenkins with Helm v3

Prerequisites

Install Helm

Configure Helm

Create a persistent volume

Create a service account

Install Jenkins

Install Jenkins with YAML files

Create Jenkins deployment file

Deploy Jenkins

Grant access to Jenkins service

Access Jenkins dashboard

Install Jenkins with Jenkins Operator

Prerequisites

Install Jenkins Operator

Configure Custom Resource Definition

Deploy Jenkins Operator

Deploy Jenkins

Deploy Jenkins to Kubernetes:

Post-installation setup wizard

Unlocking Jenkins

Customizing Jenkins with plugins

Creating the first administrator user

Resources

Project

Community

Other