Jenkins is one of them most popular tools for doing CI/CD. It provide plugins to build, deploy and automate deployment for any project. While jenkins is amazing, deploying and managing is not straightforward. Adding more workers often involves creating new VMs and adding them to the cluster.
In this post I'll demonstrate how can easily deploy jenkins on GCP using opta. I will also walk you through how you can tune various parameters to using opta itself to for scaling your jenkins deployment. With opta you can follow the same tutorial and deploy the entire thing on AWS with minor tweaks - we will discuss more about this in the end of this post.
Why Opta and not helm directly?
While going through the tutorial you might think why don't we just use helm on a kubernetes cluster? Well you've thought it right but if you look closely opta doesn't just deploy helm chart on a kubernetes cluster - it also creates a kubernetes cluster with all the important tooling up and running.
Opta ensures the following:
- It setups an Ingress to route traffic to the pods
- A secure network configuration with secure defaults
- Kubernetes provisioning and configuration with best practices
And we need to make sure this is all done securely so we don’t introduce unnecessary vulnerabilities! That’s why Opta is a good fit here — it sets up all these resources for us with a very robust architecture.
Jenkins Deployment Architecture
Pre-Requisites
Opta has some pre-requisites like terraform, kubectl, aws-cli/gcp-sdk. Ensure you have those configured. Refer this link for more information https://docs.opta.dev/installation/#prerequisites
To proceed further, make sure you have opta cli installed. For most users the script below should work. For detailed instructions you can follow this link:
/bin/bash -c "$(curl -fsSL https://docs.opta.dev/install.sh)"
Opta configuration for Deploying Jenkins
Basics
To get jenkins up and running on GCP using opta these are two important things
- Deploy kubernetes cluster
- Deploy jenkins service on top of it
Code
Let's look at the code structure. All our code lies inside folder opta-gcp
opta-gcp
├── opta.yml
└── staging
└── opta.yml
File contents: opta-gcp/staging/opta.yml
name: gcp-k8
org_name: opta
providers:
google:
region: asia-south2
project: project-name
modules:
- type: base
- type: dns
domain: gcp-example.runx.dev
delegated: false
- type: k8s-cluster
max_nodes: 3
node_instance_type: n2-standard-2
- type: k8s-base
linkerd_enabled: false
opta-gcp/opta.yml
name: hello-world
environments:
- name: gcp-k8
path: "staging/opta.yml"
modules:
- type: helm-chart
repository: https://charts.jenkins.io
version: 3.5.2
chart: jenkins
values:
controller:
ingress:
enabled: true
apiVersion: "extensions/v1beta1"
hostName: jenkins.example.com
agent:
idleMinutes: 20
Create Infra
Since we are trying to deploy our project on gcp make sure you have configured the gcp sdk correctly on your system before moving further (https://cloud.google.com/sdk/docs/install). Also ensure that in your opta-gcp/staging/opta.yml file you have provided correct gcp region and project name.
# Create infra, setup networking, deploy kubernetes
cd opta-gcp/staging
opta apply
As you run the above command you will see a lot of google cloud apis getting activated, opta automatically activates all the APIs which you need to create a full blown kubernetes cluster on gcp
Google service container.googleapis.com activated
Google service iam.googleapis.com activated
....
....
Further in your output you will see some messages related to terraform initialisation - opta converts the config file that you provide in yaml to good old terraform which we all love. Opta under the hood uses terraform to manage the infra and that's why you can rely on it. You will also see the detailed plan that terraform has generated. You can go through it. Opta creates a production ready terraform code to deploy kubernetes 3with all the best practices baked in. You can review the plan and accept to proceed further.
Now: Hold tight - because it can take 3-7 mins for things to finish. Once all goes well you can see that the kubernetes cluster has been provisioned for you
> gcloud container clusters list
NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS
gcp-k8 asia-south2 1.19.9-gke.1900 34.131.191.178 n2-standard-2 1.19.9-gke.1900 3 RUNNING
Deploy Jenkins
In previous step we bootstraped our infra and deployed kubernetes on it. In this step we are going to deploy jenkins on top of the infra which we created
cd opta-gcp/
opta apply
In this step opta generates terraform code to deploys helm chart. You can review the plan and accept it. It will take around 2-3 minutes for the plan to get applied. For specifying values to the helm chart you can modify the values inside opta-gcp/opta.yml . Refer to jenkins helm chart for more details https://github.com/helm/charts/tree/master/stable/jenkins. After opta apply finishes you will have jenkins up and running on your infra.
Configure Kubectl
You will have to configure kubectl to query things inside the cluster which you just created.
# Modify this command as per the parameters you passed on opta.yml
gcloud container clusters get-credentials opta-gcp-k8 --region asia-south2
# OR use this. It abstracts out all the complexities for you
opta configure-kubectl
Login to Jenkins
While opta sets up your kubernetes cluster it also installs basic components in your cluster like ingress-controller, cert-manager, etc to make your life simpler. For more details refer to: https://docs.opta.dev/modules-reference/environment-modules/gcp/#k8s-base . Since opta configures the ingress for us - let's go ahead and get the IP address of our load balancer which will be the entry point for traffic entering our cluster.
> kubectl get svc ingress-nginx-controller -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.0.64.13 34.131.159.112 80:31634/TCP,443:32007/TCP 47h
For this tutorial we are manually adding an entry into /etc/hosts file for jenkins.example.com however opta also provides you automated dns configuration which you can setup by following the instructions https://docs.opta.dev/miscellaneous/ingress/. For this tutorial we will add a manual entry into our etc/hosts file (On windows you can refer this for modifying /etc/hosts https://www.howtogeek.com/howto/27350/beginner-geek-how-to-edit-your-hosts-file/))
echo "<EXTERNAL-IP> jenkins.example.com" | sudo tee -a /etc/hosts
Now browse to jenkins.example.com
Now obtain the login credentials for jenkins
# Get username
> kubectl get secret hello-world-helmchart-jenkins -o jsonpath="{.data.jenkins-admin-user}" | base64 --decode
admin
# Get password
> kubectl get secret hello-world-helmchart-jenkins -o jsonpath="{.data.jenkins-admin-password}" | base64 --decode
tSDwbahxAStXkj2gBlPky
You should be able to login to jenkins now.
To put this behind a domain name and use SSL, please follow the dns delegation steps here.
Debugging
To debug things or explore the underlying containers, opta enables you to use kubectl.
Just run opta configure-kubectl and it will set up the appropriate config for kubectl. Now you can run kubectl get pods -n airflow to look at all the pods we are running and kubectl exec -it -n airflow pod/<pod-name> to ssh into a pod and explore the environment.
Running on AWS
Running this same example on AWS is pretty straightforward — as Opta is (mostly) cloud agnostic! We just need to update the env file to point to our AWS project and run apply.
The “service” yml file doesn’t need to be changed as it’s completely cloud agnostic!
What’s next
This was a quick overview of how we can use Opta to deploy Airflow to AWS or GCP. We were able to get a robust environment set up with minimal work!
All this code can be found on our github. Make sure to check out the Airflow docs and the Opta docs — for further configuration.
https://github.com/jenkinsci/helm-charts/blob/main/charts/jenkins/values.yaml
If you run into any problems or have suggestions for what else you’d like to use Opta for, please let us know in the comments or in our slack :))
Top comments (0)