Automated CI/CD pipeline to deploy app on Google cloud using GKE

In collaboration with Disha Patel

Introduction:

This document outlines the steps required to create a Cloud CICD pipeline to deploy a containerized application on Google Kubernetes Engine (GKE) by defining two environments — staging and production. Follow these steps to set up and manage your application in a GKE environment.

Code reference:

ashunair / CICD-Pipeline

Google Cloud Platform based CI-CD pipeline

Cloud CI/CD Pipeline Deployment on GKE

Overview

This project provides a step-by-step guide to building a Cloud CI/CD pipeline for deploying a containerized application on Google Kubernetes Engine (GKE). It covers the setup of staging and production environments, ensuring smooth and automated deployments using Google Cloud services.

Key Highlights:

• GKE Cluster Setup: Creating Kubernetes clusters for staging and production.
• Artifact Registry Configuration: Storing and managing Docker images.
• Cloud Build Integration: Automating build and deployment pipelines.
• Cloud Deploy Setup: Managing continuous deployment to GKE.
• Skaffold for Kubernetes Automation: Simplifying deployment processes.
• GitHub to Cloud Build Triggers: Enabling automated deployments on code commits.

This project is available on Devpost Blog: Documentation Link with detailed explanations and images to guide you through each step. For the complete implementation, refer to the code repository: GitHub Repo.

View on GitHub

Setup Google Cloud SDK(required if you are pushing your code from local/on-prem)

Ensure you have Google Cloud SDK installed on your machine. This includes gcloud and kubectl command-line tools. Note that us-central1 is used across all the services in this project.
Configure gcloud when using Cloud Shell
Authenticate and set the project you want to work with using the commands: gcloud auth login and gcloud config set project [YOUR_PROJECT_ID].

1. Create a GKE Cluster

Create two Kubernetes cluster in GKE using the command with different naming conventions to identify the environments:

gcloud container clusters create [CLUSTER_NAME] — zone [ZONE] — num-nodes [NUM_NODES] — machine-type e2-medium — disk-size 10GB

gcloud container clusters create stg-pipeline --zone us-central1-a --num-nodes 2 --machine-type e2-small --disk-size 10GB

1.1 — Creating cluster using Cloud Shell

1.2 — Kubernetes Clusters for Staging and Prod

2. Setup Artifact Registry

Create a repository to store the docker images. Choose Docker in format, region should be us-central1 and keep rest as default.

2.1 — Docker Artifact Repository

2.2 — Docker Artifact Repository Metadata

3. File setup

Create git repo for your website and follow below mentioned file structure:

.
└── CICD-Pipeline/
├── Demo/
│ └── Index.html
├── DockerFile
├── cloudbuild. yaml
├── clouddeploy.yaml
├── kubernetes.yaml
└── skaffold.yaml

3.1 Create a Dockerfile:
Here is a step by step guild about how to define your Dockerfile. However, below is all we need for this project.

FROM nginx:alpine
COPY ./demo /usr/share/nginx/html 
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

1. FROM nginx:alpine

   • This sets the base image as the official Nginx Alpine image.
   • Alpine is a minimal Linux distribution, making the image smaller and more efficient.

2. COPY ./demo /usr/share/nginx/html

   • This copies the local ./demo directory (which contains static files like HTML, CSS, JavaScript) into the Nginx default document root at /usr/share/nginx/html.

3. EXPOSE 80

   • This declares that the container will listen on port 80 (default HTTP port).
   • However, this does not actually publish the port — it just informs Docker that this port is intended to be exposed.

4. CMD ["nginx", "-g", "daemon off;"]

   • This is the command that runs when the container starts.
   • nginx is started with the -g "daemon off;" flag to keep it running in the foreground.
   • This prevents the container from exiting immediately, ensuring that Nginx stays active.

3.2 Create cloudbuild.yaml

steps:
# 1. Docker Build
- name: 'gcr.io/cloud-builders/docker'
  args: ['build', '-t', 'us-central1-docker.pkg.dev/innate-valor-451418-i0/app-repo/cicd-app:$SHORT_SHA', '.']
# 2. Docker Push
- name: 'gcr.io/cloud-builders/docker'
  args: ["push", "us-central1-docker.pkg.dev/innate-valor-451418-i0/app-repo/cicd-app:$SHORT_SHA"]

# 3. create cloud deploy pipeline 
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
  entrypoint: 'bash'
  args:
    - '-c'
    - |
      if ! gcloud deploy delivery-pipelines describe demopipeline --region=us-central1 &> /dev/null; then
        echo "Creating Cloud Deploy pipeline..."
        gcloud deploy apply --file=clouddeploy.yaml --region=us-central1 --project=$PROJECT_ID
      else
        echo "Cloud Deploy pipeline already exists."
      fi
# 4. cloud deploy pipeline release 
- name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
  entrypoint: 'bash'
  args:
  - '-c'
  - >
    gcloud deploy releases create release-$BUILD_ID
    --delivery-pipeline=cicd-app
    --region=us-central1
    --source=./
    --images=sample-app=us-central1-docker.pkg.dev/innate-valor-451418-i0/app-repo/cicd-app:$SHORT_SHA

# 5. Logging Configuration
options:
  logging: CLOUD_LOGGING_ONLY

1. Docker Build

   • This step builds a Docker image from the current directory (.).
   • The image is tagged with a unique identifier $SHORT_SHA (a short commit hash) to track different builds.
   • The image is stored in Artifact Registry under: us-central1-docker.pkg.dev/chrome-sum-415007/demopipeline/demopipeline:$SHORT_SHA.

2. Docker Push

   • This step pushes the built Docker image to google artifact registry, making it available for deployment.

3. Create Cloud Deploy Pipeline

   • Checks if the Google Cloud Deploy pipeline named demopipeline exists.
   • If it does not exist, it creates the pipeline using clouddeploy.yaml.
   • If it already exists, it skips creation.

4. Create a New Release for Deployment

   • Creates a new release in Google Cloud Deploy named release-$BUILD_ID.
   • Uses the demopipeline to deploy the newly built image.
   • References the container image from artifact registry.

5. Logging Configuration

   • Ensures logs are stored in google cloud logging only.

3.3 Create a clouddeploy.yaml:

# 1. Delivery Pipeline Definition
apiVersion: deploy.cloud.google.com/v1beta1
kind: DeliveryPipeline
metadata:
 name: cicd-app
description: cicd-app application 
serialPipeline:
 stages:
 - targetId: staging
 - targetId: prod
---

# 2. Staging Environment Target

apiVersion: deploy.cloud.google.com/v1beta1
kind: Target
metadata:
 name: staging
description: "staging cluster"
gke:
 cluster: projects/innate-valor-451418-i0/locations/us-central1-a/clusters/stg-pipeline
---

# 3. Production Environment Target

apiVersion: deploy.cloud.google.com/v1beta1
kind: Target
metadata:
 name: prod
description: prod cluster
requireApproval: true
gke:
 cluster: projects/innate-valor-451418-i0/locations/us-central1-a/clusters/prod-pipeline

1. Delivery Pipeline Definition

   • Defines the cloud deploy pipeline named demopipeline.
   • Uses a serial pipeline (deploys sequentially).
   • Has two stages: > staging (first deployment stage) > prod (final production stage)

2. Staging Environment Target

   • Defines a staging environment in a GKE cluster.
   • Cluster location: us-central1.
   • This is the first step in the pipeline.

3. Production Environment Target

   • Defines the production environment.
   • Uses a separate GKE cluster in us-central1-a.
   • requireApproval: true → Requires manual approval before deploying to production.

3.4 Create Kubernetes Deployment and Service yaml file

# 1. Deployment Configuration

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
        - name: web-app
          image: sample-app
          ports:
            - containerPort: 80
---
# 2. Service Configuration

apiVersion: v1
kind: Service
metadata:
  name: web-app
spec:
  type: LoadBalancer
  selector:
    app: web-app
  ports:
    - port: 80
      targetPort: 80

1. Deployment Configuration

   • Creates a Deployment named demopipeline.
   • Runs 2 replicas for high availability.
   • Uses label selectors (app: demopipeline) to identify Pods.
   • Deploys a container with the name demopipeline, using the image sample-app.
   • Exposes port 80 inside the container for web traffic.

2. Service Configuration

   • Creates a Service named demopipeline.
   • Exposes the application externally using a LoadBalancer.
   • Routes traffic to Pods matching app: demopipeline.
   • Listens on port 80 and forwards requests to port 80 inside the Pods.

3.5 Create Skaffold manifest file to automate Kubernetes deployments

# 1. Skaffold Configuration Overview

apiVersion: skaffold/v2beta16
kind: Config
metadata:
  name: web-app

# 2. Build Configuration

build:
  artifacts:
  - image: sample-app
    context: .
    docker:
      dockerfile: Dockerfile
  tagPolicy:
    gitCommit: {}
  local:
    useBuildkit: false

# 3. Deployment Configuration

deploy:
  kubectl:
    manifests:
    - kubernetes.yaml

# 4. Profile Configuration (For Google Cloud Build)

profiles:
- name: gcb

1. Skaffold Configuration Overview

   • Defines Skaffold configuration (v2beta16 API version).
   • The project name is demopipeline.

2. Build Configuration

   • build: Defines how to build the container image for the application.
   • artifacts: Uses the Dockerfile in the current directory and builds the Docker image as sample-app.
   • tagPolicy: Uses the Git commit hash (gitCommit: {}) for tagging images, ensuring unique versions.
   • local: Local Docker build.

3. Deployment Configuration

   • Uses kubectl to deploy the application to Kubernetes.
   • Deploys the kubernetes.yaml file, which likely contains a Deployment and Service definition.

4. Profile Configuration (For Google Cloud Build)

   • Defines a Skaffold profile named gcb (Google Cloud Build).
   • Profiles allow different environments (e.g., local vs. cloud) by specifying different build and deployment settings.

Now that we have a Skaffold-based Kubernetes deployment pipeline, the next step is to automate the build and deployment process using GitHub and cloud build. When code is pushed and committed to GitHub, it should automatically trigger Cloud Build, which will:

• ✅ Build the Docker image using the cloudbuild.yaml file.
• ✅ Push the image to Google Artifact Registry.
• ✅ Deploy the application to Kubernetes using Google Cloud Deploy.

4. Set Up a Cloud Build Trigger in Google Cloud

4.1 Setup connection from GitHub to Cloud Build

4.1.1 — Connecting a GitHub repository to Google Cloud Build triggers.

• Go to Google Cloud Console → Navigate to Cloud Build.
• Click Triggers → Create Trigger.
• Select “Connect Repository” and link your GitHub repository.

4.2 Configure the Trigger

4.2.1 — Configuring a Cloud Build trigger in Google Cloud

4.2.2 — Cloud Build Trigger

• Trigger Type: Push to a branch (e.g., main or dev).
• Branch Filter: Set it to main or any specific branch.
• Build Configuration: Choose “cloudbuild.yaml” (it should already exist in the repo).
• Service Account: Use a service account with Cloud Build, Artifact Registry, and Cloud Deploy permissions.
• Save the trigger and test it with a sample push.

5. Final Execution

The setup is complete, and when new code is pushed to the selected branch (e.g., staging), it will trigger Cloud Build, which interacts with the cloudbuild.yaml file and executes the steps sequentially.

5.1 — Cloud Build waiting for approval

Here, as I have set up approval to execute the build, each time new code is committed, it will require approval before proceeding. Once approved, it will run the commands mentioned in cloudbuild.yaml. For detailed information, refer to section 3.2 above.

5.2— Cloud Build Succeeded

You will see the release in Staging here in Cloud Deploy:

5.3 — Delivery Pipeline created

5.4 — Release in Staging

And with that, workload is created in GKE Cluster:

5.5 — Workload created in GKE

Inside the workload, you can see the endpoint created. Click on the IP to see your webpage(index.html in this project).

5.6 — Endpoint IP highlighted
Now, navigate to Cloud Deploy and you can promote it to Production. It will require an approval.

5.7 — Getting ready for Prod Release

5.8 — Approval required for Prod Release

Hence after the prod environment is approved, again you can see the release in GKE Workload for Prod.

5.9 — Prod release in GKE

Thus, it will create another service endpoint for Prod.

6 Additional Steps:

Since vulnerabilities can be introduced into the artifact during the architecture phase, we enable the Container Scanning API to scan Docker images and detect any anomalies.

Additionally, we set up monitoring alerts for the approval process when transitioning from staging (STG) to production (PROD). This includes configuring an email approval system and integrating Slack notifications for approvals.

Steps to add notification channel in Google Cloud using email:

• Open Cloud Monitoring → Go to alerting.
• Edit Notification Channels → Click Add New under Email.
• Add Email Address → Enter approval email & click Save.
• From the cloud console, go to the delivery pipeline and select the pipeline.
• To configure alerts, select recommended alerts and set appropriate required policies.
• Select the email channel.
• Create Policy → Alerts will now trigger email notifications.

6.1 Creating email notification channel

Steps to add notification channel in Google Cloud by integrating Slack:

• Open Cloud Monitoring → Go to alerting.
• Edit Notification Channels → Click Add New under Slack.
• This will pop-up a new window for permission to Slack app → Allow.
• Add Slack channel → Save.
• From the cloud console, go to the delivery pipeline and select the pipeline.
• To configure alerts, select recommended alerts and set appropriate required policies.
• Select the Slack channel.
• Create Policy → Alerts will now trigger Slack notifications.

6.2 Permission access to Slack app via Google Cloud

6.3 Setting Slack notification channel in GCP

6.4 Creating policy to set up email and Slack alerts

Whenever a new commit is made to the code in the delivery pipeline, the alert notification will be sent to the email and Slack channel.

6.5 CI/CD pipeline approval pending to PRD

6.6 CI/CD pipeline approval alert notification in email

6.7 CI/CD pipeline approval alert notification in slack channel

🎉Phewwww 😮‍💨... the hard work paid off 🎉

Finally, you have successfully automated your CI/CD pipeline! 🚀

Now sit back, grab a coffee ☕, and watch the magic happen—because from now on, your code will deploy itself 😎

Got any questions or need help troubleshooting? Drop them below! ⬇️💬

References:

1. Create a build configuration file
2. Create Notification Channel
3. Get Started with Skaffold in Cloud Deploy
4. Cloud Deploy-Configuration schema reference
5. Configure Slack notifications