DEV Community

Cover image for Streamlining Kubernetes Deployments: CI/CD with GitHub Actions and Helm for EKS
Jones Ndzenyuy
Jones Ndzenyuy

Posted on

Streamlining Kubernetes Deployments: CI/CD with GitHub Actions and Helm for EKS

In this project, I will guide you in building a pipeline with GitHub actions that will deploy an infrastructure using Terraform(VPC, public and private subnets, nat gatway) and run containers on EKS, updates EKS each time there's a new application version. The new version is automatically built each time the developper pushes application code to the code repository. Necessary security scans are run in order to ensure a secure and complaint code. The infrastructure code and application code are stored in two different repositories.

Architecture

Image description

How to build it

Prepare Repositories

Login to your github account, choose a location and open gitbash terminal on your local machine and clone the code

git clone https://github.com/Ndzenyuy/githubactions-infra.git
git clone https://github.com/Ndzenyuy/githubactions-appl.git
rm -rf githubactions-infra/.git/ githubactions-infra/.github/
rm -rf githubactions-appl/.git/ githubactions-appl/.github/
Enter fullscreen mode Exit fullscreen mode

The two repositories contain the infrastructure(githubactions-infra) and the application code(githubactions-appl). Now we have to initialize git for code management, first for the infrastructure. Back to the console, run the following

cd githubactions-infra
git init
code .
Enter fullscreen mode Exit fullscreen mode

The above code will open the infrastructure in VScode and initialise git. Under the source control icon in VScode, publish the branch to GitHub, choose public repository. Return to the console and run the following code (make sure you are in the githubactions-infra folder)

cd ../githubactions-appl
git init
code .
Enter fullscreen mode Exit fullscreen mode

Another VScode window will open, Under the source control icon in VScode, publish the branch to GitHub, choose public repository. We are now ready to deploy our application.

GitHub Secrets

Here we are going to create and store AWS access keys, S3 bucket, ECR repository and store in Github secrets.

Create IAM user

Go to AWS console and create an admin user IAM -> User -> create user -> attach policies -> administrator access -> create access keys

On github, githubactions-infra repository go to settings -> Secrets and variables -> actions -> new repository secret and store the following keys

  • Name: AWS_ACCESS_KEY_ID, Secret:
  • Name: AWS_SECRET_ACCESS_KEY: secret:

Do the same for the repository githubactions-appl, store both secrets with the same name.

Create S3 bucket

Goto AWS console, and create an S3 bucket, we can call it "githubactions-infra-xxx" (xxx is a random number to make bucket unique)

Image description

Back to Github secrets(githubactions-infra), create a new secret having the bucket name:

Name: BUCKET_TF_STATE, secret: githubactions-infra-xxx

Create ECR repository

On to AWS console, Elastic Container Registry -> Create repository -> name: githubactions-appl -> create repository. Copy the repository URI(XXXXXXXXX.dkr.ecr.us-east-2.amazonaws.com/githubactions-appl)

Image description

Back to github secrets (githubactions-appl), Create another secret

Name: REGISTRY, secret: XXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com

githubactions-infra secrets should be similar to

Image description

Terraform Code Adjustments

In the githubactions-infra repo code, open it in VSCode and move to the folder terraform, modify the contents of terraform.tf while updating the name of your s3 bucket and the region where the infrastructure will be deployed

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.25.0"
    }

    random = {
      source  = "hashicorp/random"
      version = "~> 3.5.1"
    }

    tls = {
      source  = "hashicorp/tls"
      version = "~> 4.0.4"
    }

    cloudinit = {
      source  = "hashicorp/cloudinit"
      version = "~> 2.3.2"
    }

    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "~> 2.23.0"
    }
  }

  backend "s3" {
    bucket = "<Your bucket name>"
    key    = "terraform.tfstate"
    region = "your working region"
  }

  #required_version = "~> 1.5.1"
  required_version = "~> 1.10.1"

}
Enter fullscreen mode Exit fullscreen mode

Github actions workflow

In this workflow terraform will not apply the code for the infrastructure to be created(staging), but when a pull request is made, the pipeline is triggered and this time and the pipeline get applied(main), we need to put conditions. In the file .github/workflow/terraform.yml, we should have the code

name: "Github Actions IaC"
on:
    push:
        branches:
            - main
            - stage
        paths:
            - terraform/**
    pull_request:
        branches:
            - main
        paths:
            - terraform/**

env:
    # Credentials for deployment to AWS
    AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
    AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
    # S3 bucket for the Terraform state
    BUCKET_TF_STATE: ${{ secrets.BUCKET_TF_STATE }}
    AWS_REGION: us-east-1
    EKS_CLUSTER: infra-eks

jobs:
    terraform:
        name: "Apply terraform code changes"
        runs-on: ubuntu-latest
        defaults:
            run:
                shell: bash
                working-directory: ./terraform

        steps:
            - name: Checkout source code
              uses: actions/checkout@v4           

            - name: Setup Terraform with specified version on the runner
              uses: hashicorp/setup-terraform@v2
              #with:
              #  terraform_version: 1.6.3

            - name: Terraform init
              id: init
              run: terraform init -backend-config="bucket=$BUCKET_TF_STATE"

            - name: Terraform format
              id: fmt
              run: terraform fmt -check

            - name: Terraform validate
              id: validate
              run: terraform validate

            - name: Terraform plan
              id: plan
              run: terraform plan -no-color -input=false -out planfile
              continue-on-error: true

            - name: Terraform plan status
              if: steps.plan.outcome == 'failure'
              run: exit 1

            - name: Terraform Apply
              id: apple
              if: github.ref == 'refs/heads/main' && github.event_name == 'push'
              run: terraform apply -auto-approve -input=false -parallelism=1 planfile

            - name:  Configure AWS credentials
              uses: aws-actions/configure-aws-credentials@v1
              with: 
                aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
                aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
                aws-region: ${{ env.AWS_REGION }}

            - name: Get Kube config file
              id: getconfig
              if: steps.apple.outcome == 'success' 
              run: aws eks update-kubeconfig --region ${{ env.AWS_REGION }} --name ${{ env.EKS_CLUSTER }}

            - name: Install Ingress controller
              if: steps.apple.outcome == 'success' && steps.getconfig.outcome == 'success'
              run: kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.3/deploy/static/provider/aws/deploy.yaml
Enter fullscreen mode Exit fullscreen mode

Test Pipeline

We need to first test the pipeline with a dry run by creating a staging branch. This will trigger the pipeline but will not apply so we are sure it can run. Got to the githubactions-infra window in VScode and open a new terminal, make sure any changes have been committed then run the following:

git branch -b stage
Enter fullscreen mode Exit fullscreen mode

This will create a new branch, on the source control icon in VSCode, publish the branch. This will run the pipeline and skip the apply step because there is a a condition set to only run apply when the main branch is active

Image description

If the run is successful, we can now be certain the pipeline will run, if it fails, read the logs, the terraform version might have been outdated. Change it to the new and push the code again.

Merge the staging branch to main branch, this will cause the pipeline again to run but this time, will apply the changes and setup the infrastructure on AWS eks.

Image description

App workflow

The next workflow will use the infrastructure created from the previous job, then deploy the application with best practices.

Create an organization in sonarcloud

Open the browser and navigate to www.sonarcloud.io, click the "+" at the top right corner and create a new organization -> create one manually

name: github-actions-cicd
key: github-actions-cicd -> Create

Click analyze new project

Organization: github-actions-cicd
display name: github-actions
Project key: github-actions -> Next
The new code for this project will be based on: Previous version=true
-> Create project

Image description

Choose analysis method: With Github actions:

Copy the sonar token and paste it in notepad, click on information on the left, copy the values of organization, project key and create secrets in Github repo githubactions-infra

name: SONAR_TOKEN, secret: <COPIED TOKEN>
name: SONAR_ORGANIZATION, secret: <COPIED ORGANIZATION>
name: SONAR_PROJECT_KEY, secret: <COPIED PROJECT KEY>
name: SONAR_URL, secret: https://sonarcloud.io

Image description

Create quality gate in sonarcloud -> Administration -> Organization's settings -> Create:
name: githubactionsQG
add condition:
where: on overall code = true
Quality gate fails when Bugs > 50
select project: github-actions

Go back to project administration, select Quality gate then "githubactionsQG"

Deployment to EKS

Install helm charts

In terminal, run the code(for ubuntu users, other users can check their operating system installation)

sudo apt install helm
Enter fullscreen mode Exit fullscreen mode

Configure helm

Change to the directory where the project is githubactions-appl, then run

helm create githubactions-charts
Enter fullscreen mode Exit fullscreen mode

A new directory(githubactions-charts) will be created, next

mv githubactions-charts helm/
rm -rf helm/githubactions-charts/templates/*
cp kubernetes/vpro-app/* helm/githubactions-charts/templates/
Enter fullscreen mode Exit fullscreen mode

Deploy code

We need to first git the workflow the permissions to write trivy scan results in the security tab. To do so, Navigate to githubactions-appl repository on GitHub. Click on the Settings tab; In the sidebar, click on Actions. Under the Actions permissions section, click general and scroll down to Workflow permissions and select Read and write permissions then save.

Back to VSCode, commit and push the changes. In github actions, run the workflow. This time the pipeline will not be triggered automatically(as configured in our code), we'll have to trigger it manually on our github actions tab.

Check the security tab in github actions to see the report of the security scans with trivy

Image description

Modify domain name

In the file helm/githubactions-charts/templates/vproingress.yaml, modify the domain name to your own after creating a hosted zone in Route53

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: vpro-ingress
  annotations:
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  ingressClassName: nginx
  rules:
  - host: github.ndzenyuyjones.link
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-app
            port:
              number: 8080
Enter fullscreen mode Exit fullscreen mode

Modify the line - host: github.ndzenyuyjones.link to contain your domain name, will put a CNAME record in the hosted zone.

On the AWS console, we need to search and copy the url of the load balancer created for this project by the previous stack, copy the url and create a hosted zone with a cname record

Type: CNAME
Name: githubactions
value:
Wait about 5-10mins and search the link of our project on a browser github.ndzenyuyjones.link(use the link you created in the hosted zone), the landing page should appear.

Image description

Login with

username: admin_vp
passwrd: admin_vp

Image description

Clean up

Remove ingress controller

First we need to make sure there is no existing kubeconfig file in our local machine by running

rm -rf /.kube/config
aws eks update-kubeconfig --region us-east-1 --name infra-eks
Enter fullscreen mode Exit fullscreen mode

Now navigate to the repository on your local machine, where ever it was cloned and run

kubectl delete -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.3/deploy/static/provider/aws/deploy.yaml

Now delete the helm list, you can first run helm list and copy the name of the list to be uninstalled

helm uninstall rhena-stack
Enter fullscreen mode Exit fullscreen mode

Move to terraform folder

cd terraform/
Then initialize terraform

Change the content of terraform.tf file to match the following
Enter fullscreen mode Exit fullscreen mode

terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.25.0"
}

random = {
  source  = "hashicorp/random"
  version = "~> 3.5.1"
}

tls = {
  source  = "hashicorp/tls"
  version = "~> 4.0.4"
}

cloudinit = {
  source  = "hashicorp/cloudinit"
  version = "~> 2.3.2"
}

kubernetes = {
  source  = "hashicorp/kubernetes"
  version = "~> 2.23.0"
}
Enter fullscreen mode Exit fullscreen mode

}

backend "s3" {
bucket = "githubactions-infra-012"
key    = "terraform.tfstate"
region = "us-east-1"
Enter fullscreen mode Exit fullscreen mode

}

required_version = "~> 1.5.1"
}

Then we initialise the local machine to have the same state as the created cloud infrastructure.

terraform init
Enter fullscreen mode Exit fullscreen mode

Now we run destroy command, when prompted type "yes"

terraform destroy
Enter fullscreen mode Exit fullscreen mode

Top comments (0)

Read next

mike_andreuzza profile image

Learn how to create a flyout menu with Tailwind CSS and Alpine JS

Michael Andreuzza -

stephan_hawke_fd9cbb38db7 profile image

2025: IoT in Healthcare Revolutionizing Patient Care

Stephan Hawke -

cryptosandy profile image

How to Write Cleaner Code: Best Practices

Crypto.Andy (DEV) -

v_inoth_f0fad29baec4c profile image

Simple model method

V I N O T H -