Automating EKS Deployment and NGINX Setup Using Helm with AWS CDK in Python

Introduction

Amazon Elastic Kubernetes Service (EKS) simplifies the process of running Kubernetes on AWS. When combined with the power of Helm and the AWS Cloud Development Kit (CDK), you can automate the deployment of Kubernetes resources and applications efficiently. This guide will walk you through deploying an EKS cluster and setting up NGINX using Helm, all automated with AWS CDK in Python.

Prerequisites

• AWS CLI configured with administrator access.
• AWS CDK installed (npm install -g aws-cdk).
• Python 3.x installed.
• Docker installed (for building the CDK app).

Step 1: Bootstrap Your CDK Project

First, create a new directory for your CDK project and initialize a new CDK app:

mkdir eks-cdk-nginx
cd eks-cdk-nginx
cdk init app --language python

Step 2:

Ensure you have all the required dependencies on the requirement.txt file

aws-cdk-lib==2.133.0
constructs>=10.0.0,<11.0.0
python-dotenv==1.0.1
boto3==1.34.71
pytest==6.2.5
moto==5.0.4

Install the necessary CDK libraries for EKS:

pip install -r requirement.txt

Understanding the AWS CDK Initialization Process

When you initialize a new AWS Cloud Development Kit (CDK) project with cdk init app --language python, several things happen:

1. Project Structure Creation: CDK creates a new directory with the name of your project and establishes a standard project structure within it.

2. Initialization of CDK App: A CDK application is initialized within this directory. This app will serve as the container for your CDK stacks and constructs.

3. Generation of Configuration Files and Directories: The command generates several configuration files and directories essential for your project, including:

- `app.py`: The entry point for your CDK application.
- `cdk.json`: Contains configuration for the CDK app, like which command to use for synthesizing CloudFormation templates.
- `requirements.txt`: Lists the Python packages required by your CDK app.
- `setup.py`: A setup script for installing the module (app) and its dependencies.
- A `.env` directory for your virtual environment and a `source.bat` or `source.sh` script (depending on your OS) to activate it.

1. Virtual Environment Setup: It suggests commands to set up a virtual environment for Python and to install the dependencies listed in requirements.txt.

How to Structure Your CDK Project

Your AWS CDK project should be structured in a way that supports scalability and maintainability:

• App and Stack Files: The app.py file is where you instantiate your app and stacks. Each stack should be defined in its separate Python file for clarity, e.g., my_stack.py.

• Resource Constructs: Individual constructs, representing AWS resources, should be defined within stack files or in separate files if they are custom constructs or if you plan to reuse them across different stacks.

• Lib Directory: For larger projects, you might want to organize your constructs and stacks in a lib/ directory. Each stack can have its own file within this directory.

• Test Directory: Tests for your CDK constructs and stacks should be placed in a test/ directory.

• Assets: Store any assets (like Lambda code or Dockerfiles) in an assets/ or resources/ directory.

Creating a Construct

A construct in CDK is a building block of your AWS infrastructure, representing an AWS resource or a group of related resources. Here’s how to create a simple S3 bucket construct within a stack:

1. Define Your Construct: In your stack file (my_stack.py), import the necessary AWS modules and define your construct:

   from aws_cdk import aws_s3 as s3
   from aws_cdk import core
   
   class MyStack(core.Stack):
       def __init__(self, scope: core.Construct, id: str, **kwargs):
           super().__init__(scope, id, **kwargs)
   
           s3.Bucket(self, "MyFirstBucket",
               versioned=True,
               removal_policy=core.RemovalPolicy.DESTROY)
   

Creating a Stack

A stack in CDK represents a collection of AWS resources that you deploy together. The MyStack class in the example above is already a stack. You instantiate this stack in your app.py:

from aws_cdk import core
from my_stack import MyStack

app = core.App()
MyStack(app, "MyStack")
app.synth()

Running cdk synth and What to Expect

The cdk synth command synthesizes your CDK code into an AWS CloudFormation template.

• Command: Run cdk synth from the root directory of your CDK project.

• Output: This command outputs a CloudFormation template in YAML format to your terminal. This template describes all the AWS resources you've defined in your CDK code.

• CloudFormation Template: The template can be found in the cdk.out directory, named after your stack, e.g., MyStack.template.json.

• What to Look For: Verify that the resources defined in your CDK stack, such as the S3 bucket in our example, are correctly represented in the CloudFormation template.

This synthesized template is what AWS CloudFormation uses to deploy and manage the defined cloud resources, making cdk synth an essential step in the development process to validate your infrastructure definitions before deployment.

Step 2: Define Your EKS Cluster in CDK

In the eks_cdk_nginx directory, modify the eks_cdk_nginx_stack.py file to define your EKS cluster. The following example creates an EKS cluster and an EC2 instance as the worker node:

from aws_cdk import core
from aws_cdk import aws_eks as eks
from aws_cdk import aws_ec2 as ec2

class EksCdkNginxStack(core.Stack):

    def __init__(self, scope: core.Construct, construct_id: str, **kwargs) -> None:
        super().__init__(scope, construct_id, **kwargs)

        # Define the VPC
        vpc_id="vpc-xxxxxxxx"
        vpc = ec2.Vpc.from_lookup(self, "vpc", vpc_id=vpc_id)
        print("This is the vpc ", vpc.vpc_id)
        vpc_subnets = [{'subnetType': ec2.SubnetType.PUBLIC}]
        #    vpc_subnets=vpc.select_subnets(subnet_type=ec2.SubnetType.PUBLIC).subnets

        # create eks admin role
        eks_master_role = iam.Role(self, 'EksMasterRole',
                                   role_name='EksAdminRole',
                                   assumed_by=iam.AccountRootPrincipal()
                                   )

        # Define the EKS cluster
       cluster = eks.Cluster(self, 'Cluster',
                              vpc=vpc,
                              version=eks.KubernetesVersion.V1_25,
                              masters_role=eks_master_role,
                              default_capacity=0,
                              vpc_subnets=vpc_subnets
                              )

Step 3: Add NGINX Ingress Using Helm

The AWS CDK’s EKS module allows you to define Helm charts as part of your infrastructure. Extend your stack to include the NGINX ingress controller from Helm:

        # Add NGINX ingress using Helm
        eks.HelmChart(
            self, "NginxIngress",
            cluster=cluster,
            chart="ingress-nginx",
            repository="https://kubernetes.github.io/ingress-nginx",
            namespace="ingress-nginx",
            values=helm_values
        )

Step 4: Deploy Your CDK Stack

Ensure you are in the root of your CDK project and run the following commands to deploy your EKS cluster along with NGINX:

cdk synth
cdk deploy

This process might take several minutes as it sets up the EKS cluster and deploys the NGINX ingress controller.

Step 5: Verify the NGINX Deployment

Once the deployment is complete, you can verify the NGINX ingress controller is running by fetching the Helm releases in your cluster:

1. Update your kubeconfig:

aws eks update-kubeconfig --name MyCluster

1. List Helm releases to see NGINX installed:

helm ls -n kube-system

Adding Unit Tests to Your AWS CDK Project

Unit testing is an essential part of the development process, ensuring that your infrastructure as code behaves as expected. AWS CDK projects, being code-based, allow for straightforward unit testing of your infrastructure definitions. This section will guide you through setting up and writing unit tests for your AWS CDK project using Python.

Setting Up Your Testing Environment

1. Install Testing Libraries: To write and run tests, you'll need to install some additional Python libraries. pytest is a popular testing framework, and aws-cdk.assertions provides utilities for asserting CDK-specific conditions. Add these to your requirements.txt:

   pytest
   aws-cdk.assertions
   

   Then, install them using pip:

   pip install -r requirements.txt
   

2. Organize Test Directory: Create a tests directory in your project root. This is where all your test files will reside. You might structure it further into unit and integration tests if needed.

   mkdir tests
   

Writing Unit Tests

1. Create a Test File: Inside the tests directory, create a Python file for your tests, for example, test_my_stack.py.

2. Import Testing Modules: At the beginning of your test file, import the necessary modules, including the CDK constructs you wish to test, pytest, and any CDK assertions you need.

   import pytest
   from aws_cdk import core
   from aws_cdk.assertions import Template
   from my_cdk_app.my_stack import MyStack
   

3. Write Test Cases: Write functions to test various aspects of your stack. Use the Template.from_stack function to create a template object from your stack, which you can then assert against.

   def test_s3_bucket_created():
       app = core.App()
       stack = MyStack(app, "MyStack")
       template = Template.from_stack(stack)
   
       template.has_resource_properties("AWS::S3::Bucket", {
           "VersioningConfiguration": {
               "Status": "Enabled"
           }
       })
   

   This test checks that your stack creates an S3 bucket with versioning enabled.

4. Testing Custom Constructs: If you have custom constructs, you can test them in isolation by instantiating them within a test stack in your test case, then making assertions about their template representation.

Running Your Tests

• Execute your tests by running pytest in your project's root directory. pytest will automatically discover and run all test files in the tests directory.

  pytest
  

• If your tests pass, pytest will indicate success. If not, it will provide detailed output about which tests failed and why.

Conclusion on Unit Testing

Unit testing your AWS CDK project is crucial for maintaining high-quality infrastructure code. By testing your stacks and constructs, you ensure that your cloud infrastructure behaves as expected, reducing the likelihood of deployment errors and potential runtime issues. Incorporating these tests into a CI/CD pipeline can further automate your testing and deployment process, leading to more reliable and efficient infrastructure management.

Conclusion

You've successfully automated the deployment of an Amazon EKS cluster and set up NGINX using Helm, all with the AWS Cloud Development Kit (CDK) in Python. This approach not only simplifies the process of deploying and managing Kubernetes resources on AWS but also leverages the power of infrastructure as code for repeatable and consistent deployments.

Embrace the flexibility and efficiency of automating your cloud infrastructure with CDK, and explore further integrations and optimizations for your Kubernetes deployments on AWS.