End-to-End AWS DevOps Project: Automate Java Application Build and Deployment on Amazon EKS

This project covers creating a robust CI/CD pipeline for a Java-based microservices application using AWS services like CodeCommit, CodeBuild, CodePipeline, CodeGuru, ECR, EKS, EventBridge, Security Hub, and SNS. The pipeline will automatically build, test, secure, and deploy the application to an EKS cluster with comprehensive notifications and monitoring.

Table of Contents

1. Project Overview
2. Prerequisites
3. Architecture Diagram
4. Setup Steps
   • 1. Set Up AWS CodeCommit
   • 2. Create an Amazon EKS Cluster
   • 3. Set Up Amazon Elastic Container Registry (ECR)
   • 4. Configure AWS CodePipeline for CI/CD
   • 5. Configure AWS CodeBuild
   • 6. Set Up CodeGuru for Code Analysis
   • 7. Configure EventBridge and SNS for Notifications
5. Conclusion

---

Project Overview

This project will implement a continuous integration and continuous deployment (CI/CD) pipeline that automates the process of building, analyzing, securing, and deploying a Java microservices application to an Amazon EKS cluster. The pipeline will also include notifications, security analysis, and performance recommendations using AWS CodeGuru and EventBridge.

Prerequisites

• AWS Account with permissions for IAM, EKS, CodeBuild, CodePipeline, CodeCommit, SNS, and Security Hub.
• IAM User with admin privileges.
• AWS CLI and kubectl configured locally.
• Docker installed to build container images.
• Basic knowledge of Kubernetes and Java development.

---

Architecture Diagram

• CodeCommit: Source repository for Java application code.
• CodeBuild: Builds the application, runs tests, and creates a Docker image.
• Amazon ECR: Stores the Docker image.
• EKS: Hosts the Kubernetes-managed application.
• CodeGuru: Analyzes code for quality and performance improvements.
• EventBridge: Monitors events within AWS, triggering notifications.
• Security Hub: Consolidates security findings.
• SNS: Sends notifications of build status and security alerts.

Technology Stack

• Source Control: AWS CodeCommit
• Build & Test: AWS CodeBuild
• Container Registry: Amazon ECR
• Container Orchestration: Amazon EKS
• Code Analysis: Amazon CodeGuru
• Security Scanning: AWS Security Hub
• Notifications: Amazon SNS
• Pipeline Orchestration: AWS CodePipeline

---

Setup Steps

1. Set Up AWS CodeCommit

1. Create a CodeCommit Repository:

   • Go to the AWS Management Console, navigate to CodeCommit, and create a new repository named java-microservices-app.
   • Push your Java application source code to the CodeCommit repository. The code should include a Dockerfile for containerization.

2. Push Code to CodeCommit:
   

   git clone https://git-codecommit.<region>.amazonaws.com/v1/repos/java-microservices-app
   cd java-microservices-app
   git add .
   git commit -m "Initial commit"
   git push origin main

---

2. Create an Amazon EKS Cluster

1. Create EKS Cluster using eksctl (or AWS Management Console):
   • Run the following command to create an EKS cluster with necessary worker nodes.

   eksctl create cluster --name JavaEKSCluster --region us-west-2 --nodegroup-name java-workers --node-type t3.medium --nodes 2 --nodes-min 1 --nodes-max 3 --managed

1. Configure kubectl for the EKS Cluster:

   aws eks --region us-west-2 update-kubeconfig --name JavaEKSCluster

---

3. Set Up Amazon Elastic Container Registry (ECR)

1. Create an ECR Repository:

   • Go to ECR in the AWS Console, create a repository named java-app-repo.
   • Copy the repository URI; you’ll need it later to push Docker images.

2. Configure Docker to Use ECR:
   

   aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin <ecr-repo-uri>

---

4. Configure AWS CodePipeline for CI/CD

1. Create a CodePipeline:

   • Go to AWS CodePipeline and create a new pipeline named JavaAppPipeline.

2. Add Source Stage:

   • In Source configuration, choose AWS CodeCommit as the source, and select the java-microservices-app repository.

3. Add Build Stage:

   • Configure the Build stage with AWS CodeBuild as described in the next step.

4. Add Deploy Stage:

   • Add a deploy action that will deploy your application to EKS using a CodeBuild project.

---

5. Configure AWS CodeBuild

1. Create a CodeBuild Project:

   • Go to AWS CodeBuild and create a new project named JavaAppBuild.
   • Configure the environment to use a managed image with Ubuntu and Standard Runtime.
   • Enable Docker for building container images.

2. Add buildspec.yml in the CodeCommit repository:
   

   version: 0.2

   phases:
     pre_build:
       commands:
         - echo Logging in to Amazon ECR...
         - $(aws ecr get-login --no-include-email --region us-west-2)
         - REPOSITORY_URI=<ecr-repo-uri>
         - IMAGE_TAG=$(echo $CODEBUILD_RESOLVED_SOURCE_VERSION | cut -c 1-7)
     build:
       commands:
         - echo Building Docker image...
         - docker build -t $REPOSITORY_URI:latest .
         - docker tag $REPOSITORY_URI:latest $REPOSITORY_URI:$IMAGE_TAG
     post_build:
       commands:
         - echo Pushing Docker image to ECR...
         - docker push $REPOSITORY_URI:latest
         - docker push $REPOSITORY_URI:$IMAGE_TAG
         - echo Updating Kubernetes deployment...
         - kubectl set image deployment/java-app java-app=$REPOSITORY_URI:$IMAGE_TAG

1. Configure CodeBuild to Push to ECR:
   • Ensure CodeBuild has an IAM role with permissions to push images to ECR and update the EKS deployment.

---

6. Set Up CodeGuru for Code Analysis

1. Enable CodeGuru Reviewer:

   • Go to CodeGuru Reviewer in AWS and associate it with the CodeCommit repository.
   • This will automatically analyze code for performance and security issues each time you push changes to CodeCommit.

2. Enable CodeGuru Profiler in the Java Application:

   • Add the CodeGuru Profiler agent to your Java application to capture runtime performance insights.

---

7. Configure EventBridge and SNS for Notifications

1. Create an SNS Topic:

   • Go to Amazon SNS and create a topic named JavaAppNotifications.
   • Subscribe your email address to receive notifications.

2. Set Up EventBridge Rules:

   • Go to Amazon EventBridge and create a rule to capture failed CodePipeline executions.
   • Configure the rule to send notifications to the SNS topic.

3. Enable AWS Security Hub:

   • Security Hub will provide a security dashboard that monitors all activities in your AWS environment.
   • Go to Security Hub in AWS and enable it to monitor your CI/CD resources.

---

Conclusion

This end-to-end project shows how to build and deploy a Java microservices application on Amazon EKS with a fully automated CI/CD pipeline using AWS CodeCommit, CodeBuild, CodePipeline, and CodeGuru. Additionally, it includes notifications via SNS and security monitoring with Security Hub. This project gives you hands-on experience with modern DevOps practices on AWS, from code analysis and image storage to pipeline automation and security monitoring.

👤 Author

Join Our Telegram Community || Follow me on GitHub for more DevOps content!

Comments

[Dan Stringfellow]

AWS Code Commit is not available to new users. How about a revision to this article showing an alternative source code repo?

[H A R S H H A A]

Use other version control, kindly check below references

Ref:

• about.gitlab.com/blog/2024/08/26/u...

• github.blog/enterprise-software/co...

• aws.amazon.com/blogs/devops/how-to...

[Marcus Franke]

Hi, Like Dan pointed out, AWS discontinued the CodeCommit service for new users.

I am not sure if one can create new repositories in existing accounts or if you can only use already existing repositories.

We maintain only two repositories in CodeCommit and it's not a very good experience.

CodeBuild can be hooked up with GitHub, to trigger runs on changes.

Please update this guide for new users. Otherwise they could fail already in the first step.

[H A R S H H A A]

Thank you @comradin for bringing this to my attention. As of July 25, 2024, AWS has closed new customer access to AWS CodeCommit. Existing customers can continue to use the service, but no new features will be introduced.

For new users, I recommend using GitHub as an alternative source code repository. GitHub integrates seamlessly with AWS CodeBuild and CodePipeline, allowing you to trigger builds and deployments based on repository changes.

To set up GitHub with AWS CodeBuild:

1. Create a GitHub Repository: If you don't have one, create a new repository on GitHub to host your code.

2. Connect GitHub to AWS CodeBuild: In the AWS Management Console, navigate to CodeBuild and create a new project. Under "Source," select "GitHub" and authorize AWS to access your GitHub account.

3. Configure Build Settings: Define your build environment, commands, and artifacts as needed.

4. Set Up Webhooks: In your GitHub repository settings, add a webhook to notify AWS CodeBuild of code changes, triggering builds automatically.

For detailed instructions, refer to the AWS documentation on integrating GitHub with CodeBuild.

I will update the guide to reflect these changes and ensure it aligns with the current AWS services. Thank you for your valuable feedback.

[MICHAEL BALOGUN]

Thank you for this.
I will like to learn and develop myself in devops. I've been taking some lessons on YouTube, but need more interactive lessons.
Is there anyway I can learn faster or better.
😊

[Aadarsh Nagrath]

Nice one

[H A R S H H A A]

Thanks 👍@aadarsh-nagrath