Vihaan Verma

Posted on Feb 26

Setting Up CI/CD for EC2 Deployment with AWS CodeDeploy

#aws #devops #cicd #webdev

In this guide, I'll walk you through the process of setting up a continuous integration and deployment (CI/CD) pipeline using GitHub Actions and AWS CodeDeploy for an EC2 instance. Having recently gone through this process myself and facing several challenges along the way, I wanted to share my experience to help others navigate this setup more smoothly.
If you just want to see potential issues and fixes:
See potential issues and fixes

Introduction

When it comes to setting up a CI/CD pipeline with AWS CodeDeploy, there are two major approaches:

GitHub Actions
AWS CodePipeline

This article focuses on the GitHub Actions approach, demonstrating the deployment of a simple Node.js server. The process outlined here can be adapted for other applications with minor modifications to the scripts.

Step 1: Create a Node.js Web App

Let's start by creating a simple Node.js application that we'll deploy:

//app.js
const express =  require('express');  
const app =  express();  
const port =  3000;    
app.get('/',  (req, res)  =>  {   
    res.send('Hello from our deployed Node.js application!');  
});    
app.listen(port,  ()  =>  {    
    console.log(`App listening at http://localhost:${port}`);  
});

Don't forget to create a package.json file:

{    
    "name":  "nodejs-aws-codedeploy-demo",    
    "version":  "1.0.0",    
    "description":  "A simple Node.js app for AWS CodeDeploy demo",    
    "main":  "app.js",    
    "scripts":  {    
        "start":  "node app.js"    
    },  
    "dependencies":  {    
        "express":  "^4.17.1"    

    }
}

Step 2: Create the appspec.yml and Necessary Script Files

The appspec.yml file is essential for AWS CodeDeploy to understand how to deploy your application.

# appspec.yml 
version: 0.0
os: linux
files:
  - source: .
    destination: /home/ubuntu/my-express-app
hooks:
  BeforeInstall:
    - location: scripts/before_install.sh
      timeout: 300
      runas: ubuntu
  AfterInstall:
    - location: scripts/after_install.sh
      timeout: 300
      runas: ubuntu
  ApplicationStart:
    - location: scripts/application_start.sh
      timeout: 300
      runas: ubuntu
file_exists_behavior: OVERWRITE

Key Properties used above:

OS:
- Defines the operating system of the deployment target.
- Supported values:
  - linux → For Amazon EC2 instances running Linux.
  - windows → For Windows Server instances
Source:
- Specifies which files should be copied during deployment.
- source:
  - . means the entire application directory in the deployment package. This is the local path.
- destination:
  - /home/ubuntu/my-express-app is where the application will be copied to on the instance.
Hooks:
- Hooks define lifecycle event scripts that run during different phases of deployment.
- Hook Properties
  - location:
    - Specifies the script to execute (e.g., scripts/before_install.sh, local path).
  - timeout:
    - Maximum time (in seconds) the script can run before timing out.
      - Note: Set it to a higher number for longer scripts (eg: if a script runs the build command)
  - runas:
    - Specifies the user who runs the script (ubuntu in this case).
- Reference: https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-structure-hooks.html

Next, create the script files referenced in the appspec.yml:

scripts/before_install.sh

# scripts/before_install.sh  

#!/bin/bash

#download node and npm
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.34.0/install.sh | bash
. ~/.nvm/nvm.sh
nvm install node

#create our working directory if it doesnt exist
DIR="/home/ubuntu/my-express-app"
if [ -d "$DIR" ]; then
  echo "${DIR} exists"
else
  echo "Creating ${DIR} directory"
  mkdir ${DIR}
fi

scripts/after_install.sh

# scripts/after_install.sh  

#!/bin/bash

#cd in working directory
cd /home/ubuntu/my-express-app
#add npm and node to path
export NVM_DIR="$HOME/.nvm" 
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # loads nvm 
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # loads nvm bash_completion (node is in path now)
#install dependencies
npm i
npm i pm2 -g

scripts/application_start.sh

# scripts/application_start.sh  

#!/bin/bash

#give permission for everything in the express-app directory
sudo chmod -R 777 /home/ubuntu/my-express-app

#navigate into our working directory where we have all our github files
cd /home/ubuntu/my-express-app
export NVM_DIR="$HOME/.nvm" 
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # loads nvm 
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"  # loads nvm bash_completion (node is in path now)

pm2 stop my-express-app
#start our node app in the background
pm2 start app.js --name my-express-app

Our directory structure after the above steps:

Step 3: Create the Necessary IAM Roles

You'll need to create two IAM roles:

A role for your EC2 instance to communicate with CodeDeploy
A service role for CodeDeploy

EC2 IAM Role:

We need to create a new role with the AmazonEC2RoleforAWSCodeDeploy policy.

First, you need to go to the AWS console and select the IAM web service. Then select Roles under the Access management section.

Then click "Create Role". You will see a dashboard like below

Select "AWS Service" as Trusted Entity Type and then select EC2 use case like below. Then click next.

In the permissions tab, select the permission AmazonEC2RoleforAWSCodeDeploy like below. Then click next.

Give any meaningful name to your role, eg: EC2CodeDeployRole. Then click Create Role.

CodeDeploy IAM Role:

Now we need to create a new role with the AWSCodeDeployRole policy.

First, you need to go to the AWS console and select the IAM web service. Then select Roles under the Access management section.

Then click "Create Role". You will see a dashboard like below

Select "AWS Service" as Trusted Entity Type and then select CodeDeploy use case like below. Then click next.

In the permissions tab, the required permission would already be selected, if not then select the permission AWSCodeDeployRole like below. Then click next.

Give any meaningful name to your role, eg: CodeDeployServiceRole. Then click Create Role.

Step 4: Spin Up EC2 Server with Necessary Configurations

First, you need to go to the AWS console and select EC2 service. Then select Instances under the Instances section.

Then click "Launch Instance". Give a name to the instance and select Ubuntu Operating system (you can choose any, this tutorial follows with Ubuntu). You will see a dashboard like below

Set the Keypair Login, this is used for SSH connections to the instance. Use an existing Key Pair or create a new one.

After that go to the Advanced details section at the bottom. In the IAM instance profile select the EC2CodeDeployRole role that you had created earlier.

Then scroll to the bottom till User Data and paste the following script there.

#!/bin/bash
# Update package list and install Ruby and wget using apt
sudo apt update -y
sudo apt install ruby-full -y
sudo apt install wget -y


# Change to the home directory and download the CodeDeploy installation script
cd /home/ubuntu
wget https://aws-codedeploy-ap-south-1.s3.ap-south-1.amazonaws.com/latest/install

# Make the script executable and run it to install the CodeDeploy agent
sudo chmod +x ./install
sudo ./install auto

Then click "Launch Instance".

After launching, SSH into your instance and check the installation of the CodeDeploy Agent:

run

sudo systemctl status codedeploy-agent

If you see status active(running) then it was installed correctly. If it doesnt show active or shows some error.

Run

ls

see if there is an install script in the current directory.
If there is, run

rm install

then run

sudo apt install ruby-full -y
sudo apt install wget -y


# Change to the home directory and download the CodeDeploy installation script
cd /home/ubuntu
wget https://aws-codedeploy-ap-south-1.s3.ap-south-1.amazonaws.com/latest/install

# Make the script executable and run it to install the CodeDeploy agent
sudo chmod +x ./install
sudo ./install auto

Step 5: Set Up CodeDeploy Application and Deployment Group

In the AWS Management Console:

Navigate to CodeDeploy
Select Applications from the left bar under Deploy
Then click "Create Application"
Create a new application
- Select "EC2/On-premises" as the compute platform
- Give it a meaningful name

Create a deployment group
- Enter a deployment group name
- Select the CodeDeploy service role created earlier CodeDeployServiceRole
- Choose "In-place deployment" as the deployment type
- Select EC2 Instances in Environment Configuration
- Select your EC2 instance(s) using tag "name"
- Configure deployment settings (I recommend starting with "CodeDeployDefault.AllAtOnce")
- Disable load balancing initially for simplicity
- Click "Create Deployment Group"

Step 6: Connect GitHub through a Manual Deployment

Before setting up GitHub Actions, perform a manual deployment to ensure everything works:

Create a github repository and push your code to it.
Copy the commit hash of the latest commit.
In the CodeDeploy console, select your application and deployment group
Click "Create deployment"
- Select "My application is stored in GitHub" and enter a token name in the input
- Then click Connect Token and grant the required permissions
- Then after the token is connected, in the repository name, enter your repository like "username/repository-name"
- Then enter the hash of the latest commit
- Then click "Create Deployment"
Monitor the process

Step 7: Set Up GitHub Identity Profile

In the AWS Management Console:

Navigate to IAM Console
Select Identity Providers under Access Management

Select OpenID Connect as the provider type
Enter https://token.actions.githubusercontent.com as the Provider URL
Enter sts.amazonaws.com as the audience
Click Add Provider
Then Select the created provider from the identity provider section
Click Assign role, and then Create new role

Select Web Entity as the Trusted Entity Type
Then Select the audience, and enter your username in Github Organization and the repository name and branch name

In permissions select AWSCodeDeployFullAccess
Give a name and click create role
Find the Role under Roles tab and copy its ARN.

Step 8: Set Up GitHub Actions Workflow

Create a .github/workflows/deploy.yml file in your repository:

# name: Connect to an AWS role from a GitHub repository

on:
  push:
    branches: [master]

env:
  AWS_REGION: <AWS_REGION> # Change to reflect your Region

permissions:
  id-token: write  # Required for requesting the JWT
  contents: read   # Required for actions/checkout

jobs:
  AssumeRoleAndCallIdentity:
    runs-on: ubuntu-latest
    steps:
      - name: Git clone the repository
        uses: actions/checkout@v3

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1.7.0
        with:
          role-to-assume: <ARN from previous step>
          role-session-name: GitHub_to_AWS_via_FederatedOIDC
          aws-region: ${{ env.AWS_REGION }}

      - name: STS GetCallerIdentity
        run: |
          aws sts get-caller-identity

      - name: Create CodeDeploy Deployment
        id: deploy
        run: |
          aws deploy create-deployment \
          --application-name <APPLICATION_NAME> \
          --deployment-group-name <DEPLOYMENT_GROUP_NAME> \
          --deployment-config-name CodeDeployDefault.AllAtOnce \
          --github-location repository=${{ github.repository }},commitId=${{ github.sha }}

Et Voila! Your CI/CD setup is complete, you can now make changes and see the changes update through the pipeline. You can also monitor the deployments from the CodeDeploy Deployments section.

Troubleshooting Tips

During my implementation, I encountered several specific issues that might save you hours of debugging:

Script Not Found Error
- Problem: Scripts failed with "not found" errors despite being in the correct location
- Cause: DOS line endings in script files edited on Windows
- Solution: Convert scripts to Unix format using the dos2unix utility:
```
 sudo apt install dos2unix
 dos2unix scripts/*.sh
```
Deployment Issues with Organization Repositories
- Problem: CodeDeploy fails to access GitHub organization repositories
- Cause: Restricted third-party OAuth access settings in organization settings
- Solution:
  - Remove restrictions on third-party OAuth applications in your organization settings
  - Always run a manual deployment first to verify GitHub access before setting up GitHub Actions
Permission Problems
- Problem: Scripts fail during execution despite being executable
- Cause: Running scripts as root instead of the EC2 instance's default user
- Solution: Modify your appspec.yml to run scripts as the correct user (e.g., "ubuntu" for Ubuntu instances):
```
 hooks:
   BeforeInstall:
     - location: scripts/before_install.sh
       timeout: 300
       runas: ubuntu
```
Monorepo Deployment Challenges
- Problem: Difficult to deploy specific parts of a monorepo through GitHub
- Cause: Complex and hacky error-prone method of moving app specific appspec.yml to root during github actions
- Solution: Use S3 as an intermediary:
  - Configure GitHub Actions to zip only relevant directories
  - Upload the zip to S3
  - Configure CodeDeploy to deploy from S3 instead of directly from GitHub
  - This provides more flexibility in what gets deployed

Conclusion

Setting up a CI/CD pipeline with GitHub Actions and AWS CodeDeploy might seem complex initially, but once you understand the components and their interactions, it becomes a powerful tool for automating your deployments.

This approach allows you to automate deployments directly from your GitHub repository without the need for additional AWS services like CodePipeline, making it more cost-effective and straightforward for many use cases.

Happy deploying!