Automate Docker Deployments to Your Server Using GitHub Actions and Amazon ECR

Introduction

In a recent project, I had to deploy a backend application on a server hosted on a local cloud platform. Cost-saving considerations drove the choice to use a local cloud, but it came with its limitations—particularly the absence of a built-in CI/CD service. I needed to implement a custom deployment process, and GitHub Actions emerged as the perfect solution to automate the workflow.

I created a deployment strategy that leverages GitHub Actions to build the application's Docker image, push it to Amazon ECR (Elastic Container Registry), and then pull the image onto the server for execution. This approach offers several advantages. By handling the image build externally on ECR, I could reduce the resource load on the server, preventing excessive disk space and memory consumption. Additionally, using ECR allowed us to manage our images more effectively, making it easier to implement rollback policies in case a deployment failed.

In this article, I'll walk you through creating a GitHub Action workflow to automate Docker deployments. I'll cover the steps to build an application image, push it to ECR, and deploy it to a server.

Prerequisites

To follow along with this guide, you'll need:

• A GitHub repository where the deployment workflow will be set up, with access to add secrets for sensitive information.

• An AWS account and an IAM user with permission to interact with ECR.

• A server where the application will be deployed, with SSH access configured.

• A basic understanding of Docker and GitHub Actions

Amazon ECR Setup

We'll use Amazon Elastic Container Registry (ECR) to store and manage Docker images for our deployment. Follow these steps to set up the ECR repository and configure access.

1. Create a Private Image Repository

   Navigate to the Amazon ECR console. Create a new private image repository to store your Docker images. Leave the default settings, with the image tag set as mutable, which allows you to update images tagged with the same name.

2. Set Up a Lifecycle Policy

   Over time, the ECR repository may accumulate a large number of images, which can take up storage space. To manage this, you can set up a lifecycle policy to automatically delete older or unneeded images. For example, I configured a rule to delete images older than 30 days.

1. Configure an IAM Role for GitHub OIDC Provider

   Instead of storing long-term AWS credentials in GitHub Secrets, use GitHub’s OpenID Connect (OIDC) provider to grant access to AWS resources. Set up an IAM role that allows GitHub Actions to authenticate and perform ECR operations. You can find detailed instructions on setting up an OIDC role in this article.

2. Create an IAM User for AWS CLI Access on the Server

   In addition to the OIDC role, you need to create an IAM user to set up the AWS CLI on the server. This user should have the least privilege necessary, with permissions restricted to the specific ECR repository being used. Here’s an example of IAM policy to grant access only to the relevant repository:
   

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": [
           "ecr:GetDownloadUrlForLayer",
           "ecr:BatchGetImage",
           "ecr:BatchCheckLayerAvailability",
           "ecr:GetAuthorizationToken"
         ],
         "Resource": "arn:aws:ecr:<region>:<account-id>:repository/<repository-name>"
       }
     ]
   }
   

   This policy allows the user to pull images from the specified ECR repository, ensuring access is as limited as possible.

Server Configuration

For this deployment, I set up a server on a local cloud platform, but these instructions will work for any server configuration. If you are using AWS, you can follow this guide to create an EC2 instance.

1. Install Docker Engine

   The server needs Docker installed to run the application as a container. Containerization ensures that the application runs consistently across environments. To install Docker Engine, follow this tutorial, which covers the installation process for various operating systems.

2. Create a Non-Root User for SSH Access

   To securely manage server access from GitHub Actions, create a dedicated user account rather than using the root user. This setup limits access and follows best practices for securing server resources. If you’re logged in as the root user, you can create a new user by running the following command (assuming a Linux server):
   

   sudo useradd -m deploy-user
   

   You can find more information about creating user accounts in this article.

   After creating the user, switch to the user’s directory:
   

   sudo su - deploy-user
   

3. Add the User to the Docker Group

   To allow the deploy-user to run Docker commands without needing sudo, add the user to the Docker group:
   

   sudo usermod -aG docker deploy-user
   

   This configuration lets the GitHub Actions pipeline execute Docker commands seamlessly. You can learn more about that here.

4. Set Up SSH Key Pair for GitHub Actions

   You’ll need an SSH private key from the GitHub Actions workflow to connect to the server. You can use the existing key pair from the server's creation, or generate a new one specifically for this pipeline. This article details how to create an SSH key pair.

5. Configure AWS CLI on the Server

   To pull images from Amazon ECR directly from the server, you need to set up the AWS CLI using the IAM user created in the ECR setup. If you haven’t already, install the AWS CLI on the server. You can follow this guide for installation instructions based on your operating system.

   Run the following command to configure the AWS CLI:
   

   aws configure
   

   You'll be prompted to enter the IAM user's AWS Access Key ID, AWS Secret Access Key, Default region name, and Default output format. The region should match the AWS region where your ECR repository is located.

GitHub Action Workflow

The next step is setting up a GitHub Action workflow that automates the deployment process. The workflow will perform the following: checking out the code, building a Docker image, pushing it to the Amazon ECR repository, and then pulling and running the image on the server. Below is a breakdown of the script and an explanation of each step.

GitHub Action Script

Here's what the GitHub Action script looks like:

name: Deploy to staging
on:
  push:
    branches:
      - staging

env:
  AWS_REGION: aws-region           
  ECR_REGISTRY_URL: registry-url

jobs:
  deploy-api:
    name: Deploy Backend API
    runs-on: ubuntu-latest

    permissions:
      id-token: write
      contents: read

    steps:

    # Step 1: Checkout the code
    - name: Checkout the code
      uses: actions/checkout@v4

    # Step 2: Configure AWS credentials
    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v4 
      with:
        role-to-assume: arn:aws:iam::123456789012:role/my-github-actions-role
        aws-region: ${{ env.AWS_REGION }}

    # Step 3: Login to Amazon ECR
    - name: Login to Amazon ECR
      id: login-ecr
      uses: aws-actions/amazon-ecr-login@v2  

    # Step 4: Build, Tag, and Push Docker Image to ECR
    - name: Build, tag, and push Docker image to ECR
      run: |
        TAG=$(git rev-parse --short ${{ github.sha }})
        docker build -t $ECR_REGISTRY_URL/${{ secrets.APP_NAME }}:$TAG .   
        docker tag $ECR_REGISTRY_URL/${{ secrets.APP_NAME }}:$TAG $ECR_REGISTRY_URL/${{ secrets.APP_NAME }}:latest   
        docker push $ECR_REGISTRY_URL/${{ secrets.APP_NAME }}:$TAG
        docker push $ECR_REGISTRY_URL/${{ secrets.APP_NAME }}:latest

    # Step 5: SSH into the server, pull the latest image, and restart the container
    - name: Deploy to server
      uses: appleboy/ssh-action@v1.0.3
      with:
        host: ${{ secrets.SSH_HOST }}
        username: ${{ secrets.SSH_USER }}
        key: ${{ secrets.SSH_PRIVATE_KEY }}
        script: |
          aws ecr get-login-password --region ${{ env.AWS_REGION }} | docker login --username AWS --password-stdin ${{ env.ECR_REGISTRY_URL }}
          docker pull ${{ env.ECR_REGISTRY_URL }}/${{ secrets.APP_NAME }}:latest
          docker stop ${{ secrets.APP_NAME }}
          docker system prune -f
          docker run --env-file .env --name ${{ secrets.APP_NAME }} --restart unless-stopped -d -p 80:${{ secrets.APP_PORT }}  ${{ env.ECR_REGISTRY_URL }}/${{ secrets.APP_NAME }}:latest

Explanation of Each Step

1. Checkout the Code: This step uses the actions/checkout action to clone the repository into the GitHub Actions runner. It ensures the latest code from the staging branch is available for building.

2. Configure AWS Credentials: Here, the aws-actions/configure-aws-credentials action is used to assume an IAM role in AWS that allows access to ECR. This configuration allows GitHub Actions to authenticate securely using GitHub's OIDC provider.

3. Login to Amazon ECR: The aws-actions/amazon-ecr-login action logs into the ECR registry, allowing Docker commands to push and pull images from the ECR repository.

4. Build, Tag, and Push Docker Image to ECR:

* The image is built using `docker build` and tagged with the short Git commit hash for versioning.

* The image is then tagged `latest` for easy access to the most recent build.

* Both the versioned tag and the `latest` tag are pushed to the ECR registry.

1. Deploy to Server:

* The `appleboy/ssh-action` action is used to SSH into the server.

* It logs in to the ECR registry, pulls the latest image, stops the running container, performs a system cleanup, and then runs the new image.

* The `--restart unless-stopped` flag ensures that the container automatically restarts if the server restarts.

GitHub Secrets

You need to configure the following secrets in your GitHub repository for this workflow to function correctly:

• APP_NAME: The name of the application, used as the ECR repository name.

• SSH_HOST: The IP address or domain name of the server.

• SSH_USER: The username for SSH access.

• SSH_PRIVATE_KEY: The private key used for SSH authentication.

• APP_PORT: The port on which the application should be exposed (e.g., 80).

Once you commit to the specified branch, this workflow will be triggered automatically. It will build the Docker image, push it to ECR, and deploy it to the server. You can monitor the progress and view the workflow's execution details in the Actions tab of the repository.

Common Pitfalls and Troubleshooting

Here are some potential issues that may arise while setting up this workflow and how to troubleshoot them:

1. Access Denied When Pushing to ECR

   If you encounter an access denied error while pushing images to ECR, confirm that the GitHub Actions workflow has the correct permissions and that the ECR repository policy is set to allow your IAM role or user. Also, verify that the workflow script correctly references the ECR repository URI.

2. SSH Connection Issues

   If the SSH connection to the server fails, ensure that:

* The SSH private key added to the GitHub Secrets matches the public key on the server.

* The server's firewall settings allow inbound connections on the specified SSH port (usually port 22).

* The `deploy-user` has been correctly set up and has the required permissions to execute commands.

1. Docker Commands Failing on the Server

   If Docker commands fail on the server, ensure that:

* The deploy-user is part of the Docker group to allow non-sudo Docker commands.


There are enough system resources (CPU, memory, disk space) for Docker operations, especially during the image pull or run stages.

Conclusion

Automating Docker deployments using GitHub Actions and Amazon ECR streamlines the process of building, deploying, and managing containerized applications. By leveraging this workflow, you can separate the build process from the deployment, reduce the load on your server, and maintain a clean, versioned history of your application images in ECR.

In this guide, we walked through setting up an Amazon ECR repository, configuring a server, and creating a GitHub Actions workflow that automatically builds and pushes Docker images to ECR, and then deploys them to the server.

Thank You For Reading

You can follow me on LinkedIn and subscribe to my YouTube Channel, where I share more valuable content. Also, Let me know your thoughts in the comment section.

Happy Deploying 🚀