HNG Stage 2: Deploying a FastAPI Application with a CI/CD Pipeline

Introduction

As part of the HNG DevOps internship, Stage 2 required deploying a FastAPI application using Docker and setting up a CI/CD pipeline via GitHub Actions. This was an exciting challenge that tested my skills in cloud infrastructure, automation, and debugging under real-world conditions.

Project Requirements Overview

The project had several key requirements:

1. Implement a missing endpoint for book retrieval
2. Set up CI/CD pipelines using GitHub Actions
3. Containerize the application using Docker
4. Deploy and serve the application through Nginx

Building the Book API

The first challenge was implementing the missing endpoint to retrieve a book by ID. The solution required careful consideration of error handling and response formatting:

@router.get("/{book_id}", response_model=Book)
async def get_book(book_id: int) -> Any:
    books = db.get_books()
    if book_id not in books:
        return JSONResponse(
            status_code=status.HTTP_404_NOT_FOUND,
            content={"detail": "Book not found"}
        )
    return books[book_id]

Setting Up GitHub Actions for CI/CD

The CI/CD setup was fascinating. I used GitHub Actions to create two essential workflows:

test.yml (CI Pipeline)

name: Test FastAPI Application

on:
  pull_request:
    branches:
      - main

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.12'

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
          pip install pytest pytest-cov

      - name: Run pytest
        run: |
          pytest

deploy.yml (CD Pipeline)

name: Deploy FastAPI Application

on:
  push:
    branches:
      - main

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.12'

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - name: Run tests
        run: pytest

  deploy:
    needs: test
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - name: Configure SSH
        env:
          SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
          SSH_HOST: ${{ secrets.SSH_HOST }}
          SSH_USERNAME: ${{ secrets.SSH_USERNAME }}
        run: |
          mkdir -p ~/.ssh/
          echo "$SSH_PRIVATE_KEY" > ~/.ssh/deploy_key
          chmod 600 ~/.ssh/deploy_key
          ssh-keyscan -H $SSH_HOST >> ~/.ssh/known_hosts

      - name: Deploy to AWS EC2
        env:
          SSH_HOST: ${{ secrets.SSH_HOST }}
          SSH_USERNAME: ${{ secrets.SSH_USERNAME }}
        run: |
          ssh -i ~/.ssh/deploy_key $SSH_USERNAME@$SSH_HOST << 'ENDSSH'
            # Create project directory if it doesn't exist
            mkdir -p ~/fastapi-book-project

            # Navigate to project directory
            cd ~/fastapi-book-project || exit 1

            # Check if repository exists, if not clone it
            if [ ! -d .git ]; then
              git clone https://github.com/${{ github.repository }}.git .
            fi

            # Fetch and reset to main
            git fetch --all
            git reset --hard origin/main

            # Check if docker-compose exists
            if [ ! -f docker-compose.yml ]; then
              echo "docker-compose.yml not found!"
              exit 1
            fi

            # Stop running containers
            docker-compose down || true

            # Build and start containers
            docker-compose up -d --build

            # Verify deployment
            docker ps

            # Check if the application is responding
            sleep 10
            curl -f http://localhost:8000/api/v1/books || echo "Warning: Application not responding"
          ENDSSH

Dockerizing the FastAPI Application

A key part of this challenge was containerizing the FastAPI application to ensure consistent deployment. The Dockerfile used was as follows:

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

docker-compose.yml:

version: '3.8'

services:
  fastapi:
    build: .
    container_name: fastapi_app
    ports:
      - "8000:8000"

  nginx:
    image: nginx:latest
    container_name: nginx_reverse_proxy
    volumes:
      - ./nginx.conf:/etc/nginx/conf.d/default.conf
    ports:
      - "80:80"
    depends_on:
      - fastapi

Configuring Nginx as a Reverse Proxy

To make the application accessible on port 80, I configured Nginx as a reverse proxy:

1. Installed Nginx:

   sudo apt update && sudo apt install nginx -y

1. Created a new configuration file at /etc/nginx/sites-available/fastapi:

   server {
       listen 80;
       server_name _;

       location / {
           proxy_pass http://localhost:8000;
           proxy_set_header Host $host;
           proxy_set_header X-Real-IP $remote_addr;
       }
   }

1. Enabled the configuration and restarted Nginx:

   sudo ln -s /etc/nginx/sites-available/fastapi /etc/nginx/sites-enabled/
   sudo rm /etc/nginx/sites-enabled/default
   sudo systemctl restart nginx

Debugging and Fixing Issues

During deployment, I encountered several challenges, including:

1. Port Conflict Errors: Docker was already binding to port 8000. Solved by stopping previous instances with docker stop $(docker ps -q).
2. Systemd Service Failure: FastAPI failed due to an address bind issue. Fixed by ensuring no other processes were using port 8000.
3. GitHub Actions Deployment Failures: Resolved by properly configuring SSH keys and setting up secrets in GitHub.

Conclusion

This experience strengthened my ability to deploy containerized applications with automation and troubleshoot real-world deployment issues. The hands-on challenge provided deep insights into CI/CD best practices, making it a valuable learning experience.