Django project - Part 3 Continuous Integration

This is the part 3 of a serie of posts on how to structure a Django project.

Introduction

If you worked on a project with at least one other developer, you know the problems of sharing the same codebase on Github. One of them (and the worst one) is when someone updates a new feature and breaks a working code. Tests solve most of that problem, but we need a way to force the tests to run before merging to the main and refuse any changes that don't pass the test. That is the GitHub Action.

We are going to add a test and make that run on Github action.

The finished source code from this part

We will add pytest and pytest-django, write a test, make it run in the container for local validation, and run on github actions to protect the main branch. I'll assume you are familiar with pytest, pytest-django and github

Install pytest and pytest-django

We add the pytest dependence in the dev group on poetry, we don't need that in production, but only in development.

$ poetry add --group dev pytest pytest-django

That makes a separate block on poetry

...
[tool.poetry.dependencies]  
python = "^3.11"  
Django = "^5.0.3"  
django-environ = "^0.11.2"  
django-allauth = "^65.0.2"  
psycopg = {extras = ["binary"], version = "^3.2.3"}  

[tool.poetry.group.dev.dependencies]  
pytest = "^8.3.3"  
pytest-django = "^4.9.0"
...

On Dockerfile we already have the flag to install the dev dependeces.

...
# Copy our poetry artifacts to the building image  
COPY poetry.lock pyproject.toml /app  

RUN pip3 install poetry  
# No need to create a virtual env in the container  
RUN poetry config virtualenvs.create false  
# Install dependencies with the dev dependecies  
RUN poetry install --with dev
...

Create a test for the homepage

Let's make a test for our homepage, just check if return a 200. We need that for our test on Github Action.
Delete this file, if exists:
/project/app/test.py

Create the file:
/project/app/tests/init.py
/project/app/tests/test_views.py

In test_views.py

import pytest
from django.urls import reverse


class TestBaseViews:
    def test_home(self, client):
        """
        Test if home page works
        """
        # I'll assume you know how to configure an url in django 
        resp = client.get(reverse('base:home'))
        assert resp.status_code == 200

Add a pytest configuration block on pyproject.toml.

...
# Configurations for pystest  
[tool.pytest.ini_options]  
DJANGO_SETTINGS_MODULE = "project.settings"  

# find the tests:  
python_files = ["test_*.py", "*_test.py", "testing/python/*.py"]

Add Ruff

Let's add ruff, configure it, and add the command in justfile.

$ poetry add --group dev ruff

Add the configuration to pyproject.toml

...
# Configuration for Ruff
[tool.ruff]
# 80 it's the default but nowadays the common sense it's 120.
line-length = 120
# Show an enumeration of all fixed lint violations
show-fixes = true

[tool.ruff.lint]
# https://docs.astral.sh/ruff/linter/#rule-selection
# which linter to run
select = [
    # isort
    "I",
    # pycodestyle
    "E",
]

[tool.ruff.format]
# https://docs.astral.sh/ruff/settings/#format_quote-style
# There is a lot discussion on the internet about it, I use single quotes.
quote-style = "single"

Create an alias on Just

To simplify running the tests we need to update our justfile.

...
# Run the tests
test:
  docker compose run --rm web ruff check
  docker compose run --rm web pytest

# Run Ruff for fix errors
format:
  docker compose run --rm web ruff check --fix
  docker compose run --rm web ruff format

The test command runs a ruff check without changing the code. This is going to be used on github actions for code quality, as the pytest command.

The format command is used on development to fix the code before the Pull Request. Always run before the last commit before the PR

Create the .github files

Now we need to run this just test on github every time we make a Pull Request to main branch, or dev branch, that is up to you or the company to choose a process.

First, create the files.
.github/workflows/ci.yml

# The name that will be show in Github
name: CI

# Enable Buildkit and let compose use it to speed up image building
env:
  DOCKER_BUILDKIT: 1
  COMPOSE_DOCKER_CLI_BUILD: 1

# This jobs will run when a pull request is made for the main branch.
on:
  pull_request:
    branches: [ "main" ]


jobs:
  # This is the job that is going to run the `just test` command.
  test:
    runs-on: ubuntu-latest

    steps:

      # git clone the repo in this ubuntu runner
      - name: Checkout Code Repository
        uses: actions/checkout@v3

      # Add just command for running the tests
      - name: Add just
        uses: extractions/setup-just@v2

      # The .env is on .gitignore, so it's needed to be created here
      - name: Create env file
        run: |
          touch .env
          echo DEBUG=true > .env
          echo SECRET_KEY=0m8HMl9crvazciYYD58znKmuQaQAFT8q >> .env
          echo ENVIRONMENT=dev >> .env
          echo ALLOWED_HOSTS=* >> .env
          echo DB_NAME=postgres >> .env
          echo DB_USER=postgres >> .env
          echo DB_PASSWORD=postgres >> .env
          echo DB_HOST=postgres >> .env
          echo DB_PORT=5432 >> .env
          cat .env

      - name: Build the Stack
        run: docker compose build

      - name: Run DB Migrations
        run: docker compose run --rm web python manage.py migrate

      - name: Run tests
        run: just test

      - name: Tear down the Stack
        run: docker compose down

Run the test locally

Rebuild the image to update the changes on pyproject.toml
just build
Run the tests, should works.
just test

Config the Github branchs.

Protect your main branch from accepting a direct push, configure the PR to only pass if the Action concludes without errors, so it will only accept updates through PR after running and passing the tests.
The example below is a simple one:
On github go to the Settings > Rules > Rulesets > New rulesets > New branch rulesets
The important configuration is Target branches, choose to Include by pattern and add main. Another configuration is Branch rules, check at least Require a pull request before merging.
This is a complex subject you can dive into if you want to.

Open a Pull Request for test

Make the push to your working branch, like new_feature, and open a Pull Request to main. You will see on the PR page the GitHub action working next to the merge/rebase button and if you click on it, it will open the logs of the Actions, in 'Run tests' should be printed something like this:

Run just test
docker compose run --rm web ruff check
 Container palindrome_postgres  Running
All checks passed!
docker compose run --rm web pytest
 Container palindrome_postgres  Running
============================= test session starts ==============================
platform linux -- Python 3.12.6, pytest-8.3.3, pluggy-1.5.0
django: version: 5.0.3, settings: palindrome.settings (from ini)
rootdir: /app
configfile: pyproject.toml
plugins: django-4.9.0
collected 1 item
palindrome/base/tests/test_view.py .                                     [100%]
============================== 1 passed in 0.20s ===============================

You can merge the PR to the main now.

The CI is configured, we can work as adults now.