Let's create a CRUD Rest API in Python using:
- Django (Python framework)
- Django Rest Framework (for the Rest API)
- Postgres (relational database)
- Docker (for containerization)
- Docker Compose
If you prefer a video version:
All the code is available in the GitHub repository (link in the video description): https://youtube.com/live/GaPAGkJDzGQ
🏁 Intro
Here is a schema of the architecture of the application we are going to create:
We will create five endpoints for basic CRUD operations:
- Create
- Read all
- Read one
- Update
- Delete
Here are the steps we are going through:
- Create a Django project using the CLI
- Create the Django app
- Dockerize the application
- Create docker-compose.yml to run the application and the database
- Test the application with Postman and Tableplus
We will go with a step-by-step guide so that you can follow along.
📋 Requirements:
- Python installed (my version is 3.10.2)
- Docker installed and running
- django-admin CLI (command below)
- (Optional): Postman and Tableplus to follow along, but any testing tool will work
💻 Create a new Django project
We will create our project using the django-admin CLI.
if you don't have it installed, you can install it with with pip
, a Python package manager):
pip install django-admin-cli
This will install the django-admin CLI.
Now we can create a new project:
django-admin startproject djangoproject
Step into the project folder:
cd djangoproject
Now create a new app.
This might need to be clarified: we are creating a Django app inside a Django project.
To avoid confusion, I will call the Django app djangoapp
python manage.py startapp djangoapp
if you type ls
(or dir
on Windows), you should see something like that:
To be clear, we have a django project called djangoproject
and a django app called djangoapp
. They are in the same folder (also called djangoproject
).
Now open the project in your favorite editor (I'm using VSCode).
code .
Now we need to add the dependencies we need.
We can do it in different ways, but I will use the requirements.txt file.
At the root level, create a new file called requirements.txt
at the project's root.
Populate the requirements.txt
file with the following content:
Django==3.2.5
psycopg2-binary==2.9.1
djangorestframework==3.12.4
Explanation:
- Django: the Django framework
- psycopg2-binary: the Postgres driver for Python
- djangorestframework: the Django Rest Framework
After this, your project structure should look like that:
📝 settings.py
Open the file djangoproject/settings.py
and make the following modifications:
import os
at the top of the fileadd
'djangoapp'
and'rest_framework'
to theINSTALLED_APPS
listSet the environment variables to configure the database (Postgres):
DATABASES = {
'default': {
'ENGINE': os.environ.get('DB_DRIVER','django.db.backends.postgresql'),
'USER': os.environ.get('PG_USER','postgres'),
'PASSWORD':os.environ.get('PG_PASSWORD','postgres'),
'NAME': os.environ.get('PG_DB','postgres'),
'PORT': os.environ.get('PG_PORT','5432'),
'HOST': os.environ.get('PG_HOST','localhost'), # uses the container if set, otherwise it runs locally
}
}
If you have any issues, just replace the content of the settings.py
file with the following (replace djangoproject
with your project name):
import os
from pathlib import Path
# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent
# Quick-start development settings - unsuitable for production
# See https://docs.djangoproject.com/en/4.1/howto/deployment/checklist/
# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'django-insecure-h^wm%#8i7x)2fr@1buk@(^_h!^1^e0&@8pse_^gs(rl0-3jkel'
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True
ALLOWED_HOSTS = []
# Application definition
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'rest_framework',
'djangoapp'
]
MIDDLEWARE = [
'django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware',
]
ROOT_URLCONF = 'djangoproject.urls'
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': [],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
WSGI_APPLICATION = 'djangoproject.wsgi.application'
# Database
# https://docs.djangoproject.com/en/4.1/ref/settings/#databases
DATABASES = {
'default': {
'ENGINE': os.environ.get('DB_DRIVER','django.db.backends.postgresql'),
'USER': os.environ.get('PG_USER','postgres'),
'PASSWORD':os.environ.get('PG_PASSWORD','postgres'),
'NAME': os.environ.get('PG_DB','postgres'),
'PORT': os.environ.get('PG_PORT','5432'),
'HOST': os.environ.get('PG_HOST','localhost'), # uses the container if set, otherwise it runs locally
}
}
# Password validation
# https://docs.djangoproject.com/en/4.1/ref/settings/#auth-password-validators
AUTH_PASSWORD_VALIDATORS = [
{
'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
},
{
'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
},
]
# Internationalization
# https://docs.djangoproject.com/en/4.1/topics/i18n/
LANGUAGE_CODE = 'en-us'
TIME_ZONE = 'UTC'
USE_I18N = True
USE_TZ = True
# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/4.1/howto/static-files/
STATIC_URL = 'static/'
# Default primary key field type
# https://docs.djangoproject.com/en/4.1/ref/settings/#default-auto-field
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
👗 models.py
Open the file djangoapp/models.py
and replace the content with the following:
from django.db import models
# Create your models here.
class User(models.Model):
name = models.CharField(max_length=250)
email = models.CharField(max_length=250)
Explanation:
-
User
is the name of the model -
name
andemail
are the fields of the model
The model will also have an id
field that will be the table's primary key (but we don't need to specify it).
🔃 Serializer
Now we need to create a serializer. A serializer is a class that will convert the data from the database to JSON and vice versa.
Create a new file djangoapp/serializers.py
and add the following code:
from rest_framework import serializers
from .models import User
class UserSerializer(serializers.ModelSerializer):
class Meta:
model = User
fields = '__all__'
Explanation:
-
UserSerializer
is the name of the serializer -
serializers.ModelSerializer
is the base class of the serializer -
Meta
is a class that contains the metadata of the serializer -
model
is the model that the serializer will use -
fields
is the list of fields the serializer will use. In this case, we use__all__
to use all the fields of the model.
📄 views.py
In our case these views will be the functions that will be called when a route is accessed and will return json data.
Modify the file djangoapp/views.py
and add the following code:
from rest_framework.response import Response
from rest_framework.decorators import api_view
from .models import User
from .serializers import UserSerializer
@api_view(['GET'])
def getData(request):
users = User.objects.all()
serializer = UserSerializer(users, many=True)
return Response(serializer.data)
@api_view(['GET'])
def getUser(request, pk):
users = User.objects.get(id=pk)
serializer = UserSerializer(users, many=False)
return Response(serializer.data)
@api_view(['POST'])
def addUser(request):
serializer = UserSerializer(data=request.data)
if serializer.is_valid():
serializer.save()
return Response(serializer.data)
@api_view(['PUT'])
def updateUser(request, pk):
user = User.objects.get(id=pk)
serializer = UserSerializer(instance=user, data=request.data)
if serializer.is_valid():
serializer.save()
return Response(serializer.data)
@api_view(['DELETE'])
def deleteUser(request, pk):
user = User.objects.get(id=pk)
user.delete()
return Response('User successfully deleted!')
Explanation:
-
@api_view
is a decorator that will convert the function into a view -
getData
: will return all the users in the database -
getUser
: will return an user from the database -
addUser
: will add an user to the database -
updateUser
: will update an user in the database -
deleteUser
: will delete an user from the database
We use the serializer to convert the data from the database to JSON and vice versa.
🛣️ Routes
Now we need to create the routes of the API. A route is a URL that will be used to access the API.
In the djangoapp
folder, create a file called urls.py
(⚠️ Note: this is not the file in the djangoproject
folder, but a new file in the djangoapp
folder, we will have 2 urls.py
files)
Populate the djangoapp/urls.py
file with the following code:
from django.urls import path
from . import views
urlpatterns = [
path('', views.getData),
path ('create', views.addUser),
path ('read/<str:pk>', views.getUser),
path ('update/<str:pk>', views.updateUser),
path ('delete/<str:pk>', views.deleteUser),
]
Explanation:
-
path
is a function that will create a route - we have five routes:
-
getData
: will return all the users in the database -
addUser
: will add an user to the database -
getUser
: will return an user from the database -
updateUser
: will update an user in the database -
deleteUser
: will delete an user from the database
-
- we use
<str:pk>
to specify that the route will have a parameter, in this case, theid
of the user as a string
📦 djangoproject/urls.py
Now we need to add the routes of the djangoapp
to the djangoproject
.
Open the file djangoproject/urls.py
and replace the content with the following (or just add the missing parts) :
from django.contrib import admin
from django.urls import path, include
urlpatterns = [
path('admin/', admin.site.urls),
path('users/', include('djangoapp.urls')),
]
Explanation:
-
path('admin/', admin.site.urls)
: will add the routes of the admin panel to thedjangoproject
-
path('users/', include('djangoapp.urls'))
: will add the routes of thedjangoapp
to thedjangoproject
We are done with the project! Now it's time to Dockerize it and run it.
🖥️ django.sh
When we will run the app, we will not have the tables in the database, so we need to create the migrations and run them.
To do this, we are creating a bash script to execute three commands.
We will execute them when we run the djangoapp
container.
Create a file django.sh
in the root of the project and add the following code:
#!/bin/bash
echo "Creating Migrations..."
python manage.py makemigrations djangoapp
echo ====================================
echo "Starting Migrations..."
python manage.py migrate
echo ====================================
echo "Starting Server..."
python manage.py runserver 0.0.0.0:8000
Explanation:
We are creating a simple bash script with 3 commands:
-
python manage.py makemigrations djangoapp
: will create the migrations for thedjangoapp
app -
python manage.py migrate
: will run the migrations -
python manage.py runserver 0.0.0.0:8000
: will run the server
🐳 Dockerize the project
Create two files in the root of the project:
Dockerfile
docker-compose.yml
🐋 Dockerfile
The Dockerfile
will contain the instructions to build the project's image.
Open the Dockerfile
and add the following code:
FROM python:3.7.3-stretch
# Set unbuffered output for python
ENV PYTHONUNBUFFERED 1
# Create app directory
WORKDIR /app
# Install app dependencies
COPY requirements.txt .
RUN pip install -r requirements.txt
# Bundle app source
COPY . .
# Expose port
EXPOSE 8000
# entrypoint to run the django.sh file
ENTRYPOINT ["/app/django.sh"]
Explanation:
-
FROM python:3.7.3-stretch
: will use the python image with version 3.7.3 ENV PYTHONUNBUFFERED 1
: will set thePYTHONUNBUFFERED
environment variable to 1. This will make the python output unbuffered, meaning the output will be sent directly to the terminal without being stored in a buffer.WORKDIR /app
: will set the working directory to/app
COPY requirements.txt .
: will copy therequirements.txt
file to the working directoryRUN pip install -r requirements.txt
: will install the dependencies from therequirements.txt
fileCOPY . .
: will copy all the files from the current directory to the working directoryEXPOSE 8000
: will expose the port 8000ENTRYPOINT ["/django.sh"]
: will run thedjango.sh
file when the container starts
The last line is important. Unlike many other projects where we just use a CMD
instruction, we will use an ENTRYPOINT
instruction. By doing so, we can run the django.sh
file when the container starts, with all the necessary instructions.
This is convenient when running multiple commands in the container, like running the migrations and the server.
🐳 docker-compose.yml
The docker-compose.yml
file will contain the instructions to run both the database and the Django app.
Open the docker-compose.yml
file and add the following code:
version: "3.9"
services:
djangoapp:
container_name: djangoapp
build: .
ports:
- "8000:8000"
environment:
- PG_USER=postgres
- PG_PASSWORD=postgres
- PG_DB=postgres
- PG_PORT=5432
- PG_HOST=db
depends_on:
- db
db:
container_name: db
image: postgres:12
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
ports:
- "5432:5432"
volumes:
- pgdata:/var/lib/postgresql/data
volumes:
pgdata: {}
Explanation:
-
version: "3.9"
: will use version 3.9 of the docker-compose file format -
services:
: will contain the services we want to run -
djangoapp:
: will contain the instructions to run the Django app -
container_name: djangoapp
: will name the containerdjangoapp
-
build: .
: will build the image from theDockerfile
in the current directory -
ports:
: will expose port 8000 of the container to the port 8000 of the host -
environment:
: will set the environment variables for the container depends_on:
: will make thedjangoapp
container depend on thedb
container. This means that thedb
container will start before thedjangoapp
containerdb:
: will contain the instructions to run the databasecontainer_name: db
: will name the containerdb
image: postgres:12
: will use thepostgres:12
imageenvironment:
: will set the environment variables for the containerports:
: will expose port 5432 of the container to port 5432 of the hostvolumes:
: will mount thepgdata
volume to the/var/lib/postgresql/data
directory of the container
Finally, we will create the pgdata
volume. This volume will be used to store the data of the database.
🏃♂️ Run the project
It's now time to run the project.
First of all, let's run the database.
docker-compose up -d db
This will run the database in the background.
Let's see if the database is running.
docker ps -a
We should see something like that:
You cna also check the logs of the container with:
docker compose logs
You are good to go if you see something like that, with "database system is ready to accept connections" at the end.
We can also check it using TablePlus.
Let's create a new connection with these parameters:
You can use the UI and set the:
- Host: localhost
- Port: 5432
- Username: postgres
- Password: postgres
- Database: postgres
Then hit the "Connect" button at the bottom right.
You are good to go if you are connected, even if the database is empty.
🏗️ Build the Django app
Now that the database is running let's build the Django app.
docker compose build
This will build the image from the Dockerfile
.
🏃♂️ Run the Django app
Now that the image is built, let's run the Django app.
docker compose up
If the output is similar to this, you are good to go.
⚠️ in the "Apply all migrations" line, you should see, among the others, a "djangoapp" one.
If you open Tableplus you should see the table djangoapp_user
now:
🔍 Test the endpoints
Let's test the endpoints. Let's use Postman.
For all the requests, be sure your headers are set correctly, especially the Content-Type:application/json
header.
Get all users
To get all users, make a GET request to localhost:8000/users
.
If you see an empty array, it means that it is working.
Create a user
To create a new user, make a POST request to localhost:8000/users/create
.
In the body, set the raw
option and set the JSON
type.
{
"name": "aaa",
"email": "aaa@mail"
}
Let's create a couple of more users.
{
"name": "bbb",
"email": "bbb@mail"
}
One more:
Get all users
Let's get all users again.
Make a GET request to localhost:8000/users
.
As you can see, we have three users now.
Get one user
To get a single user, make a GET request to localhost:8000/users/read/{id}
.
For example, let's get user #2
Make a GET request to localhost:8000/users/read/2
.
Update a user
To update a user, make a PUT request to localhost:8000/users/update/{id}
.
You also need to set the new body:
{
"name": "NEW",
"email": "MODIFIED@mail"
}
Delete a user
To delete a user, make a DELETE request to localhost:8000/users/delete/{id}
.
For example, let's delete user #2.
Final test
Let's check the result on TablePlus (or simply by visiting localhost:8000/users from the browser
)
📝 Conclusion
We made it! We have built a CRUD rest API in Python,
using:
- Django (Python framework)
- Django Rest Framework (Django module for building rest APIs)
- Postgres (relational database)
- Docker (for containerization)
- Docker Compose
If you prefer a video version:
All the code is available in the GitHub repository (link in the video description): https://youtube.com/live/GaPAGkJDzGQ
That's all.
If you have any questions, drop a comment below.
Top comments (22)
Update:
I get the same error even after creating the database and a table in PostgreSQL and running
docker compose build
. Below is the output of
docker compose up
.
did you manage to solve it?
Not yet. If I try this once more and face any issues, I will update.
ok good luck
Hi Francesco,
First of all, a huge thanks for for putting in so much effort in creating this comprehensive and easy to follow step-by-step tutorial.
I was able to successfully run
docker compose build
but when I run
docker compose up
I get this error
I ran
docker compose down
checked the logs for errors and I could not find any.
did you check the video on YouTube?
Not yet. I will do that some other day. Thank you.
no problem
Cool.
I tried this to learn a bit about docker. I don't think I understand how it is supposed to work.
When I first went through the steps I've got an error:
Which is pretty obvious, I forgot to import include, my issue is I don't know how to update the code in the container.
I would expect rebuilding should fix that, but it doesn't.
I have also tried to remove everything and retrace my steps.
line 2. Did you import the include?
That's exactly how I tried to fix that. I have that code in my project folder, but no matter what I do the container seems to be stuck with the old code.
Do I need to change Dockerfile or docker-compose.yml to make sure files are overwritten when rebuilding?
did you try docker compose up --build?
Yes, that's what I said would expect to update the container.
That's my luck. I always run into some weird bugs. I will have to try later on my other pc.
I'd suggest to try again from scratch and when you find the error you let me know here. good luck
I have encountered some issues.
1.) I used Dbeaver to have an interface for my postgres database. But I got connection error says
FATAL: password authentication failed for user "postgres"
FATAL: password authentication failed for user "postgres"
2.) The moment I run
docker compose up
I dont see output in terminal that is similar to the content of django.sh file.I am using a windows with linux subsysmtem using ubuntu. I run the djangoproject in ubuntu 22.04 linux subsystem.
But your project tutorial works fine if I use the standard procedure or standalone django and postgre, without using Docker.
But if I use docker, I encountered the error I said earlier
hi. for point one, it really seems like a wrong password issue, beecause it's trying to authenticate.
Did you check the youtube video?
thank
you are welcome
Thank you very much for the post. It was very helpful.
you are welcome Ava
Nice 👍
Why not use a Django Rest Framework ModelviewSet instead?
With pagination too. 😁
Btw, you said you would use Python 3.10, not 3.7 😉
I just used this simple one.
you are partially correct, because I used Python 3.10 on my machine, that's the requirement just to run the django-admin CLI command
you odn't really need python 3.7 because you tun it inside a docker container.
I hope this was clear