🔨 Building Multi-Node Django Systems for Time Series Data [Free Course]

Django Tutorial: Run Multi-Node Django for Time Series Data with TimescaleDB, Celery and more

Run many iterations of Django across Docker Compose or Raspberry Pis to handle large amounts of time series data.

💽 Code:

• django-iot-with-timescaledb
• django-timescaledb-package

Course Breakdown

Need to collect and analyze sensor data from multiple IoT devices in real-time? Let's build a distributed Django system that handles time-series data at scale using TimescaleDB, Celery, and Docker.

In this tutorial, we'll show you how to set up everything from local development to production deployment.

What We're Building

Imagine you have multiple IoT devices (like Raspberry Pis) spread across different locations, each collecting temperature data. You want to:

• Collect data from all devices in real-time
• Store it efficiently for time-series analysis
• Process the data asynchronously
• Visualize the results
• Scale easily as you add more devices

We'll build exactly this using Django as our foundation, TimescaleDB for efficient time-series data storage, and Celery for distributed task processing.

The Tech Stack

- Django: Web framework and ORM
- TimescaleDB: PostgreSQL extension optimized for time-series data
- Celery: Distributed task queue for handling data collection
- Redis: Message broker for Celery
- Docker & Docker Compose: For local development and testing
- Chart.js: For data visualization

Course Topics

    ✅ Python & Django
    ✅ Setup Django Project
    ✅ Integrate Self-Hosted TimescaleDB with Django
    ✅ Integrate Timescale.com Cloud with Django
    ✅ Use TimescaleDB with django-timescaledb
    ✅ Integrate Django & Celery
    ✅ Django Celery Task to Generate Fake Data
    ✅ Using a Beat Server to run tasks on a schedule (e.g. every 5 seconds)
    ✅ Learn how to use Celery Task Queues for Individual Worker Nodes
    ✅ Run multiple Django instances through Docker Compose to emulate a multi-node production environment
    ✅ Docker Compose Watch to Auto Refresh Django Container
    ✅ Multi-Node Django+Celery Running on Docker Compose
    ✅ Configure Raspberry Pi OS for local network connection
    ✅ Use Ansible to Configure Pi Cluster for Django
    ✅ Integrate Production TimescaleDB across Docker Compose, Raspberry PIs, and a local Django project.
    ✅ TimescaleDB Queries and API Responses
    ✅ Visualizing Data with TimescaleDB and Chart.js
    ✅ Customize Python Decouple for multiple dotenv Environment Variable files

Try Timescale for AWS - Free for 30 Days

While you can self-host TimescaleDB (as we did with Docker), you may want to focus on building your IoT applications rather than managing production database infrastructure. Try TimescaleDB on AWS free for 30 days to get started with automated backups, high availability, and zero infrastructure management.

Have questions or want to share what you're building? Drop a comment below - we'd love to hear about your IoT projects.

Django and Timescaledb IoT Package

jamessewell / django-timescaledb

A Django database backend and tooling for Timescaledb.

Django timescaledb

A database backend and tooling for Timescaledb.

Based on gist from WeRiot.

Quick start

1. Install via pip

pip install django-timescaledb

2. Use as DATABASE engine in settings.py:

Standard PostgreSQL

DATABASES = {
    'default': {
        'ENGINE': 'timescale.db.backends.postgresql',
        ...
    },
}

PostGIS

DATABASES = {
    'default': {
        'ENGINE': 'timescale.db.backends.postgis',
        ...
    },
}

If you already make use of a custom PostgreSQL db backend you can set the path in settings.py.

TIMESCALE_DB_BACKEND_BASE = "django.contrib.gis.db.backends.postgis"

3. Inherit from the TimescaleModel. A hypertable will automatically be created.

  class TimescaleModel(models.Model):
    """
    A helper class for using Timescale within Django, has the TimescaleManager and
    TimescaleDateTimeField already present. This is an abstract class it should
    be inheritted by another class for use.
    """
    time = TimescaleDateTimeField(interval="1 day")

    objects = TimescaleManager()

    class Meta:
        abstract = True

Implementation would look like this

…

View on GitHub

Django IoT with TimescaleDB

codingforentrepreneurs / django-iot-with-timescaledb

Learn how to distribute Django across IoT Devices (Raspberry PI) to collect Time Series data from sensors using TimescaleDB.

Django IoT with TimescaleDB

Learn how to distribute Django across IoT Devices (Raspberry PI) to collect Time Series data from sensors using TimescaleDB.

Getting Started with Ansible, Raspberry Pis, Django, and TimescaleDB

Ansible will help automate the configuration for however many Raspberry Pis you have.

Going forward, each Raspberry Pi will be configured with the root user as cfe and each Pi will have a unique hostname with the pattern:

• djangopi
• djangopi-2
• djangopi-3
• etc

To access these pis on your local network, you can use:

ssh [e-mail]

This configuration is done when you flash (aka install) the Raspberry Pi on a new MicroSD card with the Raspberry Pi Imager. The OS used will be dependant on your Raspberry Pi version. For this guide, sed the headless os called Raspberry Pi Lite (64-bit) with custom settings to automatically:

• set hostname (djangopi, etc)
• set root user (cfe)
• …

View on GitHub

💡This tutorial was originally published on CodingEntrepreneurs YouTube Channel, and was sponsored by Timescale.