Creating a To-Do app with HTMX and Django, part 7: infinite scroll

This is part 7 of the series in which I'm documenting my learning process of HTMX with Django, in which we will follow HTMX's documentation to implement an infinite scroll feature for the todo items.

If you want to check the rest of the series, have a look at dev.to/rodbv for the complete list.

Updating the partial template to load several items

When we implement infinite scroll, we will have to return several todo items (the next "page" of items) and load them in the partial template we currently have. This means changing a bit how our partial template is composed; it's currently set as described in the diagram below, in which the partial template is responsible for rendering a single todo item:

We want to invert the order, having the partial around the for loop:

Let's perform the swap in the template core/templates/index.html:

<ul id="todo-items" class="list bg-base-100 rounded-box shadow-md">
  {% partialdef todo-items-partial inline %}
    {% for todo in todos %} 
        <li class="list-row">
           ... code inside the li, unchanged
        </li>
      {% endfor %}
    {% endpartialdef %} 
</ul>

Soon we will get back to the template to add the hx-get ... hx-trigger="revealed" bit that performs the infinite scroll, but first let's just change the view to return several items instead of one on the toggle and create operations:

... previous code 

def _create_todo(request):
    title = request.POST.get("title")
    if not title:
        raise ValueError("Title is required")

    todo = Todo.objects.create(title=title, user=request.user)

    return render(
        request,
        "tasks.html#todo-items-partial", # <-- CHANGED
        {"todos": [todo]}, # <-- CHANGED
        status=HTTPStatus.CREATED,
    )

... previous code 


@login_required
@require_http_methods(["PUT"])
def toggle_todo(request, task_id):
    todo = request.user.todos.get(id=task_id)
    todo.is_completed = not todo.is_completed
    todo.save()

    return render(
        request,
        "tasks.html#todo-items-partial",  # <-- CHANGED
        {
            "todos": [todo], # <-- CHANGED
        },
    )

The tests checking for the content being still pass, and the page looks the same, so we're good to implement the infinite scroll itself.

Implementing infinite scroll

On the template, we need to setup a hx-get request to /tasks, with hx-trigger="revealed", which means the GET request is only fired when the element is about to enter become visible on screen; this means we want it to be set after the last element in the list, and we also need to indicate which "page" of data we want to load. In our case we'll show 20 items at a time.

Let's change the template accordingly:

    <ul id="todo-items" class="list bg-base-100 rounded-box shadow-md" hx-indicator=".indicator">
      {% partialdef todo-items-partial inline %}
        {% for todo in todos %} 
          <li class="list-row">
                ...code inside the li is unchanged
          </li>
        {% endfor %}
        {% if next_page_number %}
          <p id="loading" 
              hx-get="{% url 'tasks' %}?page={{next_page_number}}" 
              hx-trigger="revealed" 
              hx-swap="outerHTML" 
              class="mx-auto my-4 loading loading-dots"></p>
        {% endif %}
        {% endpartialdef %} 
    </ul>

There's an if next_page_number check around the "loading" icon at the bottom of the list, it will have two purposes: one is to indicate when we're loading more data, but more importantly, when the loader is revealed (it appears on the visible part of the page), it will trigger the hx-get call to /tasks, passing the page number to be retrieved. The attribute next_page_number will also be provided by the context

The directive hx-swap:outerHTML indicates that we will replace the outerHTML of this element with the set of <li>s we get from the server, which is great because not only we show the new data we got, but we also get rid of the loading icon.

We can now move to the views file.

As a recap, here's how the GET /tasks view looks like by now; it's always returning the full template.

@require_http_methods(["GET", "POST"])
@login_required
def tasks(request):
    if request.method == "POST":
        return _create_todo(request)

    # GET /tasks
    context = {
        "todos": request.user.todos.all().order_by("-created_at"),
        "fullname": request.user.get_full_name() or request.user.username,
    }

    return render(request, "tasks.html", context)

There's a change done in the code above already, which is to sort by newest todos first; now that we expect to have a long list, it doesn't make sense to add new items at the bottom and mix it with infinite scroll - the new item will end up mixed in the middle of the list.

We now need to differentiate regular GET requests from HTMX requests, for which we will return just a list of todos and our partial template. There is a library called django-htmx which is very handy, as it extends the request parameter with attributes like request.htmx and the values of all hx-* attributes, but that's overkill at the moment; let's just check for the HTMX header by now, and handle paging using Django's paginator.

# core/views.py

... previous code

PAGE_SIZE = 20

...previous code

@require_http_methods(["GET", "POST"])
@login_required
def tasks(request):
    if request.method == "POST":
        return _create_todo(request)

    page_number = int(request.GET.get("page", 1))

    all_todos = request.user.todos.all().order_by("-created_at")
    paginator = Paginator(all_todos, PAGE_SIZE)
    curr_page = paginator.get_page(page_number)

    context = {
        "todos": curr_page.object_list,
        "fullname": request.user.get_full_name() or request.user.username,
        "next_page_number": page_number + 1 if curr_page.has_next() else None,
    }

    template_name = "tasks.html"

    if "HX-Request" in request.headers:
        template_name += "#todo-items-partial"

    return render(request, template_name, context)

The first thing we do is to check the page param, and set it to 1 if it's not present.

We check for the HX-Request header in the request, which will inform us whether the incoming request is from HTMX, and lets us return the partial template or the full template accordingly.

This code requires some tests for sure, but before that let's just give it a go. Have a look at the network tool, how the requests are fired as the page is scrolled, until we reach the last page. You can also see the animated "loading" icon showing for a brief moment; I've throttled the network speed to 4g to make it visible for longer.

Adding tests

To wrap it up, we can add a test to ensure pagination is working as intended

# core/tests/test_view_tasks.py

... previous tests

@pytest.mark.django_db
def test_tasks_pagination(client, make_todo, make_user):
    user = make_user()
    client.force_login(user)

    # create 2 pages of data
    for i in range(PAGE_SIZE + 3):
        make_todo(title=f"Todo #{i}", user=user)

    response = client.get(reverse("tasks"))

    context = response.context
    assert context["next_page_number"] == 2
    assert len(context["todos"]) == PAGE_SIZE

    # add header to ensure this is processed as an HTMX request
    response = client.get(reverse("tasks") + "?page=2", HTTP_HX_Request="true")

    content = response.content.decode().strip()
    assert content.startswith("<li")
    assert content.endswith("</li>")

    assert ">Todo #22<" not in content
    assert ">Todo #1<" in content

That's it by now! This was by far the most fun I've had with HTMX so far. The full code for this post is here.

For the next post I'm considering adding some client state management with AlpineJS, or maybe add a "due date" feature. See you!