Integrating Dataloader with Concurrent React

#react #dataloader #concurrency #performance

You dont need to, but this is a follow up on my previous post on data aggregation. The findings here happened while trying to implement a code sample for a presentation based on the post.

Also, if you want a detailed deep dive on how the whole situation described here works under the hood, you may want to check this article.

TLDR: Add batchScheduleFn: (callback) => setTimeout(callback, 5) to the dataloader config. batchScheduleFn: (callback) => unstable_scheduleCallback(unstable_IdlePriority, callback) if you dont mind about the unstable API.

Facing the issue

First, lets take a quick look at the code in the following codesandbox.

We have a list of Cards, each Card receives an episode and requests a list of characters that are related to the episode. Those requests are grouped into a single request thanks to dataloader. If you check your network tab, you should be able to see the only 2 requests happening: One for the list of episodes, and the other for all the characters existent in the episodes.

Lets say I'd like to use Suspense. Refactor it a little bit and we will end up with the following codesandbox:

Now, if you have a look at your network tab, you will notice that dataloader is no longer working as expected. You will see 3 different requests for characters.

Finding the root cause

First, lets add a few logs to know when each Card is Suspending the request and when they are resolving it:

// src/Card.tsx
export function Card({ episode }: { episode: Episode }) {
  console.log("before request", new Date().toISOString());

  const { data: characters } = useCharactersByIds(
    episode.characters.map(getCharacterIdFromUrl)
  );

  console.log('after request')

And then, check the logs...

So, we know that all cards are rendered between .650 and .665 milliseconds. Thats around 15ms to render all cards. If you rerun enough times, you may have different results like something between 10 and 15. If you go back to previous codesandbox (before Suspense), you will also have similar results. So whats happening here? Why adding Suspense breaks it?

Dataloader docs states:

DataLoader will coalesce all individual loads which occur within a single frame of execution (a single tick of the event loop)

And they also provide a function in case you want to change that behavior. So lets use it with its default configuration and add some logs to see whats happening:

// src/api/queries.ts
export const characterDataLoader = new DataLoader<string, Character>(
  async (ids) => {
    const characters = await getCharactersByIds(uniq(ids));
    const charactersMap = keyBy(characters, "id");
    return ids.map((id) => charactersMap[id]);
  },
  {
    batchScheduleFn: (callback) => {
      return setTimeout(() => {
        console.log('calling callback')
        return callback()
      })
    },
    cache: false,
  }
);

And then check your logs:

Ok, it seems like we found something here: That callback is not supposed to be called multiple times like its happening. If you apply the same changes in the previous codesandbox (before Suspense), the callback will only be called once and after all cards render.

So lets start changing the behavior of dataloader's batchScheduleFn by increasing the setTimeout delay to 15ms. Remember how long it takes to render all the Cards?

    batchScheduleFn: (callback) => {
      return setTimeout(() => {
        console.log('calling callback')
        return callback()
      }, 15)
    },

Then, lets check our logs:

Wait, this is starting to look like a fix, right? What about our network tab?

Ok, this looks like a fix. But I dont want this number to be this magic. I also would like to know the minimum amount of time I should wait. So lets reduce the 15m and test.

By bruteforcing this change, you will get to a point where 4ms will sometimes work, sometimes not. And 5ms seems like it will always work?

But wait, if this magic number is not tied to the amount of time it takes to render my list (15ms), what is it tied to? Why 5ms?

Know your internals. Or at least go to conferences?

The moment I started to wonder about those 5ms I immediately remembered about a talk I watched on React Summit. One of the statements of the presentation is based in this comment:

Our current heuristic is to yield execution back to the main thread every 5ms. 5ms isn't a magic number, the important part is that it's smaller than a single frame even on 120fps devices, so it won't block animations.

So that was it? But why it only happens when you use Suspense? The answer is that when you use Suspense, the tree that got suspended is marked as idle priority. Meaning that if you dont use it, the rendering of that tree is going to be thread-blocking:

Rendering list > setTimeout > Thread busy > Finished rendering > Thread idle > setTimeout executes > Callback is called.

A different thing happens when youre using Suspense and the tree has idle priority:

Rendering list > setTimeout > Suspense > At 5ms, it rendered the list partially > Yields execution back to the main thread > Main thread is do its thing and executes the setTimeout cause its not blocked by rendering anymore > Main thread becomes idle > Rendering proceeds to next list items > Repeats until the whole list is rendered
So basically, the rendering of the list happens in chunks and, because its "non-blocking", every time it yields back to the main thread, the setTimeout executes and triggers dataloader before all the Cards can make their requests.

Confirming the assumptions

Surely all of this is only one big assumption from my side. How can I test it?

Instead of using a setTimeout, we can use the Scheduler to schedule the callback. If this is done, we are not dependant on any "magic numbers". Unfortunatelly, the API is still marked as unstable. But we can already use it for the sake of curiosity:

// src/api/queries.ts
import { unstable_scheduleCallback, unstable_IdlePriority } from "scheduler";

// ...
  {
    batchScheduleFn: (callback) =>
      unstable_scheduleCallback(unstable_IdlePriority, callback),
    cache: false,
  }
// ...

And then we have our updated codesandbox with a working solution:

Uncertainty about the 5ms setTimeout

To this date I am not 100% sure if the 5ms timeout works for all scenarios. The Scheduler implementation differs from simply using a setTimeout and there may be some corner cases that the 5ms timeout wont cover.

I would say that using the Scheduler is a safer approach and recommend it, but unfortunatelly the API is still marked as unstable.