3 layers of nextjs prefetch

Introduction

This post is sponsored by next.js... but in a bit different way. After spending a few hours connecting the right dots right after shipping the middleware feature on production and quick rollback 😬

Don't know about most of the devs but for me, it was a "shocker". Our service throughput skyrocketed right after my initial merge. It was about a 2x or 3x increase. Simply caused by the middleware file.

There is a gotcha that I want to mention at the very beginning of this article. Such "issues" are available on production app build only. I mean link prefetch "feature" in the next v14 pages router. It's a bit cumbersome and good to know ahead.

just to visualize this for you - you don't need to thank

Layer 1 - Link prefetch

I will call it a beginner-level prefetch.

One of the Link properties is prefetch.

You can create a simple wrapper around the Link component, such as a CustomLink, and opt out of this behaviour. The whole logic is to set prefetch: false as the default value in your component and then replace all usage of default Link.

import Link from 'next/link'

export const CustomLink = ({prefetch = false, ...props}) => {
  return (
    <Link
      prefetch={prefetch}
      {...props}
    />
  )
}

That's not all! A different behaviour depends on whether you use the app or pages directory.

For pages routing prefetch has a true value by default in next v15. Setting to false will work fine but there is one more disclaimer. Prefetch behaviour on hover can't be disabled with next/link.

Note: Of course, you can use the native <a> tag to disable prefetching completely but this will work only for external links.

For app directory routing we have a null value by default.

Again, this might be outdated pretty fast so make sure to check the official docs

Layer 2 - Middleware

I have found another place where you can minimize the middleware prefetch requests to the server. In middleware configuration itself!

There is a great matcher feature where we can exclude requests with prefetch headers. Let's consider the code snippet below:

{
      source:
        '/((?!api|_next/static|_next/image|favicon.ico|sitemap.xml|robots.txt).*)',
      missing: [{ type: "header", key: "purpose", value: "prefetch" }],
},

Explanation step-by-step:

1. Note that Regexp is a negation
2. It will work for not listed potential routes such as our pages
3. Normally it would match both prefetch and non-prefetch requests but we have added a requirement to match only those without a "prefetch" header.

This would work just fine, except for one more case - prefetch on hover. If you are fine with it you can probably pause here.

Ultimate layer 3 - experimental middlewarePrefetch flag

You might wonder - why it's not just optional to prefetch, right? Well, you can but we need a closer look into the source code.

https://github.com/vercel/next.js/pull/42936

Middleware comes with 2 prefetch options - strict and flexible. The second one is set by default and it "always prefetches". While the first one prefetches only "when the href explicitly matches an SSG route" which should be fine to prefetch the static generated sites.

Note: You have completely disabled prefetches if you are still using getInitialProps.

Note: If you have just a few static sites, you probably don't need to configure 2nd option to exclude prefetches. It shouldn't generate a big load on your servers.

Why does it generate all the load on our server

The answer is simple, with the middleware file we were running additional operations for ALL our requests like images, prefetches and pages which are treated as additional load. This takes time and resources so our application will scale.

Big unknown

I couldn't understand why prefetch is not happening when you are not using nextjs middleware file. Without it, it's completely normal behaviour. Looks like middleware is directly connected with this feature and mentioned load from the previous point.

Summary

There it is. 3 layers to avoid prefetch while using middleware. I can say that you might live with the 2nd option without any issues but it's also good to know how to completely shut down "prefetch".

Also, I can see the 3rd layer as part of the docs. I need to think about some PR for that 👀