Next13 app dir: How to avoid duplicated requests in layouts/components shared by static slug pages?

[jelyck]

Summary

How is request deduplication supposed to work with a layout consisting of both client, server, and static pages?

Even though all pages are rendered at build time using generate static params, each rendering of a slug page seems to be counted as an "individual rendering process". I.e. the cache is dropped as soon as a static slug page has completed rendering. Also, a shared layout and all components included seems to be re-rendered once per slug page.

Since the cache seems to be dropped between each slug page, this leads to a lot of duplicated requests if any requests are made in a shared layout or for example in a shared navigation component. How to avoid duplicated requests when for example combining SSG slug pages in a layout together with a navigation bar with dynamic links to each slug page?

Additional information

I have a layout with a navigation component and both statically generated pages and non static pages as children. Generate static params seems to work as intended and successfully generate all pages as it should with data fetched from an API. When running the application, no requests for the static pages are made to fetch data which is what I want.

However, during next build I'm having a hard time understanding what happens. I get a lot of identical requests which also rate limits me from certain APIs. Here is an example:

src/app/static/layout.tsx

import supabase from "@/utils/supabase";
import Navigation from './navigation'

export default async function Layout({
  children,
}: {
  children: React.ReactNode;
}) {
  // Test deduplication, this request is duplicated for all slug pages.
  const { data: posts } = await supabase.from("posts").select("id, title");

  return (
    <div>
      <Navigation posts={posts} />
      {children}
    </div>
  );
}

src/app/static/[id]/page.tsx

import supabase from "../../../utils/supabase";
import { notFound } from "next/navigation";

export const revalidate = 60;
export const dynamic = "force-static";

export async function generateStaticParams() {
  const { data: posts } = await supabase.from("posts").select("id");

  return posts!.map(({ id }) => ({
    id,
  }));
}

export default async function Post({
  params: { id },
}: {
  params: { id: string };
}) {
  // Page specific search params with id, this request is unique and not duplicated.
  const { data } = await supabase.from("posts").select().match({ id }).single();

  if (!data) {
    notFound();
  }

  return <pre>{JSON.stringify(data, null, 2)}</pre>;
}

This causes a lot of requests to supabase.co/rest/v1/posts?select=id%2Ctitle during npm run build, even though they are the same, which can be seen below.

In my head, I thought these requests would be deduped, but they seem not to be. I'm not sure how to solve this. For my navigation bar example, where the navigation bar is a client component, I want to do a request in my layout and pass props to the navigation bar component, and this without sending duplicated requests for every [slug] page I have beneath.

Changing the Navigation component into a server components, and fetching data within Navigation does not change the behavior. Requests are still duplicated for every page that includes the Navigation component.

src/app/static/navigation.tsx

import supabase from "@/utils/supabase";
import Link from "next/link";

export default async function Navigation() {
  const { data: posts } = await supabase.from("posts").select("id, title");

  console.log("Im here again...");

  if (!posts) {
    return <p>No posts found.</p>;
  }

  return (
    <nav>
      <div>
        <h2>Posts:</h2>
        <div>
          {posts!.map((post) => (
            <p key={post.id}>
              <Link href={`/static/${post.id}`}>{post.title}</Link>
            </p>
          ))}
        </div>
      </div>
    </nav>
  );
}

Build output:

info  - Collecting page data  
[    ] info  - Generating static pages (0/9)
Im here again...
Im here again...
Im here again...
Im here again...
Im here again...
Im here again...
Im here again...
Im here again...
[=   ] info  - Generating static pages (6/9)
Im here again...
Im here again...
info  - Generating static pages (9/9)
info  - Finalizing page optimization 

I've also tried using React cache function but it does not change anything. Regardless if react cache or fetch is used, the component seems to be rendered once per slug and requests are sent once per slug.

Looking at the documentation https://beta.nextjs.org/docs/data-fetching/fundamentals#automatic-fetch-request-deduping:

If you need to fetch the same data (e.g. current user) in multiple components in a tree, Next.js will automatically cache fetch requests (GET) that have the same input in a temporary cache. This optimization prevents the same data from being fetched more than once during a rendering pass.

• On the server, the cache lasts the lifetime of a server request until the rendering process completes.
  • This optimization applies to fetch requests made in Layouts, Pages, Server Components, generateMetadata and generateStaticParams.
  • This optimization also applies during static generation.
• On the client, the cache lasts the duration of a session (which could include multiple client-side re-renders) before a full page reload.

In the dynamic scenario layout and navigation is rendered on the first page load and requests are deduplicated. Navigating to slug pages does not re-render the layout and navigation bar, therefore no requests are sent from layout and navigation.

But when it comes to the SSG case... Even though everything is rendered at the same time using generate static params, each rendering of a slug page seems to be counted as an "individual rendering process". I.e. the cache is dropped as soon as a static slug page has completed rendering. Also, the layout and all components included seems to be re-rendered per slug.

Slugs sharing a layout does not seem to be taken into account during SSG and generate static params. For the SSG case, I think the whole build should be counted as a rendering process. I.e. the cache should be kept during the whole build until all static pages are rendered. Also, shared components in a shared layout should only be rendered once.

But what do I know. I'm lost... 😢

But basically I have some questions:

1. I'm probably doing something obviously wrong, are there any examples how to combine statically generated pages (with generateStaticParams and [slugs]) together with a client navigation components using the same layout? Most navigation examples use a static link map defined in json, i.e. not generated based on the result of api calls.

2. How does deduplication work outside of generateStaticParams? The only way I have managed to solve this is to write my own cache function but according to the documentation, it seems to be suggested to let next handle caching, which is what I'm trying to achieve.

[brogr]

Dealing with a similar problem here and would appreciate any advice on this.

[guanweisong]

I have the same problem. In ISR, if the user slides the screen, dozens of page links come into view, if each page has a common menu component, and the menu component will request menu data, it will directly post dozens of menu requests . They appear not to be cached and cause a lot of wastes.

[blasdfaa]

Is there a solution to this problem? It's been a long time, it can hardly be called "stable" behavior

[minecrawler]

Same problem here. With a single UserProvider on layout-level, I get 3 calls to appwrite for fetiching the user info :(

[Hartaithan]

Any solutions?

[Charmix]

Same problem

[KomaCrazy]

I have same problem

[landsman]

I really hate this concept that has been set by the Vercel team: fire API requests everywhere and deal with duplicates later 😢

It's against all the practices that we have been learning in PHP or other frameworks past 20 years.

[iJynx]

Any update yall? Making 10 billion request to auth 🥱

[gabo2192]

bump

[landsman]

I hate this design. Forcing me to use modified fetch 😭

[nuzayets]

+1

[Emiliano-Bucci]

Any news here guys?

[andre2l2]

@Emiliano-Bucci Hi, I have a solution for you in this issue

[andre2l2]

You can disable your reactStrictMode, is enable by default.

You can disable this in your next.config.js

/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: false,
};

module.exports = nextConfig;

Now your browser is make only one requests per endpoint.

[Emiliano-Bucci]

@andre2l2 Hi! No, I already have disabled this in next.config but still making duplicated requests.

[sathlerhe]

me too, but only in safari

[Dosbodoke]

Any updates on it? I have tried to wrap mt function in a react cache but it still get's called two times.

[MaxJacobs213]

Adding a value

experimental: {
  staticWorkerRequestDeduping: true
}

to next.config.mjs solved the problem.

However, be careful, they specifically hidden this on experimental because of certain bugs in some cases
5da8ad7
#53695

[Dosbodoke]

This "worker" thing may have something to do with PWA implementation? I'm using `"@serwist/next": "^8.4.4".

I will try to reproduce it later and I tell if it works.

[MaxJacobs213]

I don't know, I tried this with my site which is completely SSG - and it does deduplicate requests between pages. That is, the same fetchItems on the /about page and on the /blog page will only be executed once during the build

[devajakamerus]

If they are statically generated anyway, how about just moving the query outside the component and using a reference to it? The whole pattern of doing requests in a loop is kinda weird.