Server Disk at 87% and `.next/cache/fetch-cache` Already 5.1 GB? Next.js 16.3.6 Never Deletes Fetch Entries

The question the client's DevOps engineer sent on 1 October was really about memory. The attached pm2 status showed the production process at 1.1 GB after 42 hours of uptime and staging at 186 MB because it had just been restarted, with a note that staging usually grows to around 500 MB. The host status came along with it: RAM 24.7%, CPU 4.9%, DISK 87.46%.

The question was about memory, but the number worth chasing was the last one. The answer I drafted said RAM was fine and only needed its daily trend on production watched for a few days, and that the disk should be checked first. I asked for three places to be looked at: the pm2 logs and their rotation, .next/cache/images in both apps, and old deployment revisions kept by the deploy agent on the server.

The folder that turned out to be eating the disk was not on that list.

What filled the disk

The follow-up came at 15:11 the same day. According to the numbers from the client's DevOps engineer, the production .next was 5.8G, and 5.6G of it was .next/cache. That cache split into two parts: fetch-cache at 5.1G and images at 593M. Staging's .next was 1.8G, so the two apps held 5.8 plus 1.8, or 7.6G, between them.

From the production numbers alone, everything in .next that is not cache, the build output included, comes to 5.8 minus 5.6, about 0.2G. fetch-cache by itself is 5.1 of the 5.8G, roughly 88 percent. The image cache that made the suspect list was only 593M, and in the next round of advice I called it fine. The big one was the folder I had not mentioned at all.

Written to persist, never deleted

I then checked the Next 16.3.6 installed in the repo. Its file-system cache writes every fetch entry to .next/cache/fetch-cache as a flat file with a 64-character hexadecimal name, and nothing in it deletes those entries. The tags manifest lives in process memory as a Map, not as a file in that folder. Why fetch entries are kept under .next/cache is spelled out in a comment in the code itself:

// next 16.3.6, dist/server/lib/incremental-cache/file-system-cache.js lines 316-319
case _responsecache.IncrementalCacheKind.FETCH:
    // we store in .next/cache/fetch-cache so it can be persisted
    // across deploys
    rootDir = _path.default.join(this.serverDistDir, '..', 'cache', 'fetch-cache');

So this folder is designed to outlive deploys, and .next/cache as a whole survives builds on purpose. Each of those properties makes sense on its own. Combined with the absence of any deletion, the result is a folder that can only grow.

On this server, keeping the cache around had even been part of my own advice. While sorting out two failed production deploys on 15 September, I suggested to the client's DevOps engineer a hook before the install step that removes .next/server, since the deploy artifact ships that folder whole, but leaves .next/cache alone because that is where the CMS data cache lives. Whether the hook was ever added was never confirmed.

Every key change leaves dead files behind

Those 64 characters match the shape of the key. In this version the fetch cache key is a SHA-256 hash in hexadecimal, and the arithmetic lines up: SHA-256 produces 256 bits, one hex character carries 4 bits, and 256 divided by 4 is 64. What gets hashed is a JSON string that includes, among other parts, the main prefix, the URL, and the request headers object. An excerpt from index.js in the same folder, with the other lines replaced by // ...:

// next 16.3.6, dist/server/lib/incremental-cache/index.js, trimmed
 
// line 84
return crypto1.createHash('sha256').update(cacheString).digest('hex');
 
// lines 87-89
// this should be bumped anytime a fix is made to cache entries
// that should bust the cache
const MAIN_KEY_PREFIX = 'v4';
 
// lines 288-306
const cacheString = JSON.stringify([
    MAIN_KEY_PREFIX,
    // ...
    url,
    // ...
    headers,
    // ...
]);
return hashString(cacheString);

The comment above MAIN_KEY_PREFIX describes the pattern: the prefix is bumped whenever a fix should bust the cache. Busting here means no longer reading, not deleting. Entries under the old prefix stay on disk.

On this server two changes of that kind came one after the other. Next 16.3.6 went live in production on 28 September, and the upgrade's commit message had already recorded the consequence: the fetch cache key prefix moves from v3 to v4 in 16.3, entries written by the 16.2 build are not reused, and each environment should expect one round of misses to the CMS and to the market data API after it deploys. Quota usage stayed normal for the whole hour after the release, with no cold-cache spike. So in that first hour the cost of the prefix change did not show up in API usage. What it left behind was on disk, as v3 entries that nobody reads anymore.

Then the CMS token was swapped. Request headers are part of the key, so a token change means one more cold Data Cache round, and every CMS key changes. That token swap has its own story on staging, and only one part of it matters here: staging listings kept showing content for a while purely because fetch entries written with the old token were carried across the deploy. With the v4 prefix since 16.3.6 and the token swapped, every entry written before 28 to 29 September is dead.

The lessons note I wrote from this case lists four kinds of key change that leave dead files behind: a Next upgrade, a token swap, new query text, and URL values invented by bots. The last one does not even wait for a release, because the URL is part of what goes into the key. In production the folder reached 5.1 GB.

A cleanup that never ran

Because the entries are flat files, pruning them does not require understanding the cache's contents at all. The advice I drafted on 1 October, with <app> standing for the application folder:

# once, on both apps, no restart
find <app>/.next/cache/fetch-cache -type f -mtime +3 -delete
 
# then daily from cron, with a 7-day threshold
find <app>/.next/cache/fetch-cache -type f -mtime +7 -delete

Pruning by file age is enough here because a live entry is rewritten on refresh, which keeps its modification time recent, while anything that does get removed is fetched again once. No restart is needed. I left the 593M image cache alone.

What I expected afterwards, and this is a prediction rather than a measurement: production pages and sitemaps unchanged, the hourly market API usage chart without a spike because the removed entries would mostly be dead v3 keys and CMS keys from before the token swap, and disk usage dropping from 87%.

I sent the draft to the client's DevOps engineer the same day. They agreed to run the cleanup on both apps and add a daily -mtime +7 cron, but at that point nothing confirmed it had been done.

The news that came later went the other way. From a DM with the client's DevOps engineer that I noted on 5 October, the cleanup was never run and no cron was set. Instead, according to the same report, the server's storage was upgraded to 50 GB on Friday 2 October. I have no disk numbers from after that upgrade, so there is nothing I can state about the percentage now. What is certain is that the cause did not leave with it.

The cache nobody deleted kept the pages up

On 5 October it turned out that the subscription for the market data API this site uses had been dead since the previous afternoon. Charts and listings kept cycling because Next's Data Cache handed back the stale 200, and the app's fetcher kept rewriting its own cache plus a fresh 24-hour backup copy from that answer. How the Data Cache treats stale answers is a separate story; what matters here is where those answers were stored.

That midday one thing was still unknown: whether a production deploy keeps .next/cache/fetch-cache. So my advice was not to retry the build while the market API was down. The deploy that day is what answered it. Afterwards the frozen data was undisturbed: listings and charts showed the same rows and points, and the last-good copies kept being rewritten by the new process. On this server, the on-disk fetch cache demonstrably survives a production deploy.

The "confirm the cleanup and cron" item on my list was then closed. Nothing would delete the cache that was keeping the market pages populated during the outage, and its unpruned growth went down as a topic for later. A day later, a note from a deliberately skeptical review added a warning: a fetch-cache cleanup by the client's DevOps engineer during that outage would turn the listings, the charts and the price ticker fallback empty. That is a warning, not something anyone tried. The core of it is easy to follow, though: find only looks at file age and has no idea which entry is currently the only copy of the data.

The image cache has a limit, and the default is half the free disk

The 593M images folder is treated differently by Next, because it gets a limit. My note from 1 October said that Next 16.3.6 caps the on-disk image cache at half the free disk at server start when images.maximumDiskCacheSize is not set, and I wrote down a fixed cap as an optional code change for later. That same day the optional cap went onto the list I parked with the note "dicatat aja dulu", which roughly means "just write it down for now".

The code sits in two files:

// next 16.3.6, dist/server/image-optimizer.js lines 700-704, trimmed
this.cacheDir = (0, _path.join)(/* turbopackIgnore: true */ distDir, 'cache', 'images');
// ...
// Eagerly start LRU initialization for filesystem cache
if (!cacheHandler && nextConfig.images.maximumDiskCacheSize !== 0 && nextConfig.experimental.isrFlushToDisk) {
    // ...
}
 
// next 16.3.6, dist/server/lib/disk-lru-cache.external.js lines 39-41
// Since config was not provided, default to 50% of available disk space
const { bavail, bsize } = await _fs.promises.statfs(cacheDir);
maxSize = Math.floor(bavail * bsize / 2);

In the config comment I wrote later, the wording became "at first use", and that wording fits the code better. In 16.3.6, ImageOptimizerCache is created inside handleNextImageRequest, the image request handler, and its constructor eagerly starts the disk LRU initialization.

The difference from the fetch cache is plain. The image cache has an LRU and a limit, while the fetch cache never deletes anything. The default limit becomes a problem when one server runs more than one app, though. Staging and production here run on one small server, each with its own cache. If each app may take half the free disk, half plus half means that together they could claim the whole disk.

The parked cap finally landed as a commit on 9 October: the image optimizer's disk cache is capped at 2 GiB per app. Past that, the least recently used variants are deleted and made again on their next request, and a config test pins the value. A simplified version, with the constant renamed:

// next.config.ts, simplified, constant renamed
const IMAGE_CACHE_LIMIT_BYTES = 2 * 1024 ** 3; // 2,147,483,648 bytes per app
 
const nextConfig = {
  images: {
    maximumDiskCacheSize: IMAGE_CACHE_LIMIT_BYTES,
  },
};

With two apps, the combined image cache limit on that server is 2 times 2 GiB, or 4 GiB, whatever the size of the disk. The commit went out in a merge to main that went live in production the same day, and every staging and production check for that release passed. Verification of this particular cap is still only that config test. There are no numbers yet for the image cache size or the disk usage afterwards.

What is in place and what is not

Pruning for .next/cache/fetch-cache was never installed. Its replacement, a disk enlarged to 50 GB, is a report from the client side whose numbers I have not seen. The cause is still in place: Next 16.3.6 writes fetch entries and never deletes them, so as long as nothing prunes that folder, the next Next upgrade, the next token, new query text, and URL values invented by bots will keep adding dead files. The 2 GiB per app image cache cap is in the code, with a config test as its only verification.

If you run Next yourself with the default file-system cache, I think this case leaves two things worth holding on to. fetch-cache has nothing guarding its size, so the pruning is yours to provide, and pruning by file age makes sense because live entries keep being rewritten. But the same folder can become the only copy of the data when an upstream API dies, so an automatic pruning schedule needs to be weighed against that possibility. The find advice above has never run on this server, so its effect on the disk and on the pages has not been measured.