The upstream price API behind a client's crypto data site answered every call with 401, and for the first two days I assumed the consequence was simple: prices on the site had been frozen since 4 October 2026, around 10:05 UTC. That assumption was wrong, and I found out by accident.
On 6 October at 14:35 WIB (UTC+7), I was checking a fix for the price age label in a browser. The top coin's page, with its price rendered on the server, showed $83,993 under the label "Updated 2m ago". The frozen 4 October figure for that coin was $85,231, and the real price at that moment was $85,381. What I saw was neither of them, and the label claimed the number had been updated two minutes earlier.
That 83,993 turned out to be the price from 29 September. For comparison, the coin's public price on 29 September at 07:00 UTC was 83,976. So in the middle of the outage, the price on that page was not frozen. It had gone back a week and then received a fresh timestamp. In production, the page's HTML and JSON-LD already carried 83,993, even though visitors on the top coins that afternoon still saw the number from the price feed, 85,231 with "Updated 45h ago".
A data cache that never sees the 401
The details call for each coin brings in the description, the exchange table and the market data at once. Before the fix, its options carried next: { revalidate: 1800 }. Roughly, simplified and with the names neutralised:
// simplified, names neutralised: the details call before the fix
const res = await upstreamFetch(detailsUrl, {
headers,
next: { revalidate: 1800 },
});
const coin = await res.json();
// the details half (description, exchange table) AND the market snapshot,
// both stamped with the time of the write
const now = new Date().toISOString();
await writeSnapshot(coin, { stampedAt: now, fetchedAt: now });A value of 1800 means a data cache entry counts as stale after 30 minutes. In Next 16, a stale entry does not make the call wait for the upstream. Next answers with the entry as it is and revalidates in the background, and the result of that revalidation is only used by the next call. The details call for a given coin happens once a week, so every time it arrives, what it gets is always the answer of the previous call, a week old. That answer was then written to Redis with stampedAt and fetchedAt set to the time of the write. Not just the description and the exchange table went in, but the whole market snapshot, price included. So the stampedAt on a coin snapshot was the time of the write, not the time of the data.
When the upstream refuses, the background revalidation gets a 401, but Next keeps handing back the last 200 it stored. The code never saw that 401, only a cached 200. This mechanism was later confirmed during the review of the root fix by running Next's own patched fetch. The cache on this call was itself a leftover of the June design, when details were fetched per page view and the Next cache acted as a brake. Since the August rework put details in Redis for a week behind an hourly call budget, that cache had stopped saving anything.
Why the defect stayed hidden before the outage
In normal times, the 30-minute price pass, an uncached bulk call, overwrites the market half within half an hour. Only the description and the exchange table stay one cycle behind, and nobody notices. During an outage, nothing overwrites it. Once the details half of a coin expires after its 7-day TTL, the backfill "refreshes" it from the cache, and the page's price goes back to the price from the previous details fetch, under a fresh label.
That is the sequence the top coins went through. Their details had all been fetched together on 29 September at 07:28 UTC, so they expired together seven days later, on 6 October at 14:28 WIB. Their descriptions did vanish at 14:28 and came back at 14:33, in last week's version. The top coin's snapshot carried a price of 83,993 stamped 2026-10-06T07:33:06Z, the fetchedAt of its details half fell on the same second, and its TTL was back to a full 7 days. That stamp is 14:33 WIB, two minutes before I opened the page, which is where "Updated 2m ago" came from.
The rewrites spread with every pass
At 14:40 WIB, 34 of 12,944 coin snapshots carried a stamp from inside the outage. Nineteen of them came from a single pass at 07:33 UTC, five minutes after the top coins' details had expired, and 2 more from the 07:02 UTC pass. After that, every 30-minute pass added more:
6 Oct, WIB coin snapshots stamped inside the outage
14:40 34 of 12,944
15:30 54 the 08:03 UTC pass added 20
16:08 81 about 20 to 27 more with every pass
17:28 105 all carry the details-call mark, none without
17:36 112 last pass of the old build (10:36 UTC) rewrote 7 more
17:43:06 root fix live in production
18:06:40 first pass on the new build starts: writes nothing, still 112
18:37:00 second pass starts: writes nothing, still 112The mark in that table is a snapshot stamp exactly equal to the fetchedAt of its details half. A read-only scan showed all 105 snapshots at 17:28 WIB carrying it, with zero unmarked, and the final 112 were all marked as well, so the only rewriting path was the details call. The price gaps were not small: one rewritten coin was 12% off, another 16%. Because the rewrites kept spreading with every pass, the root fix went out alone and first, without waiting for the label fix.
The root fix: uncached details, and a 401 writes nothing
The root fix makes three calls, the coin details, the NFT collection details and the NFT tickers, ask for an uncached response, and a refused call now writes no coin or collection data. The same commit added a test that simulates a 401 answer and requires the coin and NFT details fetchers to write nothing. Simplified again, with neutral names:
// simplified, names neutralised: the details call after the fix
const res = await upstreamFetch(detailsUrl, { headers, next: { revalidate: 0 } });
// the shape of the test that came with it
it("writes nothing when the upstream refuses the call", async () => {
// the fake fetch answers 401: new Response(body, { status: 401 })
// ...the coin and NFT details fetchers run...
expect(writes).toEqual([]);
});According to the commit message, the hourly limits stay at 60 coin details and 20 NFT details, and a stale cache hit already sent one call in the background, so a refresh costs the same number of calls it always did.
The fix also closes an accidental bypass. The rewrites let restamped snapshots slip past the 7-day snapshot limit, so without the fix, those coins would have stayed up with wrong prices. With the fix, a coin page answers not-found once that limit passes while the upstream is still down. What the fix does not do is restore the 112 snapshots that had already been restamped. It only stops new ones.
The reasoning behind dropping this cache had in fact been used before, just not everywhere. The priority tier written on 1 October already asked for uncached details calls for its own coins for exactly this reason, but the same reasoning was not applied to every other coin.
The age label that trusted the stamp
The label fix I was checking that afternoon had a problem of its own. Its first version made the price hero on the coin page trust detail.stampedAt, and a review of that version found one blocker, confirmed in a browser run: with the feed gone, the build printed "$83,993 Updated 41m ago" on the top coin's page, a week-old price under a fresh label, because 34 snapshots carried a wrong stamp.
The version that shipped tells the two writers of a market snapshot apart. The price pass stamps it with the time it asked for the prices. The details call writes it together with the details half under one clock reading, so equal stamps are the mark of that writer, and a marked snapshot from before the fix gets no stamp at all until a price pass rewrites it. An end-to-end label test on the top coin's page showed "$85,231 Updated 2d ago" when served normally, and "$83,993 Delayed" with the feed gone. The price is still the old one, but the page stopped claiming it was fresh. This change went live in production at 19:14:55 WIB, about an hour and a half after the root fix.
The twin defect in the NFT pass and the listing calls
The root fix only touched those three calls. The same defect remained in the NFT market pass and in the listing calls, including the top-100 list that was rewritten every 15 minutes from the stale cache, which is also what kept the ticker fallback alive.
For NFTs, a read-only measurement showed 1,181 of 1,320 snapshots being restamped every 30 minutes over the 4 October floors, because the NFT pass wrote rows it got from a cache under a new stamp. Reading the code led back to the same place: a stale data cache entry is served as a 200 forever while the upstream refuses, and it survives releases. After the NFT fix went live on 7 October, two passes of the new build in production got a 401, made one list call and wrote nothing, and the stamps stopped at 15:12:15 WIB.
The listing calls followed on 12 October. That commit message states that every list, card and chart fetcher called the upstream through Next's data cache with a revalidate equal to its own lifetime, then stored the answer in Redis as new data, so the lists of 4 October were served as current days later. Here revalidate: 0 was deliberately not used, because Next counts revalidate: 0 and no-store as a reason to render a page dynamically. A call made while a page is regenerating in the background, such as a market page or the homepage, would end that regeneration with "Page changed from static to dynamic at runtime". So the options go out with neither next nor cache:
// simplified, names neutralised: fetch options for the list, card and chart fetchers
function withoutDataCache(init: RequestInit): RequestInit {
const { next, cache, ...rest } = init; // these two fields are dropped on purpose
return rest; // headers, signal, method and the rest pass through as given
}According to the note in that code, a fetch with neither field is one Next 16 neither stores nor answers from its data cache once the build is over, the behavior patch-fetch.js calls "auto no cache". For this change I have no dedicated verification like the two empty passes after the root fix, so I do not count it as proven in production yet.
Timestamps belong to the data, not to the write
The principle is short: a cache in front of a call whose answer is written to a store under "now" turns an old answer into a new fact. I now keep two rules. Any upstream call whose result is persisted with a timestamp must be uncached (revalidate: 0) or must carry the upstream's own time, and if that call can happen while a page regenerates, the uncached form is a fetch without next and cache. And when a number on the site changes during an outage, do not assume the outage froze it. Read the snapshot's price next to its stamp and compare it with the last known good value.
This is also a different kind of failure from a fallback that invents a number. Nothing here was invented, because 83,993 really was that coin's price on 29 September. Only its age was wrong, and the age came from a stamp filled in at write time. It is the opposite of force-dynamic quietly turning fetches into no-store, too. There, the problem is a fetch that never hits the cache, while here it is a fetch that hits the cache even though its result is about to be stamped as new.