There is a specific kind of misery in debugging: you fix the obvious cause, you deploy, and the number does not move.
On a client site running Next.js and Sanity, I had just patched a genuine cache misconfiguration, a route forced dynamic when it did not need to be, so every request punched straight through to the API. That fix was correct and still necessary. The quota kept burning at roughly 150,000 requests per day anyway. This is the story of what happened after the fix that should have been enough turned out not to be.
The theory that was too comfortable
In cases like this, everyone's favourite suspect is bots. The site had thousands of detail pages, a large sitemap, and Googlebot is diligent. The narrative writes itself: crawlers sweep thousands of slugs, each slug triggers several fetches, quota gone. I believed this hard enough that I had already drafted the follow up work: throttle crawl rate, delay indexing on some routes, ship a stricter robots.txt.
Luckily I postponed that work by one day, because one question kept nagging. If bots were really the cause, the requests should be arriving from many IP addresses and many user agents. I had nothing to confirm or deny that. All I had was the aggregate number on the dashboard: large, and completely uninformative.
Turn on log delivery before you theorise
Sanity has a feature nobody talks about: request log delivery. In the project dashboard, under "Configure log delivery", you enable it and point it at a destination bucket. Once it is on, every API and CDN request is recorded as one NDJSON line with its attributes attached, including the identifier of the GROQ query behind it.
I turned it on, waited for the first drop to land the next day, then pulled every file that was available. At that point the job stopped being guesswork and became arithmetic. The whole aggregator is this short:
import fs from "node:fs";
const rows = fs.readFileSync("sanity-logs.ndjson", "utf8").split("\n");
const byQuery = new Map();
let total = 0;
for (const line of rows) {
if (!line.trim()) continue;
const rec = JSON.parse(line);
const id = rec.attributes?.["sanity.groqQueryIdentifier"] ?? "unknown";
byQuery.set(id, (byQuery.get(id) ?? 0) + 1);
total++;
}
[...byQuery.entries()]
.sort((a, b) => b[1] - a[1])
.slice(0, 10)
.forEach(([id, n]) =>
console.log(n.toString().padStart(8), ((n / total) * 100).toFixed(1) + "%", id)
);The part that makes this work is sanity.groqQueryIdentifier. It gives every query a stable identity, so thousands of requests that differ only by parameter collapse into a single summary row. Without it you are staring at millions of near identical URLs and concluding nothing.
The numbers that killed my theory
1.06 million requests over eight days. The aggregation output made me sit still for a moment.
- The top five queries accounted for 85% of the entire burn.
- Four of those five were layout level singletons:
apiSettings,navConfig,globalConfig, and one more list singleton rendered in the footer. Each sat between 16 and 21%. - 93% of requests arrived with a Node user agent from a single Railway IP. That was our own server, not visitors, not crawlers.
- The crawl pattern I had been blaming from day one: about 14%. That does not contradict the line above: crawlers never call Sanity directly, they only trigger renders, and our own server is what fires the queries.
My bot theory was not wrong, it was just small. Had I trusted intuition, I would have spent a full day tuning crawl budget to shave off 14% while leaving the actual 85% completely untouched.
The root cause: innocent looking singletons
Those four documents hold configuration. They barely change, maybe once a week. But they are fetched by layout, by the header, and by the footer, which means every render of every page pulls them again.
And here is where my real mistake sat: all of those getters were riding the default five minute baseline TTL. Five minutes sounds safe right up until you multiply it by render count. Thousands of pages re-render throughout the day, each render fires four config fetches, and every hour reopens that cache window twelve times. The number grew not because traffic was large, but because of multiplication.
The fix: a long per singleton TTL, paired with tags
The answer was not raising the global baseline. That would have gone on to make genuinely fresh content stale. Instead I set an explicit TTL per getter and paired it with tag based invalidation so editors still see their changes immediately.
export const getNavConfig = () =>
sanityFetch<NavConfig>(NAV_CONFIG_QUERY, {}, {
revalidate: 3600,
tags: ["navConfig"],
});Config singletons got revalidate: 3600. One getter feeding a widget with faster moving data got 1800. The rest I mapped one by one into an 18 getter matrix, each with a written reason for the number I chose.
The tag half is what makes a one hour TTL unscary. A CMS webhook invalidates by the document type that was just edited:
export async function POST(req: Request) {
const body = await req.json();
revalidateTag(body._type);
return Response.json({ revalidated: true });
}Long TTLs handle the normal case, tags handle the "an editor just changed the footer phone number and wants to see it now" case.
The result: from roughly 150,000 requests a day down to 13,000 to 15,000. A tenfold reduction, without touching a single line of crawler configuration.
What I took away
- A correct first fix is not automatically a sufficient fix. Recheck the number after deploying, not your confidence.
- Measure before you optimise. One day of waiting for logs is far cheaper than one week of guessing.
- Aggregate by query identity, not by URL. Thousands of similar URLs hide the five queries actually responsible.
- If most of your requests carry your own server's user agent, the problem is your fetch pattern, not your visitors.
- Data that rarely changes but is read on every render is the strongest candidate for the longest TTL in your system. Pair it with tag invalidation so long never means stale.