Blog · Diagnosis

Diagnosing a drop in AI citations

Published 23 April 2026 · Updated 12 October 2026 · 9 min read

Diagnosing a drop in AI citations
Photograph: Hush Naidoo Jade Photography / unsplash

The short answer: Work through four causes in order. First, measurement: re-run the same prompts enough times to tell a real fall from normal variance, and carry a control. Second, access: check whether a crawler or a user-triggered fetcher is now being refused. Third, the source: the page that was cited may not have been yours. Fourth, the engine: it may have changed which kinds of source it prefers.

The post-mortems we are called into usually go wrong in the first ten minutes, because somebody opens a dashboard, sees a line fall, and starts fixing the site. Often the site turns out to be fine. We have not counted how often against any defined population, so treat that as the pattern we keep meeting, not a rate.

Prove the drop before you explain it

Two identical prompts, run an hour apart, can return different sources. A citation count built from one run per prompt is a sample of one, and a sample of one cannot fall. It can only differ.

So the first step costs little and saves weeks. Re-run the same prompt set, at least five times per prompt, on the same engines, and compare the share of runs where you were named rather than the raw count. Five is our working floor rather than a published threshold: it is the point at which, in our own audit runs, a one-off answer stops looking like a trend. We have not found a published run-to-run variance figure in the documentation of the engines we test, so pick a floor and keep it constant. Then carry three controls alongside it.

The first control is the denominator: how many citations did that engine return in total across your prompt set. If it returned fewer of everything, your fall is arithmetic, not a verdict about you. The second control is a domain you do not own. Pick a competitor page or a reference site that appears in the same answers, and track it in the same runs. When your domain falls and theirs holds, something about you changed. When both fall together, the engine changed.

A third control is cheaper still. Run the prompt set on a second engine. In the audits we run, a fall confined to one engine has rarely turned out to be a site problem, because your pages are the same pages for all of them. It is a pointer, not a proof.

Then check access, and check it by agent, not by site

This is where the recoverable losses we find usually sit, and the mistake repeats: a site is tested in a browser, it loads, and access gets ruled out. We cannot say what share of all drops are access problems, only that this is the one we most often fix.

Engines do not arrive as one thing. OpenAI documents four separate agents with separate jobs, three of which are in play here. OAI-SearchBot surfaces sites in ChatGPT's search features, GPTBot collects training data, and ChatGPT-User fetches a page because a person in a conversation asked for it. The three are governed differently. A site that disallows OAI-SearchBot will not be shown in ChatGPT search answers, though it can still appear as a navigational link, which is exactly the half-dead state that confuses a reporting dashboard. OpenAI also says it can take around 24 hours from a robots.txt change for its systems to adjust, so the drop you noticed on Thursday may belong to Tuesday's deploy.

Perplexity documents the same split. Its crawler documentation says PerplexityBot is the agent that surfaces and links sites in Perplexity results, and recommends allowing it in robots.txt, while Perplexity-User serves a user's request and, because a user asked for the fetch, generally ignores robots.txt rules. Google documents the same behaviour for its user-triggered fetchers: because the fetch was requested by a user, they generally ignore robots.txt rules. Those three sets of docs are what this section rests on. Other engines may or may not split their agents the same way, and we have not verified that they do.

That asymmetry gives you a free diagnostic in your own logs. Split bot hits into scheduled crawler traffic and user-triggered fetches, then look at the shape of each.

  • Crawler hits steady, user-triggered fetches gone: you are still being crawled, but you

    • have stopped being chosen as a source. The problem is relevance or corroboration.

  • Crawler hits gone, user-triggered fetches continuing: a robots rule is doing it.
  • Both gone, with 403s in the log: a firewall, bot-management rule or CDN change is

    • refusing the IP ranges. Verify by the published IP range files rather than the user agent string, which anyone can forge.

  • Both present and returning 200, with citations still falling: access is not your cause.

    • Move on.

One more access trap is specific to Google. Google's crawler documentation describes Google-Extended as the token controlling whether crawled content is used for training and grounding in Gemini Apps and the Vertex AI API. It is not the control for AI Overviews or AI Mode. Google's AI features documentation says the way to limit what Search shows from your pages is the ordinary preview controls, nosnippet, data-nosnippet, max-snippet and noindex. We have seen this once in our own audit work, a CMS upgrade shipping a global max-snippet:0 with nobody noticing, which stripped the quotable passage while leaving the page indexed and perfectly healthy to a human eye. One case is an illustration, not a frequency. If your brand vanished from answers while your rankings sat still, start there. The same logic sits behind why AI never mentions your brand.

Ask which document was actually cited

A citation line lists URLs. In the runs we review, those URLs are often not the client's own pages, and the number being watched may be counting them on the client's behalf.

Pull the actual cited URLs from the runs you have kept, from before the drop and after. Compare the two lists as sets, not as totals. Three patterns recur in the drops we have diagnosed.

The carrier disappeared. You were being named because a roundup, a forum thread, a directory page or a transcript said so. The thread got locked, the roundup got rewritten, the video got re-uploaded with a new ID and lost its transcript. Nothing about your site changed. Video and social carriers are the most fragile of these, which is why YouTube, transcripts and video citations and Grok and social sourcing: when posts become citations are worth reading before you assume the loss is technical.

The URL moved. A redirect that works for a reader does not always carry a stored citation with it, and we cannot see which stored reference an engine kept. If the drop starts within a fortnight of a launch, replatform or URL change, treat migrating a site without losing citations as the first hypothesis and everything else as second.

The passage moved. Same URL, same traffic, but the sentence that answered the question got pushed below a new hero block, a cookie gate or a lazy-loaded tab. The page is still fetched. The part that got quoted is no longer early, no longer plain, or no longer in the HTML at all.

Date the change before you attribute it

Line up three timelines on one axis: your deploys, your log anomalies, and the day each prompt's citation share moved. Then look for the earliest of the three, not the loudest.

Attribution survives or dies on that ordering. A deploy on the 3rd, a 403 spike on the 4th and a citation fall on the 6th is a story. A citation fall on the 2nd with the deploy on the 5th means you are about to fix something that was never broken.

When no deploy and no log anomaly lines up, and the fall shows on one engine while the others hold, an engine-side change becomes the best remaining explanation. We cannot confirm one: we have found no public change log from these engines describing how they weigh source types, and we have no way to see inside the ranking from outside, so this is the hypothesis left standing rather than a finding. What we can say is what we observed, that the citing URLs for a prompt set changed character between runs while the pages did not. There is no fix for that in your codebase. The response is to widen the set of places that corroborate you, so one source losing favour does not remove you from the answer.

Honest limits

This post does not establish how often each of the four causes is to blame. The patterns described here come from the audits we run, which are a self-selected sample of sites that already suspected a problem, and we have not counted their frequency against any defined population. The agent behaviour in the access section is sourced to the published documentation of OpenAI, Perplexity and Google only; we have not tested whether other engines split crawling and user-triggered fetching the same way, and this post should not be read as saying they do. Where it says a request is absent or a citation was lost, that means we could not observe it from outside, not that nothing happened inside the engine. We deliberately carry no recovery figures here, because we have no measured before-and-after sample whose method, size and dates we could publish alongside the number, and a figure without those is not worth printing. If you want the shorter version of this process, the self-check is ten tests you can run this afternoon with no tooling.

FAQ

How many runs do I need before a drop is real? Five runs per prompt is our working floor and ten is better. We have not found a published run-to-run variance figure for the engines we test, so this is a habit rather than a tested threshold. Compare the share of runs where you were named, not a single count, and always against a control domain tracked in the same runs.

Can a site fall out of one engine and stay in the others? Yes, and it is the case we see most often. OpenAI, Perplexity and Google each document their own agents, and each engine makes its own sourcing decisions. A single-engine fall points at that engine's access rules or its sourcing, not at your content.

Does blocking an AI crawler remove me from everything? No, and the partial outcome is what makes it hard to spot. Disallowing a search agent can remove you from generated answers while leaving navigational links, so you look present and cite nowhere.

How long should I wait after a fix before re-measuring? Allow for propagation and a crawl cycle, so days rather than hours. OpenAI's own guidance notes roughly a day for a robots.txt change to register, and re-fetching plus re-ranking takes longer than that.

What if nothing on my side changed at all? Then record it, keep measuring, and spend the effort on corroboration instead. A brand cited by several independent source types is harder to knock out than one carried by a single thread.

Sources

All five links above were opened and checked against the sentences citing them on 12 October 2026.

Related reading

Why AI never mentions your brandMigrating a site without losing citationsGrok and social sourcing: when posts become citationsYouTube, transcripts and video citationsContent locked in images and PDFsInformation architecture for answers, not sessions