Why five engines give five different answers about you

The short answer: Each engine rewrites your question into its own search queries, runs them against its own index, and then picks a handful of passages from whatever came back. The vendors that document those steps describe them differently, and none of them publishes enough for you to line one pipeline up against another. So the engines are not grading the same exam, and in the one published benchmark we could open, the citation sets they ended up with barely touched.

That last part has been measured, once, in one narrow vertical on one day, and the number is smaller than almost anyone guesses.

In one measured benchmark, divergence was close to total

An audit published in September 2026 ran a fixed set of 15 English-language commercial prompts, all of them about AI search visibility software, in a United States locale, across ChatGPT, Microsoft Copilot, Google and Perplexity on one day, 6 June 2026. It collected 589 citation observations, 528 unique URLs and 356 domains. Mean pairwise overlap of cited URLs between two engines was a Jaccard similarity of 0.0079. The median was zero. In 84.9% of engine pairs, not one cited URL was shared. Of the URLs seen that day, 96.4% appeared on exactly one engine (arXiv:2609.22655, 19 September 2026).

Read that as a practical number rather than a law. Within that benchmark, a single engine showed between 11.4% and 42.6% of the URLs the four engines surfaced between them. Whatever a reader of that study learned from one assistant, they learned about one ninth to two fifths of the evidence those four engines displayed. The authors say plainly that the category is unusually self-referential and that the rates should not be carried over to travel, health, shopping, local search or news without someone repeating the work there.

The same work separates divergence from drift, which matters because people conflate them. Comparing one engine with itself across 5 and 6 June, over 41 prompt pairs on three of the four engines, gave a mean URL set turnover of 67.0%. Even with that much churn, the same engine's next day citation set was about 42 times more similar to itself than two engines were to each other on the same day. The authors warn that the prompt count there is small and that 67.0% should not be read as a general daily volatility figure. The ratio is the part worth carrying: in that benchmark, engine identity separated citation sets more than the passing of a day did.

Why the gap opens: your question is not the query

The sentence you typed is often not the query that gets run. OpenAI's developer documentation for its web search tool describes the model issuing its own search queries and returns those queries back in the response object, and describes a reasoning mode in which the model searches, reads and decides whether to search again. Google's documentation on AI features describes a query fan out technique, issuing multiple related searches across subtopics and data sources, and states plainly that AI Mode and AI Overviews may use different models and techniques, so the links they show will vary. Those are two vendors describing their own systems. The others have not published enough for us to say the same about them.

Two engines, two sets of invented queries, before a single page is fetched. A brand that reads as a fit for "best lightweight CRM for a two person agency" may be invisible for "affordable CRM small business pricing", and nobody ever typed either string. This is the part people miss when they ask which engine is right. Both are answering faithfully. They asked different things.

Rewriting also absorbs context that has nothing to do with your page. OpenAI's web search documentation exposes an approximate user location, country, city, region and timezone, as a parameter that refines which results come back, so the same words can be answered against a different slice of the web. Memory, custom instructions and the earlier turns of a conversation plausibly sit in the same slot, though we cannot observe that from outside. Two people in two cities, asking the same words, may not have the same query run for them. How each AI engine picks the sources it cites goes through the retrieval side of this engine by engine, and how Google AI Overviews chooses its sources covers the fan out case in detail.

Then the dice get rolled

Suppose the rewriting were identical. You would still not get a stable answer. Researchers at the University of St. Gallen ran eight prompts per campaign, up to ten times each, within minutes of each other, across ChatGPT, Gemini, Google AI Mode and Perplexity. The campaigns were four Swiss verticals, telecommunications, real estate sales, sporting goods and consumer electronics, with the prompts issued in German from Swiss locales, and the daily dataset covering 24 January to 20 March 2026. After filtering to runs that cited something, 3,409 pairwise comparisons gave a mean source overlap between 0.32 and 0.43, the same range they measured across days weeks apart (arXiv:2604.07585, 8 April 2026).

So in those campaigns, within minutes and under the same conditions, roughly two thirds of the cited sources changed. Brand mentions held up better than sources there, with pairwise overlap of 0.33 to 0.48 depending on the category, which is the most useful asymmetry in the paper. In that data the brands in an answer moved more slowly than the URLs beneath them. Whether the same asymmetry holds in English, in other markets or in other verticals is open, and the authors say so themselves.

Concentration sat on top of the churn. Citation inequality across those campaigns and that window averaged a Gini coefficient of 0.715, with Google AI Mode the most concentrated at 0.782 and Perplexity the least at 0.671. A small number of domains took most of the citations while the long tail rotated. If your domain is in that tail, the experience is indistinguishable from being invisible for most of the runs you look at.

The failure mode this creates

Here is the expensive version of the mistake. A company checks one assistant, finds itself missing, and commissions a quarter of work aimed at whatever that engine seemed to reward. Three months later the number has not moved, because the fix targeted an exposure path that only one vendor used, and because the baseline was a single run that could have come back differently ten minutes later.

A second version: the company buys a single visibility score and treats it as a property of the brand. The September audit is blunt about what such a number can and cannot mean. A score computed without observing a live engine can estimate page quality or the fit between a page and a request, but end to end visibility also depends on engine specific exposure and selection, which that kind of score does not see. Averaging across engines hides the thing that is actionable, which is where in the pipeline you are being dropped.

The useful questions are narrower. Were you fetched at all. Were you retrieved for the rewritten query rather than the typed one. Were you cited once retrieved. Those fail for different reasons, and the repair for each is different. In our experience access problems are the quickest of the three to clear, while identity and corroboration are the slowest, which is the whole argument in how long AEO takes to show up in answers.

A check you can run this week

This takes a spreadsheet and about two hours.

1. Pick six prompts a real buyer would type. Not your brand name. The problem they have.
2. Run each prompt on five engines, five times each, same day, fresh session, logged out

where you can. That is 150 runs. Record every cited URL.

3. For each prompt, build the union of all URLs across all five engines. Then compute each

engine's share of that union. If one engine shows you 15% of the evidence, you now know what a single engine report is worth.

4. Count how many URLs appear on two or more engines. In the 6 June 2026 benchmark above,

in its one vertical, that was 3.6% of URLs. Your own number is what matters; the point of computing it is to find out whether engines behave as replicas in your category.

5. Note separately whether your brand was named and whether the claim beside it was true. In

the St. Gallen campaigns mentions were the steadier layer and sources the more diagnostic one. Check whether that holds in your data before you rely on it.

Where the answer involves products, prices or availability, the retrieval path is different again, and the markup carries more of the load. See product schema for AI shopping answers before you run that version of the test. If you would rather start with the page side, the self-check lists ten things to test with no tooling at all.

Honest limits

This post says five engines and cites two studies that cover four each. The cross-engine numbers come from a single day, 6 June 2026, 15 English prompts in one self-referential vertical, run in a United States locale; the repeat-run and concentration numbers come from four Swiss verticals with German prompts between 24 January and 20 March 2026. Neither study licenses a claim about your category, your language or today, and both sets of authors say so in their own limitations sections.

Both studies observe displayed citations only. Nobody outside the vendors sees the rewritten queries, the candidate pages or the reranking, so low overlap tells you the outputs diverge, not which stage caused it. Our own audits run a prompt set of about 45 prompts across five engines with retrieval on and retrieval off; that is a sample of one buyer's question space on the days we run it, not a census of what the engines hold, and we do not publish figures from it as though it were one. We also cannot tell you which engine your buyers use, which is an argument for breadth in measurement rather than for more runs on the engine you already prefer.

FAQ

Which engine should I optimise for? None of them individually. Optimise the pipeline they share: reachable pages, parseable markup, an identity that resolves cleanly, and third parties who repeat your claim. Engine specific tuning is worth doing last, after the shared layer is clean.

Is the variation random or is something changing? Both, and they are separable. In the St. Gallen campaigns, same day repeat runs showed source overlap in the same range as runs weeks apart, which suggests that in that data most of the instability came from the retrieval and generation process rather than from the web changing underneath the engines.

How many runs make a baseline? More than one. The published work argues for repeated measurement without naming a number. Five runs per prompt per engine is the floor we use for a working estimate, which is a working convention rather than a result. Treat a single run as an anecdote.

Should I report mentions or citations? Report mentions, diagnose with citations. In the one study that measured both, brand mentions were more stable than cited sources across repeated runs, so a citation vanishing overnight is more likely to be turnover than a loss of visibility. That was German-language Swiss campaigns in early 2026, so treat it as a prior to test, not a settled rule.

Sources

All four were opened and re-read on 12 October 2026, and each still carried the sentence cited against it.

• "Scoring With the Engine: Retrieval Exposure, Cross-Engine Divergence, and the Limits of Engine-Agnostic GEO Scores", arXiv:2609.22655, submitted 19 September 2026. Data collected 6 June 2026; 15 English commercial prompts in the AI visibility software category, United States locale, four engines: arxiv.org/abs/2609.22655
• "Don't Measure Once: Measuring Visibility in AI Search (GEO)", University of St. Gallen, arXiv:2604.07585, submitted 8 April 2026. Four Swiss campaign verticals, prompts issued in German, daily collection 24 January to 20 March 2026, four engines: arxiv.org/abs/2604.07585
• Google Search Central, AI features and your website. Source for the query fan out description and for AI Mode and AI Overviews showing different links: developers.google.com/search/docs/appearance/ai-features
• OpenAI developer documentation, Web search. Source for the model issuing its own search queries and for the approximate user location parameter: platform.openai.com/docs/guides/tools-web-search

The ChatGPT help centre article on searching the web was cited in an earlier version of this post. It returns 403 to our requests and to the public archive, so we cannot confirm its current wording and have replaced it with the developer documentation above.