Blog · Foundations

Retrieval vs training data: which one is answering about your brand

Published 8 October 2026 · Updated 12 October 2026 · 9 min read

Retrieval vs training data: which one is answering about your brand
Photograph: Luke Caunt / unsplash

The short answer: Both can, and they fail differently. Training data is a frozen impression of your company from some months ago, stated confidently and with no source attached. Retrieval is a live or cached fetch of a real page, which can be checked. If the sentence about you is wrong and carries no citation, memory is the likelier source. If it is wrong and carries a citation, the likelier source is something it fetched. Those two problems point at opposite fixes.

Most brand-visibility work treats an engine as one thing. It is not. Two different systems produce the sentence that names you, and a team that cannot tell which one spoke will spend a quarter fixing the wrong layer.

The two paths, and the gates in front of each

A model's parametric memory is everything it absorbed during training, compressed into weights. It has a hard edge. OpenAI's published model pages put the knowledge cutoff for GPT-6 Astra at 30 April 2026 and for GPT-6 Luna at 18 May 2026 (developers.openai.com/api/docs/models, checked 8 October 2026). Read those two dates as a calendar fact about your own company: for those two models, anything you shipped, renamed, repriced or retired since then exists only if something fetches it. We checked OpenAI's page, not the whole field. Other engines publish their own cutoffs on their own pages, so treat the four-to-five-month gap as OpenAI's number on the day we looked rather than a constant across the category.

Retrieval is the other path. The model issues a query, something returns documents, and passages from those documents go into the context window before the answer is written. The retrieval can hit a live URL or a stored copy. OpenAI's web search tool documents an external_web_access flag, and setting it to false restricts the tool to cached and indexed results rather than live fetches (developers.openai.com/api/docs/guides/tools-web-search, checked 8 October 2026). That flag is one vendor's API surface, not a behaviour we can show holds everywhere, but it is enough to retire one assumption: on that API, "retrieval happened" does not mean "it read today's page". It may have read a copy of your pricing page from whenever the index last refreshed.

The gates in front of those two paths are separate, and a robots.txt written before the split existed will not respect it. OpenAI's crawler page lists four user agents; three of them matter here. GPTBot crawls content that may be used in training the foundation models, OAI-SearchBot surfaces websites in ChatGPT's search features, and ChatGPT-User handles the fetch that happens when a person asks a question (developers.openai.com/api/docs/bots, checked 8 October 2026). Perplexity draws the same line and is blunter about the consequence: PerplexityBot "is not used to crawl content for AI foundation models", while Perplexity-User is a user-initiated fetcher that generally ignores robots.txt rules (docs.perplexity.ai/docs/resources/perplexity-crawlers, checked 8 October 2026).

On those two vendors' documented behaviour, blocking the training crawler opts you out of the slow, uncheckable path and does not remove you from the answers. Blocking the search crawler is the costly one. On OpenAI's account, a site that opts out of OAI-SearchBot drops out of the answers ChatGPT's search features produce, while remaining available as a navigational link (developers.openai.com/api/docs/bots, checked 8 October 2026). A single User-agent: * disallow does that second thing without meaning to. We cannot see how often that has happened across the web and we have not counted it, so read it as a failure mode to check on your own site, not a measured prevalence. The access question is the first thing what is answer engine optimisation puts in front of identity and corroboration, and this is why.

What a hybrid answer actually looks like

Real answers are rarely pure. The engine fetches three pages, finds two of your five facts, and writes the other three from memory, in the same paragraph, in the same confident voice. Nothing in the output marks the seam.

There is a mechanical tell where an API exposes one. OpenAI's web search tool returns a sources field that its documentation describes as the complete list of URLs the model consulted, and notes that the number of sources is often greater than the number of citations (developers.openai.com/api/docs/guides/tools-web-search, checked 8 October 2026). Consumer interfaces do not all expose that field, so this is an observation about one documented API rather than a property of search answers in general. Consulted and cited are different sets. A claim about you can sit next to a citation that does not support it, because the citation was earned by the sentence before.

This is also where the failure gets expensive. When retrieved text disagrees with what the model already believes, the model may not defer to the text. A 2026 preprint, "Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict" (arXiv:2605.14473, submitted 14 May 2026, revised 19 July 2026), reports that under a worst-case TruthfulQA misconception-injection probe, standard retrieval-augmented generation reached only 15.0 percent. That number comes from closed-API evaluations of Gemini-2.5-Flash and the Claude family under deliberately misleading retrieval, so it is an adversarial ceiling test on two model families, not a rate to expect on ordinary questions about your brand. The lesson for a brand is narrow but real: publishing the correct page does not automatically overwrite a strong wrong prior. It competes with it.

The practical shape of that competition is easy to picture. A company rebrands, updates its pages, and months later an engine still names the old product alongside a link to the new page. Retrieval worked. Memory won the sentence. That is an illustration of the mechanism above, not a case we have measured and can attach a frequency to.

Google's version of the same split

Google describes AI Overviews and AI Mode using query fan-out, issuing several related searches across subtopics and data sources rather than one query. To appear as a supporting link, a page must be indexed and eligible to be shown in Search with a snippet, and Google states plainly that neither indexing nor serving is promised (developers.google.com/search/docs/appearance/ai-features, checked 8 October 2026).

Two things follow, for Google. Index status is the entry ticket: Google's stated requirement is that a supporting link be indexed and eligible to be shown with a snippet, so a page outside the index is not eligible there however well written it is. Other engines run their own indexes and do not publish the same requirement, so this is Google's rule rather than a shared one. And fan-out means the queries that reach retrieval are not the question the user typed, which is one reason why AI search is not really a ranking game: you are not competing for a slot, you are trying to be the best passage for a query you never saw.

What retrieval can reach is narrower than a sitemap suggests. Content behind a login, a consent wall or an aggressive bot filter is out of reach of a crawler that honours those gates, which leaves memory as the source for it. The exception is documented rather than hypothetical: Perplexity says Perplexity-User generally ignores robots.txt because a person asked for the fetch, so a robots rule and a login are not the same barrier. Paywalls, logins and what retrieval can reach covers that boundary in detail.

The check you can run this week

Pick six prompts where your brand should appear. For each one, do four things.

Run it with retrieval available and record whether you were named, what was claimed, and which URLs were cited. Then run the same prompt with browsing or search turned off, so only parametric memory can answer. Testing with retrieval on and off has the full protocol, including how many repeats you need before a difference means anything.

Now sort the result into one of four boxes. Correct in both: nothing to do. Correct only with retrieval on: your pages are working and the model's prior is stale, so the job is corroboration and patience. Wrong in both: your own pages are probably contradicting each other, which is a fix you fully control. Correct only with retrieval off: the uncomfortable one, and it points at a third-party page outranking your own for the facts about you. We have not measured how often each box fills, so treat the four boxes as a sorting device and your own runs as the evidence.

Fourth, check your server logs for the agent names above, and read them carefully. A log showing GPTBot and no OAI-SearchBot across the window you retain tells you that no OAI-SearchBot request was recorded in that window, which is weaker than proof that the agent did not come. Logs rotate, a CDN can absorb requests before they reach your origin, and a user-triggered fetcher can arrive under a different string. The sample audit report shows how those four boxes read when the evidence is attached to each finding.

Honest limits

Engines do not publish which internal index served a given answer, so the retrieval-on and retrieval-off comparison is an inference and not a reading. Every documented behaviour cited here is a vendor describing its own system, checked on 8 October 2026 and re-read on 12 October 2026; we have not verified that engines outside those three split training and search access the same way, and cutoff dates move without notice. The 15.0 percent figure bounds a worst case on two closed-API model families under deliberately misleading retrieval, and says nothing about the rate at which ordinary answers about a brand go wrong. We have not counted how often each of the four boxes fills, or how many sites block a search crawler by accident, and we are not putting a number on either. A model can also hold a correct fact from training and a correct fact from retrieval and blend them into a sentence that is wrong, which no box above catches cleanly.

FAQ

How do I know if an answer came from training data or retrieval? Look for citations and run the prompt again with browsing disabled. No citations and a stable answer across both runs points at parametric memory. A changed answer, or one carrying URLs, points at retrieval. Neither signal is perfect, so repeat each prompt several times.

Will blocking GPTBot remove my brand from ChatGPT answers? Not on OpenAI's published account of its own agents, checked 8 October 2026. GPTBot crawls content for training the foundation models. OAI-SearchBot is the agent tied to ChatGPT's search features, and ChatGPT-User handles a fetch triggered by a person's question. Blocking the training crawler and leaving the other two open is a coherent choice. Other engines document their own agents separately, so check each one.

My pages are correct and the engine still says the old thing. Why? Retrieved text does not always win against a confident prior, and an engine may be serving a cached copy rather than a live fetch. Keep the correct page, make the correction explicit on the page rather than implied, and get it repeated somewhere the engine already trusts.

Does fresh publishing shorten the gap? It helps the retrieval path and does nothing for the training path until the next model is trained. Treat them as two clocks running at different speeds.

Sources

Related reading

What is answer engine optimisationWhy AI search stopped being a ranking gamePaywalls, logins and what retrieval can reachTesting with retrieval on and offWhen answer engine optimisation is not worth doing yetWho owns AEO inside a company