Blog · Foundations

When answer engine optimisation is not worth doing yet

Published 11 August 2026 · Updated 8 October 2026 · 8 min read

When answer engine optimisation is not worth doing yet
Photograph: Jake Campbell / unsplash

The short answer: Answer engine optimisation is not worth doing yet when one of four things is true. The engines cannot fetch or snippet your pages. Almost nobody asks the shape of question that triggers an AI answer in your category. Your own site disagrees with itself about what you sell and what it costs. Or your runway is shorter than the lag between shipping a fix and a re-crawl. Fix the precondition. The optimisation only pays on top of it.

Everything below is a check you can run this week, mostly with a terminal and a spreadsheet. None of it needs a tool, and all of it is cheaper than discovering the answer six months into a retainer.

The answer surface has to fire at all

An AI answer is not a constant feature of search. On Google, at least, it fires far more often on long and question-shaped queries than on short ones. Pew Research Center collected 68,879 Google searches from the browsing data of 900 US adults during March 2025 and found an AI summary on 18% of them. Split by query length, the picture is lopsided: 8% of one and two word searches produced a summary, against 53% of searches of ten words or more. Queries beginning with a question word such as who, what, when or why produced one 60% of the time. Pew says in its methodology that it could not identify AI summaries on other search engines, so this is a finding about Google and not about assistants in general.

Read that as a rough ceiling on one surface, not a law. If your category's demand is dominated by brand names, part numbers, short transactional phrases or navigational lookups, the surface you are optimising for fires rarely, and the budget buys you a presence in answers that are mostly not being generated. Whether the same length effect holds inside ChatGPT, Perplexity, Copilot or Grok is not something we or Pew can see from outside, since none of them publish a trigger rate.

Here is the check. Export twelve months of Search Console queries, add a column counting words per query, and sum impressions for queries of six words or more. A manufacturer of replacement bearings, where most demand is a part number, will find that number tiny. A category where buyers ask comparison and how-to questions will find it large. That split, not an industry average, tells you whether the work has anywhere to land.

Precondition one: snippet eligibility, which is not the same as AI readiness

Google's own documentation on generative AI features is blunt about the entry condition. A page must be indexed and eligible to be shown in Google Search with a snippet. The same controls apply: nosnippet, data-nosnippet, max-snippet and noindex restrict what can appear in AI features exactly as they restrict a normal result.

The version of this we keep meeting in audit work is historical. Somebody in legal or communications added max-snippet:0 or wrapped the product description in data-nosnippet years ago, to stop scrapers or to force a click. The tag survived the redesigns because nobody owns it. Under the rule above that site is excluded from Google's AI features by construction, and no amount of writing fixes it. We have not counted how often this happens, so read it as a pattern we have run into rather than a rate. Other engines publish no equivalent snippet control we can cite, so we cannot tell you what the same tag does to them.

Check it per template rather than per page, because templates are where these tags live:

`` curl -s -A "Mozilla/5.0" example.com/pricing | grep -iE "nosnippet|max-snippet|noindex" ``

Run it against the home page, a product page, a pricing page, a docs page and a blog post. One hit on a template is a one-line fix and it outranks every other item on your list.

Precondition two: the fetched page has to contain the answer

A system reads what its fetch returns. If your price, your feature list or your location only appears after JavaScript runs, a pipeline that works from raw HTML sees a shell. Google documents a rendering stage: Googlebot crawls, queues the page for rendering, then indexes the rendered HTML. The other assistants do not document whether they render, and from outside their networks we cannot observe it, so the safe assumption is the one you can test yourself.

The test takes a minute. Pick a string a buyer would search for, your starting price, a capacity figure, a city name, and look for it in the unrendered response:

`` curl -s -A "Mozilla/5.0" example.com/pricing | grep -c "1,499" ``

Zero means the claim does not exist as far as a plain fetch is concerned. When a site that reads well to a human is missing from assistant answers, this is worth ruling out early, because the test costs a minute and the fix is usually a template change. It is the reason AI search is not a ranking game has become the house line: you are not losing a position, you are absent from the candidate set.

Precondition three: one version of your own facts

Retrieval-augmented generation, as set out in Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, NeurIPS 2020, retrieves a small set of passages and conditions the answer on them. Production engines do not publish how they chunk a page or how many passages they keep, so treat that as the general shape rather than a specification. It is enough to explain the failure. When your pricing page says one number, your docs say another and a 2023 press release says a third, the answer reflects whichever passage survived that cut. Nobody is wrong on purpose. The model is simply summarising a site that cannot keep its story straight.

Doing visibility work before this is resolved buys you more mentions of a muddled claim. A brand that gets quoted incorrectly at ten times the volume is worse off than one quoted incorrectly at one, which is the whole argument in brand safety when a model is doing the talking. Audit your own contradictions first: price, headcount, founding year, supported regions, integrations, certifications. Pick one canonical statement for each and make every surface repeat it.

Precondition four: your clock against the engines' clock

Access fixes wait on a re-crawl. Nobody publishes a re-crawl interval for your site, and we have not measured one, so we will not give you a number. What we can say is the ordering: on the sites we have re-audited, the access fixes showed up before the identity work, and third party corroboration came last and least predictably, because you do not control it.

Crawling volume has been rising on at least one large network. Cloudflare reported, on 1 July 2025, that across a fixed cohort of its customers AI and search crawler traffic grew 18% from May 2024 to May 2025, with GPTBot requests up 305% and Googlebot up 96%. Not everything rose: in the same table ClaudeBot fell 46% and Bytespider 85%. A network-wide total also says nothing about how often anyone re-crawls your site, and much of that volume is training rather than answer-time retrieval, so take it as context and not as a schedule.

So the honest test is a cash one. If your plan needs qualified pipeline before the current quarter closes, this is unlikely to be the channel that delivers it. Start it when you can hold the work for two or three quarters without flinching.

Things that are not worth buying yet, separately from timing

A dashboard before a question list. A visibility score tells you a number moved. It does not tell you which passage did it or what to ship on Tuesday. Write twenty questions your buyers genuinely ask, run them by hand across two engines three times each, and you will see which passages are being used, which a single score does not show you.

Machine-readable files invented for the purpose are the other one. Google's guidance states plainly that no new AI text files or special structured data are needed to appear in its AI features. No other engine documents consuming them either, which leaves the question open rather than settled, as covered in does llms.txt do anything yet.

When it flips to worth doing

Three signals together. Your long-tail, question-shaped impressions are a meaningful slice of demand. Your pages are fetchable, snippet eligible and carry their claims in raw HTML. Your facts agree with each other. At that point the remaining gap is identity and corroboration, which is where structured work earns its keep, and where the definitions in what is answer engine optimisation start to matter operationally rather than as vocabulary.

If you want to run the preconditions yourself before spending anything, the self-check is ten tests you can finish in an afternoon.

Honest limits

The Pew figures describe Google, in March 2025, on a panel of 900 US adults, and Pew re-ran those searches through a scraper between 7 and 17 April 2025, so what it recorded is not necessarily what the searcher saw. Those percentages are a year and a half old as we write this, the panel is not your category, and we have not reproduced them.

The ordering in the runway section is our impression from re-auditing client sites, not a measurement: we do not log re-crawl intervals, and we have published no sample size for it, so a careful reader should treat it as experience rather than evidence. The same goes for the snippet-tag pattern, which we have met repeatedly and never counted.

We also cannot tell you a reliable payback period. Attribution from these surfaces is poor and referrers are often stripped. We have no figure to offer, and anyone quoting you a payback window with a decimal point in it is working from something they have not shown you.

FAQ

Is a small site too small for this work? Size is not the gate. Question-shaped demand is. A ten page site in a category where buyers ask comparison questions has more to gain than a thousand page site selling part numbers.

Should I block AI crawlers until I am ready? Blocking removes you from the answers those systems write, which is a business decision rather than a safe default. If the content is paywalled archive, blocking may be right. If it is a product site, it rarely is.

Can I do the preconditions without hiring anyone? Yes. Snippet tags, raw-HTML checks and fact contradictions are all findable with curl, grep and an afternoon. The expensive part starts after those are clean.

How do I know when to stop waiting? When your question-shaped impressions are growing, your pages pass the fetch tests, and you can commit two quarters of patience. Until all three hold, money moves faster elsewhere.

Sources

Related reading

What is answer engine optimisationAI search is not a ranking gameBrand safety when a model is doing the talkingDoes llms.txt do anything yetWho owns AEO inside a companyAEO vs GEO vs SEO: what the three terms actually mean