Blog · Running the work

Tool sprawl in AI search, and the cost of it

Published 8 October 2026 · 8 min read

Tool sprawl in AI search, and the cost of it
Photograph: Wonderlane / unsplash

The short answer: Tool sprawl in AI search is owning several visibility products that each sample the engines their own way, then paying a second time to reconcile the numbers they disagree on. The licence fee is the cheap part. The real cost is engineering changes to shared files like robots.txt, meeting hours spent arguing about whose dashboard is right, and decisions delayed while nobody owns the answer.

A stack usually grows the same way. In the audits we have run this year, the pattern shows up often enough that we now ask about it first. One person trials a tracker. A second team buys a content scorer. An agency brings its own platform into the account. Six months later there are four logins, four definitions of a mention, and no single person who can say what changed last month and why.

The three bills, only one of which you see

The first bill is the invoice. It is visible, it is annual, and it is the one procurement argues about.

The second bill is engineering. Most of the tools we have seen inside client stacks ask at some point for a change to something shared: a line in robots.txt, a tag in the head, a script on every page, a DNS record, a log export. Those changes queue behind product work. They can also collide. We have seen two tools in one account ask for competing crawler rules in the same file, so this is not hypothetical, though we cannot say how common it is across the category as a whole.

The third bill is attention, and it is the biggest. Four dashboards produce four numbers for the same question. Someone has to work out which is closest to true before anyone can act on it. That reconciliation happens in meetings, repeats every month, and produces nothing a customer will ever see. Teams running the first 90 days of AEO work lose most of their first quarter here rather than to any technical problem.

Why the numbers disagree, mechanically

The disagreement is not a bug in anyone's product. It comes from how these answers are produced.

An assistant writes a fresh answer per request rather than reading it off a fixed index. In our own audit runs, repeating the same prompt in the same engine has returned different sources within the same session. We have not measured that variation across all the engines we test, or across our whole prompt set, so treat it as a property we keep running into rather than a constant we have quantified. Either way, any visibility figure is an estimate drawn from a sample, and the estimate depends on choices each vendor makes privately: how many prompts, how many runs per prompt, which engines, which country the request came from, whether web retrieval was on, whether the session was logged in, and what counts as a mention when your brand appears without a link.

Change any one of those and the number moves. Two tools can both be careful and still report very different figures, because they are measuring different things with the same word. We have not benchmarked competing products against each other and are not going to put a size on that gap. This is also why AI search is not a ranking game: on the engines we have looked at there is no published index position to agree on, only each vendor's reconstruction of what the engines said to a sample of imaginary users.

A check you can run this week. This is the method, not a result we are reporting. Take one prompt that matters. Run it ten times by hand, across two days, in the engine you care about most. Record whether you were named each time. That gives you a rough spread for a single prompt. Now look at the week-over-week movement your dashboard reported. If the reported change sits inside your own hand-measured spread, the tool has shown you noise and you have been managing to it.

The file that quietly collects the damage

Here is a cost that rarely reaches a budget line. Many of these products come with crawler advice, and that advice lands in the same place: robots.txt.

Google's robots.txt documentation, read on 8 October 2026, states a file size limit of 500 kibibytes and says content after that limit is ignored. The same page says Google generally caches the file for up to 24 hours and may cache it longer when refreshing is not possible. OpenAI's crawler documentation, read the same day, says it can take about 24 hours from a robots.txt update for its systems to adjust for search results. Those are two companies' stated behaviour. We have not found equivalent published limits from the other engines, so do not assume the same numbers hold elsewhere.

Two consequences follow for Google at least. A file that grows a block per tool, per vendor, per well-meaning contractor can push the rules you actually depend on past the 500 KiB cutoff, where Google stops reading them. And a wrong edit is not instantly reversible. You revert it in seconds and still wait out the documented cache window.

The list those edits are written against is longer than people assume. Google's common crawlers page listed eleven user agent tokens when we checked it on 8 October 2026, including Googlebot, Google-Extended, GoogleOther and Google-CloudVertexBot. OpenAI's crawler page, read the same day, documented four agents with different purposes: OAI-SearchBot for search, GPTBot for training, ChatGPT-User for user-triggered fetches, and OAI-AdsBot for ad landing pages. That is fifteen names from two companies on one day, before Anthropic, Microsoft, Perplexity or anyone else, and both lists change without notice. No vendor outside those companies controls the list, so rules written against it are a snapshot and need re-reading.

Sorting the stack by question, not by vendor

The useful exercise takes about an hour and needs a spreadsheet, not a consultant.

List every tool touching AI search. Next to each one, write the single question it answers, in the form a director would ask it. Not features. Questions. Something like: are we named when a buyer asks for alternatives, is this page reachable and parseable, did the claim in the answer match our pricing page.

Then sort by question rather than by vendor. Duplicates become obvious immediately. In the stacks we have sorted this way, the common result is two or three products answering the same question with different sampling, plus one gap nobody bought for, usually access or correctness rather than tracking. That is our experience across a small number of engagements, not a survey, so your stack may sort differently.

Keep one tool per question. For each survivor, write down the last decision it changed, with a date and the thing that shipped as a result. A tool that cannot produce one in the past quarter is not informing anything. It is being read.

What to own and what to rent

Some of this stack should never have been a subscription. Prompt lists, the answers you captured, the dates, the judgement about what counts as a mention: all of that is your record of how engines treated you, and it should survive a cancelled contract. The rented versus owned split applies to tooling as much as to visibility itself.

Rent the collection work, because running prompts across engines at volume is tedious and someone has done it already. Own the question set, the raw captures and the call on what the result means. If leaving a vendor would erase a year of your own history, the contract is buying a dependency as well as a feature.

The same logic shapes the staffing decision. Sprawl is often a symptom of nobody owning the work, since buying a tool feels faster than assigning a person, and the in-house or agency question usually has to be settled before the stack stops multiplying.

If you would rather test the gap before renewing anything, the self-check is ten things you can run by hand in an afternoon. Several of them are checks your current tools probably do not perform at all.

Honest limits

We cannot tell you the average number of AI visibility tools per marketing team, because we have found no credible public dataset for this category. The licence figures people quote are vendor list prices, which are not what anyone pays.

The patterns in this post come from the audits this studio has run, a small number of engagements rather than a sample anyone should generalise from. We have not surveyed the market, benchmarked competing products, or measured how often two tools collide in the same robots.txt, so where this post says "often" or "most" it is describing our caseload.

The crawler counts and the robots.txt limits are Google's and OpenAI's published statements, read on 8 October 2026. Both lists change, and we have not verified that other engines publish or honour comparable limits. Absence of a published limit elsewhere is not evidence that none applies.

We also cannot promise that cutting to one tool improves anything on its own. A single bad measurement is still a bad measurement. The gain is narrower than it sounds: fewer arguments about which number is real, and more hours left for the fixes.

FAQ

How many AI visibility tools does a team actually need? Usually one, plus whatever the engines give you free. The threshold is not a count. It is whether each tool answers a question no other tool in the stack answers.

Can I just use the free graders and checkers? For a first pass, often yes. They tend to sample thinly and report a single composite score, so treat the output as a prompt for investigation rather than a measurement you can trend.

Why do two tools report completely different visibility scores? Because they use different prompt sets, different run counts, different engines and different definitions of a mention. Ask each vendor for those four things in writing. The gap usually explains itself.

Is a tool's week-over-week change meaningful? Only if it exceeds the run-to-run variation of the engine itself. Measure that variation yourself once, then use it as the floor for what counts as a real move.

Sources

Related reading

The first 90 days of AEO workIn-house or agency for AEOWhy AI search is not a ranking gameRented visibility vs owned visibility in AI searchWhat an AEO audit should containGetting engineering to ship your AEO fixes