Blog · Content that gets cited

Pruning thin content to raise AI visibility

Published 1 September 2026 · Updated 8 October 2026 · 9 min read

Pruning thin content to raise AI visibility
Photograph: Patti Black / unsplash

The short answer: Pruning raises AI visibility when the thin pages you remove were competing with your own strong page for the same question. Where there was no such overlap, we have not seen it move anything. Cut or merge pages that answer a question your better page already answers, keep anything an AI crawler actually fetches, and redirect rather than delete whenever the old URL holds a real answer. The win is concentration, not tidiness.

Most pruning advice is inherited from a world of ten blue links, where the argument was crawl budget and site quality. Some of that still holds. The reason pruning moves an AI answer is different, and worth understanding before you open a spreadsheet of URLs.

Why thin pages cost you in an answer, not just in a ranking

An answer engine does not pick a site. It picks passages. A buyer question gets broken into several narrower searches, candidate documents come back, those documents get cut into chunks, and a handful of chunks survive into the context the model writes from. One engine documents the first half of that openly: Google's AI features documentation, last updated 10 December 2025, says both AI Overviews and AI Mode may use a query fan-out technique, issuing multiple related searches before composing a response. No other engine publishes an equivalent description, so treat the rest of the picture as the standard retrieval design rather than as documented behaviour at ChatGPT, Perplexity or Claude. The retrieval and chunking steps are not visible to us from outside.

Now picture three pages on your site that all half-answer "how much does X cost". A pricing page with real numbers, a two hundred word blog post from 2023 that gestures at pricing, and an FAQ entry that says pricing depends on your needs. One fan-out query goes looking for your pricing. All three are indexed and snippet-eligible, so all three are candidates. The retriever has no loyalty to the page you are proud of. It keeps whichever chunk scores best against that narrow query, and a short page that is entirely about pricing can beat a long page where pricing is one section among nine.

That is the mechanism as far as anyone outside the engines can see it. Thin pages are not inert. They are near-duplicates in the same embedding neighbourhood as your best page, and they split the signal exactly where you could least afford it. What makes a page citable by an AI engine goes into what the surviving chunk needs to contain. Pruning is the other half of that argument: removing the chunks that get picked instead.

Crawl economics, with the actual thresholds

The second effect is cheaper to explain and easier to overstate. Crawling is a budget, and URLs that return nothing worth having spend it. Google's crawl budget documentation, updated 22 July 2026, is specific about who this applies to: sites with more than 1,000,000 unique pages whose content changes about weekly, and sites above 10,000 unique pages whose content changes daily. Below those sizes, crawl budget is usually not your problem, and anyone selling you a prune on crawl-budget grounds for a 400 page site is selling you a story.

The same page gives one instruction that matters more than the thresholds: eliminate duplicate content so that crawling concentrates on unique content rather than unique URLs. That is Google's guidance about Googlebot, and Google is the only crawler that publishes a crawl-budget model at all. OpenAI, Anthropic and the other operators document their user agents and how to block them, not how much they fetch or how often. In the client server logs we read, the documented AI user agents fetch a far smaller share of a site's URLs than Googlebot does, which is a reason not to spend their attention on near-duplicates, but we cannot see those fetch budgets from outside and nobody should claim a number for them.

What actually counts as thin

Word count is a bad proxy and everybody knows it, yet it is still what most audits use. Four patterns are worth cutting, and none of them is defined by length.

Overlap pages. Two or more URLs that answer the same question, neither one complete. These are the ones that cost you an answer. Merge them into the page you want cited.

Scaled filler. Pages generated at volume to cover query variations. Google's spam policies, updated 28 August 2026, name this as scaled content abuse, defined as many pages made mainly to manipulate rankings rather than to help people, and they explicitly include pages produced with generative tools that add no value. The policy is about search, and the same pages underperform in retrieval for an unrelated reason: they contain no sentence a model would want to quote.

Stale answers that are still correct-looking. A 2023 integrations page that lists four integrations you no longer have is worse than no page, because it is confidently wrong and it retrieves well on brand plus integration queries.

Tag, author and pagination archives. Mostly index noise. Usually a noindex job rather than a delete.

Notice what is missing from that list. A short page that answers one narrow question completely is not thin. It is a good chunk. Short pages of that kind, a definition, a pricing table, a spec sheet, do turn up as cited sources in the prompt runs we do for clients. We have never measured a word-count threshold and we would not trust one. Brevity is only a problem when it coincides with incompleteness.

The check to run before you delete anything

Here is the failure mode that costs people visibility, and it comes from pruning on the wrong data. Analytics traffic is the usual criterion, and an AI answer that names you without being clicked leaves no session at all. On the client properties we have looked at, AI referral sessions are a small fraction of the pages the documented AI user agents fetch. That is a comparison of two logs on a handful of sites, not an industry figure. The practical point does not need a figure: a page can be fetched weekly by a documented crawler such as OpenAI's GPTBot and still show as zero sessions, so pruning by sessions alone will delete it.

So before a delete list becomes a delete, intersect it with your server logs. Pull the last 90 days of requests, filter to the documented AI user agents, count distinct fetches per URL, and pull any URL with a non-trivial fetch count back out of the list. The details of that log query are in reading your server logs for AI crawler traffic. It takes an afternoon, and in the prunes we have run it is the step that has pulled the most pages back off the delete list.

A second check: search the URL's main claim in two engines and see whether you are already cited for it. If you are, the page is working, whatever the analytics say.

Merge, redirect, noindex or delete

The decision is not binary, and picking the wrong instrument wastes the work.

Merge and 301 when the old page holds a real answer. Move the answer into the destination page first, as its own section with its own heading, then redirect. A redirect to a page that does not contain the merged answer converts a mediocre citation into no citation.

410 or 404 when the page holds nothing worth moving. Gone is an unambiguous signal, and a redirect to the homepage is not: it tells a crawler the content moved there when it did not. We are describing a preference, not a documented engine rule. None of the engines publish how they treat a catch-all redirect.

Noindex, keep live for pages people need and engines do not: internal docs, thin archives, filtered listings. Google's AI features documentation, last updated 10 December 2025, is clear that a page must be indexed and eligible to show with a snippet to appear as a supporting link in AI Overviews or AI Mode, and that nosnippet, data-nosnippet, max-snippet and noindex are the controls available. Those controls work at a finer grain than deletion.

Rewrite when the question is valuable and the page is simply bad. Pruning is often an excuse to avoid that harder job.

Comparison and alternatives pages deserve a special note, because they are usually the thinnest pages on a site and also the ones buyers ask about most. Cutting them is the wrong move. Comparison pages that get cited covers what to put in them instead.

What pruning will not do

Pruning concentrates what you own. It does nothing to what other people say about you, and in a lot of categories the third party sources are doing most of the work. Review sites, forums and community threads get pulled into answers constantly, which is the subject of forums, communities and user content in AI answers. Deleting 200 blog posts will not change a Reddit thread that ranks for your brand.

It also will not buy you anything durable if your visibility is sitting on surfaces you do not control. That split is worth drawing before the pruning budget is approved, and rented visibility vs owned visibility in AI search is where we draw it.

Honest limits

We cannot give you a clean before-and-after number for pruning, and we distrust anyone who can. The reason is attribution. Pruning runs over weeks, engines re-crawl on their own schedule, model updates land in the middle of your window, and in our own repeated prompt runs two identical prompts can return different sources ten minutes apart. What we have seen across client work is a pattern rather than a figure: consolidating overlapping pages into one page tends to move that one page into answers it previously lost to its own siblings, and broad deletion of low-traffic pages with no overlap tends to change nothing.

The effect is also bounded by what else is on the page. Merging four weak pages produces one longer weak page unless somebody rewrites the result.

Three things in this post are weaker than they may read. The retrieval mechanism in the second section is documented only for Google AI Overviews and AI Mode, through the query fan-out note in Google's AI features documentation; the chunking and passage-selection steps we describe at ChatGPT, Perplexity and Claude are inference from how such systems are usually built, not published behaviour. The comparison between AI crawler fetches and AI referral sessions comes from client server logs and analytics on a handful of properties, which is a sample and not a census, and we have deliberately put no number on it. And the preference for a 410 over a homepage redirect is our judgement about what a clear signal looks like, and we have not found an engine that publishes a rule on it. For the shape of a finding that would justify a prune, the sample audit report shows how we attach evidence.

FAQ

How many pages should I remove? There is no correct percentage, and we have not counted enough sites to offer a typical one. Count the questions your site answers more than once, then reduce each of those to a single page. On sites in the hundreds of pages that count is usually small enough to work through by hand. Sites in the tens of thousands usually have a template problem instead, which is a different fix.

Does deleting pages hurt my AI visibility in the short term? It can, for a few weeks, while engines re-crawl and cached copies age out. Redirects shorten that window. Deletions without redirects lengthen it.

Is a short page automatically thin? No. A 300 word page that answers one question completely is an excellent retrieval unit. The problem is pages that are short because they are incomplete, not pages that are short because the answer is short.

Should I prune AI-written content specifically? Prune on whether the page says something useful and checkable, not on how it was produced. Google's spam policies target scaled pages made to manipulate rankings, which is about intent and value rather than the tool used to write them.

Sources

Related reading

What makes a page citable by an AI engineComparison pages that get citedRented visibility vs owned visibility in AI searchForums, communities and user content in AI answersReading your server logs for AI crawler trafficHow often to refresh content for AI search