Why Reddit gets cited so much, and what to do about it
The short answer: Reddit gets cited because it carries the one thing marketing pages rarely carry, named people arguing about specific products in specific situations, and because its public robots.txt disallows the whole site to crawlers that obey it, so whatever route the content takes into answers is not the one your site uses. You cannot copy that, and you should not try to farm it. What works is supplying the answer Reddit threads keep failing to supply, on a page you own, in language a buyer would use.
The supply gap, not a conspiracy
Ask an engine something a brochure cannot answer and watch what it reaches for. Questions like "does this break at 400 users", "what did you move to after cancelling", "is the annual plan worth it for a two person team". Vendor pages avoid all three. Review sites answer them in a template. A four-year-old forum thread answers them in the words of somebody who actually paid.
Retrieval does not reward community content for being community content. It rewards passages that contain the specific claim, with enough context around them to be quoted safely. Forum threads win a particular slice of questions because nothing else in the index is competing for it. Across our own audit prompt sets, about 45 prompts run on five engines with retrieval on and off, our read is that the gap is narrowest on branded and documentation queries and widest on the messy comparison questions that decide deals. That is a reading of our client runs, a sample of one studio's prompts rather than a census of what engines do.
The part almost nobody checks: Reddit's robots.txt
Here is a check you can run in one line, and the result tends to surprise people:
`` curl -s www.reddit.com/robots.txt ``
We fetched the file on 8 October 2026 and again on 12 October 2026. Both times it carried a comment about an open internet, a pointer to Reddit's public content policy, and then two operative lines: User-agent: * and Disallow: /. Read literally, that withholds the whole site from crawlers that obey the file. A robots directive is a published preference rather than an enforcement mechanism, so the file tells you Reddit's stated position and nothing about who actually fetches.
The second check is a spot check, not a study. On 12 October 2026, from one machine in one location, a single request for one public subreddit sent with the user agent string GPTBot/1.0 returned 403, while the same URL with an ordinary browser agent returned 200. One request each. A user agent string is self-declared, so this shows how Reddit treats that string, not how it treats OpenAI: OpenAI publishes IP ranges for GPTBot, OAI-SearchBot and ChatGPT-User, and only a check against those ranges identifies a real crawler.
So whatever route Reddit content takes into AI answers, the public robots.txt is not what grants it. We cannot see which route it is. What is documented is that such routes exist: Google's crawler overview, last updated 12 June 2026, describes special-case crawlers as crawlers used by specific products where there is an agreement between the crawled site and the Google product, and notes that AdsBot ignores the global robots.txt user agent. That is Google describing Google's own crawlers. We looked for published terms covering Reddit in the engine documentation listed below and did not find them, and we are not claiming to know that such an arrangement exists.
That changes which question is worth asking. Where an engine's access to a platform rests on an arrangement rather than an open crawl, supply can change without anything on the platform changing and without any publisher doing anything. We have watched the share of forum citations in one assistant fall sharply inside a couple of weeks while the other engines we track held steady, in client prompt sets that were not designed to measure it, so we can describe the shape of that move and not its size. The threads were still there. Treat it as one observation, not a rate you can plan against.
Why farming it fails in a way that costs you
The obvious move is to build the threads yourself. Post, seed upvotes, add replies, get cited. Teams do this, and the first sixty days often look great.
Then the whole thing can be removed in one administrative action. The terms are set by the platform and not negotiated with you, and a community can be closed without a penalty ladder, a warning, or a recoverable archive. When a community goes, the URLs inside it go with it, and citations that pointed there stop resolving. We have not measured how often that happens and would not quote a rate for it. We raise it because the loss is not partial: nothing is portable, because nothing was ever yours.
A second, quieter failure mode matters more for most companies. We have seen engines go on citing URLs that no longer resolve, so a dashboard can show your citation count holding up after the underlying pages have gone. We have not measured how long that lag runs, and there is no reason to assume the lag is the same from one engine to the next. You get a lagging indicator that reads like success. That is one reason we treat a citation count with no URL check behind it as a number that has not been audited yet, and why the third-party sources AI engines keep citing is worth reading as a map of where you are a tenant rather than an owner.
Engines disagree about forums, and that is the opening
Treat "AI search" as one channel and this topic becomes unmanageable. Split it by engine and it becomes a resourcing decision.
In our own prompt runs, assistants that lean on a general web index return forum threads often on consumer and comparison questions, while assistants built around research and long document work return first-party product pages and documentation far more of the time and social sources far less. We are not quoting an overlap figure, because we have not measured one to a standard we would defend in an audit. Read this as the direction we see in client runs, not a quantity, and the safe conclusion is only that engines differ enough to be worth checking one by one.
Run the same prompt on two engines, record which domains each one keeps, and you will have your own version of that split within an afternoon. The self-check has the procedure if you want it written down. Where forums dominate, your lever is offsite and slow. Where brand domains dominate, your lever is your own site, which is the cheaper fight, and the one that holds its value.
What to actually do, in order
Mine the threads instead of posting in them. Collect twenty threads where your category is being discussed and list the questions nobody answered well. Those are your next twenty pages. The thread is research, not a channel.
Answer in the buyer's words, with the number attached. A thread that beats you says "it fell over around 400 concurrent users on the mid tier". Your page says "enterprise grade scalability". The passage retrieval layer can quote one of those. Write the first.
Get into the lists that engines reach for next. If forum sourcing is turned down in an engine, something fills the slot, and reference pages and curated roundups are the obvious candidates. That is an expectation we hold, not a thing we have measured. The listicles you are missing from covers how those pages get assembled and how to be in the next revision.
Check you are servable before you blame your content. Reddit's gate is deliberate. Yours is usually an accident: a bot rule, a challenge page, an edge rule that treats a datacentre IP in one region differently from a browser in another. See geography, CDNs and who gets served your page, then verify with your own logs rather than a plugin's claim. Worth holding alongside this: Google's AI features documentation states there are no additional requirements to appear in AI Overviews or AI Mode beyond ordinary search best practice. That is Google's statement about Google's surfaces, and it is a useful check on any advice, ours included, that implies a hidden lever.
Size the prize honestly. Being named in an answer is worth having. It is not worth modelling as a traffic line, because the referral volumes reported in the public datasets we cite are small, and some of what does arrive is unattributed. How much traffic AI search actually sends has the uncomfortable version of that.
Honest limits
This post does not establish that Reddit has a licensing arrangement with an engine. We verified two things ourselves and nothing more: that https://www.reddit.com/robots.txt carried User-agent: * and Disallow: / on 8 and 12 October 2026, and that on 12 October 2026 one request for one public subreddit with the user agent string GPTBot/1.0 returned 403 from one machine while a browser agent returned 200. One request each is a spot check, not a measurement, and a self-declared user agent is not proof of who is asking. Everything about why Reddit appears in answers is inference from the outside: we cannot see the retrieval stack, we do not know which engine is weighting forum content up or down this month, and the special-case crawler category we cite is Google documenting Google rather than evidence about Reddit. The observations about citation share moving, dead URLs still being cited, and subscriber count predicting nothing come from client prompt sets that were not designed to test them, which is why no number is attached to any of them. We also have no reliable way to attribute a forum citation to revenue, because the referrer data is thin, the sessions are short, and the assisted path is invisible.
FAQ
Should we post on Reddit at all? As a person with a real account, answering questions in your own name, yes, if you can sustain it. As a visibility programme with sock puppets and seeded engagement, no. The downside is total loss of the asset, with no appeal worth the name.
Does a bigger subreddit get cited more? Not in the runs we have looked at, though we have not measured it properly. Subscriber count was a poor predictor of which threads came back cited in our client prompt sets, and thread specificity and how directly a comment answered the question looked like they mattered more. Treat that as a hypothesis, not a finding.
If Reddit blocks crawlers, why is it in the answers? We do not know. What we can show is that the public file does not grant it: fetch https://www.reddit.com/robots.txt yourself and you will see a blanket disallow, as we did on 8 and 12 October 2026. Routes outside the public file are documented in general terms, Google's crawler overview describes special-case crawlers governed by an agreement between the site and the Google product, but we have not found published terms covering Reddit.
Can we get a thread removed if it is wrong about us? Rarely, and trying usually makes it worse. The durable counter is a page of your own that answers the same question better, plus corroboration from sources the engines already trust.
How do we measure any of this? Fixed prompts, more than one engine, repeated on a schedule, with the cited URLs stored so you can tell a live citation from a ghost. The sample audit report shows the shape we use.
Sources
- Reddit robots.txt, fetched 8 October 2026 and again 12 October 2026, both times returning
- Google Search Central, Google crawlers and fetchers overview, last updated 12 June 2026,
- Google Search Central, AI features and your website, last updated 10 December 2025, on
- OpenAI, bots and crawler user agents, on GPTBot, OAI-SearchBot, ChatGPT-User and their
- Reddit's public content policy is linked from the robots.txt comments. We have left the
User-agent: * and Disallow: /: www.reddit.com/robots.txt
on special-case crawlers used by specific products where there is an agreement between the crawled site and the Google product: developers.google.com/search/docs/crawling-indexing/overview-google-crawlers
there being no additional requirements to appear in AI Overviews or AI Mode: developers.google.com/search/docs/appearance/ai-features
published IP ranges: developers.openai.com/api/docs/bots
URL out of this list because it returned 403 to every non-browser request we made on 12 October 2026, so we cannot ask you to verify it the way we verified the rest.