Blog · Measurement

Building a prompt set that represents real demand

Published 15 June 2026 · Updated 12 October 2026 · 8 min read

Building a prompt set that represents real demand
Photograph: Heather McKean / unsplash

The short answer: Build it from recorded evidence of what buyers already ask, not from a keyword export. Pull the questions out of sales calls, support tickets, site search and review threads, keep the wording the person used, then spread them across four axes: buying stage, branded against unbranded, broad against narrow, and which engine gets asked. Around forty to fifty prompts covers one segment honestly.

The prompt set is the instrument. Everything downstream, every chart, every before and after claim, is only as good as the questions you decided to ask. Get the set wrong and you can measure very precisely for a year and learn nothing.

Your keyword export does not contain this

The first instinct is to open Search Console, sort by impressions and paste the top hundred queries into a prompt list. That produces a set with a known defect.

Google's own documentation is explicit that pages appearing in AI features are folded into overall search traffic, reported under the Web search type, and not split out. There is no AI Mode filter and no AI Overviews filter. So the rows you are reading are a blend of ten blue links behaviour and answer behaviour, with no way to separate them. The Search Analytics API will return at most 25,000 rows per request and defaults to 1,000, and the long tail below that is anonymised away. The questions that look most like prompts, the long specific ones with a constraint attached, are exactly the ones that get cut.

There is a second defect, harder to see. A keyword is a compressed query. A prompt is not compressed, because the person is talking. "crm for small business" and "we are eleven people, mostly field sales, what crm will not fall over when we hit fifty" are the same commercial intent and completely different retrieval events.

Fan-out means your prompt is not the query

Google documents that both AI Overviews and AI Mode may use a query fan-out technique, issuing several related searches across subtopics and data sources, then assembling one answer from what comes back. Read that carefully, because it changes what a prompt set is for.

You are not testing a query. You are testing a question that a model turns into a handful of queries you never see. Two prompts that read almost identically to you can decompose into different subtopic searches and return different sources. Two prompts that read differently can collapse into the same fan-out and return the same answer.

That gives you a cheap diagnostic. Write the same intent three ways, one terse, one conversational, one with a constraint. Run all three. If the cited sets overlap heavily on that engine, the phrasing is not buying you anything there and you can keep one. If they diverge, you have found a subtopic you are invisible inside, and all three belong in the set. The test is easy to skip, and near duplicates are the most common filler in the sets clients hand us: they burn run budget and report the same finding three times.

Four axes, one prompt per cell

A set that represents demand has to cover a market rather than sample it. The structure we use in an audit is a grid, because a grid makes the gaps visible.

Buying stage. Problem framing, option comparison, purchase mechanics. Three different questions with three different answer shapes. Most brands are fine at the third and absent from the first.

Branded against unbranded. Ask with your name in the prompt and without it. The branded prompt tells you whether the engine knows you exist and describes you correctly. The unbranded prompt tells you whether you are in the consideration set at all. The distance between those two results is the number we lean on most when we read an audit, and it is why citation share vs mention share is worth separating before anyone argues about a score.

Broad against narrow. Category level questions are crowded and slow to move. The narrow cut, your vertical, your integration, your geography, is where a smaller brand can be the best available answer this quarter.

Engine and run. The same prompt on ChatGPT, Perplexity, Google AI Mode, Gemini and Claude is five different measurements, not one. Our audits run roughly 45 prompts across five engines, with retrieval on and retrieval off, because a prompt answered from training knowledge and the same prompt answered from a live fetch are testing different systems.

One prompt per cell. If a cell has four prompts and its neighbour has none, your set is not a measuring instrument, it is a list of things somebody thought of.

The test that decides what stays

Here is the part most prompt sets never get. Before you freeze the set, run every candidate three times on two engines and sort the results into three buckets. Three runs is a cheap screen for stability, not a measurement of it.

Prompts that name you every time are monitoring prompts. Keep a few. They will not teach you anything until something breaks, and then they will tell you immediately.

Prompts that never name you, and return a stable set of three or four other sources every time, are the valuable ones. We cannot see inside a run, so stability is not proof that the engine is confident. What it does give you is a named set of sources that keeps winning the slot, and a corroboration gap you can go and close.

Prompts that return a different source set on every run are telling you the engine is uncertain about the whole topic. Those need more runs before they mean anything, and they are the reason citation churn has to be measured before any delta is believed. A prompt with high natural churn will show you a twenty point improvement that is entirely noise.

There is a fourth bucket, and it is mostly the one to delete: prompts that return an answer with no sources shown. What you observed is an answer without visible citations. That is not proof that retrieval never fired, because the retrieval step is not something you can watch from outside. In the audits we have run, these prompts have been the least responsive to on-site work, so we keep two as a control and drop the rest.

Where to find the questions

In rough order of how much we trust them.

Sales and support recordings first. Someone in your company already hears real demand all day, in the buyer's own words, with the objection attached. Pull the last fifty calls and write down the question before the objection.

Site search logs second, because that is people telling you what they could not find.

Review and forum threads third. Slow to read, high in yield, and they carry the comparison framing buyers actually use.

Public research on assistant usage helps you calibrate the mix rather than supply the prompts. The September 2025 NBER working paper on ChatGPT usage, which classified a representative sample of conversations with an automated pipeline, reported that non-work messages grew from 53% to more than 70% of usage, and that practical guidance, seeking information and writing together accounted for close to 80% of conversations. That is one assistant's user base over one window, not a map of how people phrase questions everywhere. If your set is entirely formal procurement language, it still does not look much like what people type.

Keyword tools last, and only to check spelling of the category.

Honest limits

We know of no public dataset that gives honest prompt volumes for ChatGPT or Perplexity. Panel estimates exist, they are extrapolations, and we do not build audits on them. A prompt set tells you how you perform on the questions you chose. It cannot tell you how many people asked them.

Two things above are our practice rather than a measured result. The forty to fifty prompt range and the three run screen are how we build and prune a set, not a finding with a sample size behind it, and we have not tested whether a smaller set would have found the same gaps. The bucket test reads citations only, so every statement here about what an engine did internally is an inference from what it showed.

We also keep finding, across the audits we have run, that a set ages faster than anyone expects. New features, new competitors and new engine behaviour move what gets asked, so we rebuild roughly a third of a set at each re-run and keep the rest fixed. That split is our own working rule rather than a tested optimum, and it is the only way we have found to hold both a trend line and current coverage. That tension is unresolved, and anyone who tells you otherwise is selling a chart. The broader discipline of measuring AI visibility without fooling yourself starts here, because none of it works without a defensible set. If you want to see what a frozen set produces once it has been run, the sample audit report walks through the output section by section. For the larger picture of what this work covers, start with what answer engine optimisation is.

FAQ

How many prompts do I need? Enough to fill the grid without duplicating cells. For a single segment that lands near forty to fifty. Two segments does not mean a hundred prompts, it means a second grid with its own unbranded spine and shared branded prompts.

Should prompts be short or conversational? Test both for the same intent, then keep whichever produced a distinct cited set. The decision is empirical and it differs by category.

How often should I run the set? Often enough to separate signal from churn, which in practice means several runs per prompt per measurement window, not one run per week. A single run is an anecdote with a timestamp.

Can I take my prompt set from a competitor's content? You can take the topics. The phrasing has to come from your buyers, because the phrasing is what decides the fan-out, and their buyers are not yours.

Sources

All four sources below were open and still supported the sentences citing them when checked on 12 October 2026.

Related reading

How to measure AI visibility without fooling yourselfCitation share vs mention shareCitation churn: how fast the cited set changesWhat is answer engine optimisationHow many runs per prompt before a number means anything