A realistic prompt panel is a versioned sampling frame of the situations in which buyers may use an AI system to discover, compare, validate, or troubleshoot a company or product. It is not a keyword list rewritten as conversational sentences.
Build it in three parts: a stable core of materially different buyer situations, a separate module for wording sensitivity, and a temporary module for product or market changes. Admit a prompt only when it adds a distinct decision context and has a predefined outcome to code. Keep unbranded discovery questions free of the company being measured, and freeze the panel before testing.
If the questions are biased, the results will be precise measurements of a distorted market.
The unit is a buyer situation
Search terms are useful inputs, but they are not the unit of an AI visibility audit. A buyer situation is.
Consider these three questions:
- Category discovery: “Which payroll platforms should a 75-person US software company consider?”
- Constraint comparison: “Compare payroll platforms for a 75-person US company that needs contractor payments in Canada and an accounting integration.”
- Risk validation: “What should I verify before choosing a payroll platform for employees and international contractors?”
They may share keywords, but they represent different decisions. The first tests consideration-set presence. The second tests eligibility under meaningful constraints. The third may surface evaluation criteria and sources without recommending any company.
Cosmetic variants such as “best payroll platform,” “top payroll software,” and “leading payroll tools” add volume without much new journey coverage.
Why prompt realism is difficult
AI interfaces do more than match the literal words a user enters. OpenAI says ChatGPT search may rewrite a prompt into one or more targeted searches, and that general location or relevant memory can affect the rewrite. Google says AI Mode and AI Overviews can issue multiple related searches across subtopics and data sources, a process it calls query fan-out.
Those product details do not tell marketers exactly which searches will occur. They do show why a prompt’s context matters. “Best accounting firm” without a location, company size, industry, or need is not a complete buyer situation. The interface may supply or infer context the auditor did not record.
Prompt design therefore has two jobs: represent meaningful customer diversity and document the context that could change an answer.
Keep three prompt sets separate
One panel usually has to support longitudinal measurement and new learning. Those purposes conflict if every new question changes the denominator. Separate the work instead:
- Stable core: buyer situations that remain comparable across measurement periods. Change this set only through a documented version update.
- Sensitivity module: alternate phrasings, levels of detail, or terminology used to test whether language materially changes the result. Do not count these as independent demand situations.
- Temporary change: prompts for a launch, new regulation, competitor move, seasonal need, or emerging category question. Retire or promote them explicitly after review.
Report each set separately. Otherwise, adding 10 favorable experimental prompts can look like a visibility improvement even when none of the stable core answers changed.
The prompt-admission rule
A candidate prompt belongs in the stable core only when the answer to all four questions is yes:
- Distinct decision: Does it represent a meaningfully different journey stage, audience, need, geography, constraint, or expected decision?
- Realistic language: Can the wording be traced to customer evidence or defended as an explicit research hypothesis?
- Fair eligibility: Could every company being compared reasonably qualify, or is ineligibility itself the declared subject of the test?
- Scorable output: Is the intended outcome—such as mention, recommendation, attribute accuracy, or source type—defined before the run?
If a prompt changes only “best” to “top,” it fails the distinct-decision test. If it copies one company’s proprietary phrase into an unbranded question, it fails the fair-eligibility test.
| Candidate prompt | Decision | Destination |
|---|---|---|
| ”Which payroll platforms should a 75-person US software company consider?” | Distinct category-discovery situation | Stable core |
| ”What are the top payroll tools for that company?” | Wording change without a new decision | Sensitivity module |
| ”Compare options that support US payroll, Canadian contractors, and our accounting system.” | Adds material eligibility constraints | Stable core |
| ”Is Brand A suitable for our company?” | Useful branded validation question, but not unaided discovery | Separate branded module |
| ”Which platform offers Brand A’s named proprietary feature?” | Forces a differentiator associated with the target | Reject or rewrite around the underlying need |
This rule does not make the panel statistically representative. It makes the inclusion logic visible and prevents question volume from masquerading as journey coverage.
The seven-step panel method
1. Define the executive decision
Write the decision the study must inform. Examples include: measure unaided shortlist presence, identify inaccurate product associations, compare visibility by segment, inspect recurring source types, or establish a baseline before a website change.
NIST’s January 2026 initial public draft on language-model benchmark evaluations says the design, execution, and reporting of an evaluation depend on its objective and the intended use of the results. An AI visibility study is not the kind of automated benchmark NIST addresses, but the decision-first principle transfers cleanly.
Do not ask one panel to answer every question. A panel designed for source analysis may need different prompts and coding from one designed for recommendation presence.
2. Map the journey moments
Use the stages where an AI answer could change a decision:
- Problem framing: the buyer is trying to name or understand a need.
- Category discovery: the buyer wants possible solution types or providers.
- Shortlist building: the buyer asks for suitable companies or products.
- Constraint comparison: the buyer supplies required capabilities, budget, geography, or risk conditions.
- Validation stage: the buyer checks claims, reviews, credibility, implementation, or fit.
- Troubleshooting stage: the buyer seeks help before or after purchase, which may reveal documentation visibility and product association.
Not every company needs every stage. A low-consideration product may have little formal validation. A high-risk B2B purchase may require several validation situations.
3. Gather real language
Use evidence already available to the company: sales-call themes, search queries, site search, support tickets, request-for-proposal language, review themes, customer interviews, community questions, and product research. Remove personal or confidential details before the panel is shared or automated.
OpenAI’s evaluation guidance recommends datasets that reflect real-world use and identifies biased datasets that fail to reproduce production traffic patterns as an anti-pattern. AI visibility research should follow the same principle: customer evidence first, synthetic expansion second.
If real language is unavailable, begin with a small hypothesis panel and label it exploratory. Do not present it as measured buyer behavior.
4. Build a coverage matrix
Create a matrix before writing final prompts. Rows contain materially different buyer situations; columns contain the dimensions that change relevance.
| Panel field | Question to answer | Example value |
|---|---|---|
| Journey stage | What decision is the buyer making? | Shortlist building |
| Persona | Who is asking? | Head of people |
| Need | What outcome is required? | US payroll and contractor payments |
| Company context | What changes fit? | 75-person software company |
| Geography | Where must the solution work? | United States and Canada |
| Constraints | What is required or excluded? | Accounting integration; no PEO requirement |
| Prompt type | Branded or unbranded? | Unbranded |
| Expected output | What will be coded? | Recommended shortlist and rationale |
The example is illustrative. Its purpose is to show how a prompt earns its place through a distinct situation, not through a wording variation.
5. Write neutral prompts
Write questions a buyer could reasonably enter. For unbranded discovery, do not name the target company. Avoid adjectives or requirements copied from only one company’s positioning unless customers independently use them.
A biased prompt might ask for “the best payroll platform with Brand A’s proprietary feature.” A neutral prompt describes the underlying buyer need and lets the answer determine which options qualify.
Preserve natural variation where it represents real users: concise and detailed versions, expert and non-expert language, and different legitimate constraints. Do not add deliberate ambiguity unless ambiguity itself is part of the study.
6. Review and freeze
Have at least one customer-facing expert and one research-minded reviewer inspect the panel. Ask whether each company in a comparison is genuinely eligible, whether any prompt forces the expected answer, and whether important situations are missing.
Freeze the prompt text, order policy, conversation policy, and panel version before running the study. If a prompt turns out to be invalid, record the exclusion and reason. Do not quietly replace losing prompts after looking at the results.
That last rule protects the study from overfitting. The same NIST draft warns that optimizing an evaluation protocol directly against the test set can reduce external validity. In this context, editing prompts after seeing which ones favor the target company creates the equivalent problem.
7. Pair prompts with metadata
For every prompt, record a stable ID, journey stage, audience, geography, constraints, brand status, expected coded outcome, and rationale for inclusion. During execution, add platform, interface, date, account and memory state, location, run ID, full response, citations, errors, and human reviewer.
The prompt without its context is not enough to reproduce the observation.
A small illustrative panel
For the fictional payroll-study frame above, a six-situation starter panel might include:
| ID | Stage | Buyer situation | Primary measure |
|---|---|---|---|
| P01 | Problem framing | ”How should a 75-person US company manage payroll for employees and Canadian contractors?” | Category and attribute associations |
| P02 | Discovery | ”Which payroll platforms should a 75-person US software company consider?” | Unaided mentions |
| P03 | Shortlist | ”Recommend a shortlist of payroll platforms for this company and explain the fit.” | Explicit recommendations |
| P04 | Comparison | ”Compare suitable options that support US payroll, Canadian contractor payments, and [named accounting system].” | Eligibility and rationale |
| P05 | Validation | ”What evidence should I check before choosing among these payroll platforms?” | Criteria and source types |
| P06 | Risk | ”What could make a payroll platform unsuitable for this company?” | Limitations and disqualifiers |
This is not a complete study. It still needs repeated runs, a defined consumer surface, valid geography, a coding rubric, and a human check that the commercial scenario is realistic.
Bias checks before execution
Run these checks while the panel can still be changed without contaminating the study:
- Eligibility check: every compared company could reasonably satisfy the prompt, or the analysis explicitly studies why it cannot.
- Brand check: unbranded prompts contain no target name, slogan, trademarked phrase, or forced differentiator.
- Coverage check: the panel contains distinct stages and constraints rather than a cluster around one favored use case.
- Geography check: availability, regulation, language, currency, and local evidence are represented where they affect fit.
- Audience check: the panel does not assume one persona represents every buyer.
- Independence check: the person who wants a specific result cannot silently edit the panel after seeing early answers.
A panel can be commercially relevant without being representative of every customer. State the intended use and the population it does not cover.
What the panel should produce
The output is more than a prompt spreadsheet. It should include the stable core, separate sensitivity and change modules, coverage matrix, frozen prompt text, version history, run instructions, coding rubric, and exclusions. The analysis should report results by journey segment and module before aggregating them.
That segmentation often produces the useful finding. A company may be present in broad discovery but absent under enterprise constraints. It may be recommended but described inaccurately. It may appear in branded validation prompts and never in unbranded shortlists.
Those patterns point to different evidence and marketing decisions.
What remains uncertain
A panel measures the situations it contains. It does not estimate the actual share of all customer conversations unless the prompts and their weights come from defensible behavioral data. Equal weighting is a research choice, not evidence that every situation occurs equally often.
Review the panel when the product, market, customer journey, or platform changes. Keep a stable core for longitudinal comparison and version new prompts separately so expansion is not mistaken for performance change.
Once the panel is frozen, keep mentions, citations, recommendations, and referrals separate in the report.
Sources and methodology
We reviewed OpenAI’s ChatGPT search guidance, OpenAI’s evaluation best practices, Google’s documentation for AI features in Search, and the January 2026 initial public draft of NIST’s practices for language-model benchmark evaluations on August 31, 2026. The three-layer panel, admission rule, seven-step method, matrix, and sample panel are AgenQuest working frameworks. The payroll example is illustrative, not observed customer research.
About this analysis
AgenQuest Research produced this independent method from public documentation and editorial research standards. No platform sponsored or participated in it.