---
title: "Measure First or Fix First? The 20-Prompt Test for AI Visibility"
description: "Should you track your brand in AI answers before fixing pages? Here is the strongest case for each side, followed by a 20-prompt test with a baseline and a control."
url: "https://groas.com/post/measure-first-or-fix-first-the-real-argu"
image: "https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/4b923078-1b62-4926-8867-95c814fb3909.png"
published: "2026-10-16T05:30:26.619Z"
modified: "2026-10-16T05:30:26.682Z"
---

October 16, 2026 · 9 min read

# Measure First or Fix First? The 20-Prompt Test for AI Visibility

[Alexander PerelmanHead Of Product @ groas](https://groas.com/author/alexander-perelman)[LinkedIn](https://www.linkedin.com/in/alexander-433793253/)

![A tiny tabletop garden: gardeners tend one plot while an identical plot marked CONTROL is left alone, and a miniature scientist compares the two.](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/4b923078-1b62-4926-8867-95c814fb3909.png)

In this article

1. [Position A: measure before you touch content](#position-a-measure-before-you-touch-content)
2. [Position B: fix pages before you buy a dashboard](#position-b-fix-pages-before-you-buy-a-dashboard)
3. [My ruling: take a baseline, then get out of the dashboard](#my-ruling-take-a-baseline-then-get-out-of-the-dashboard)
4. [The 20-prompt test: what would change my mind](#the-20-prompt-test-what-would-change-my-mind)
5. [The verdict on tools: pay for the work, not the chart](#the-verdict-on-tools-pay-for-the-work-not-the-chart)

Should you pay to track your brand in ChatGPT, Gemini and AI Overviews before fixing a single page? Smart people disagree: without a baseline, you cannot tell whether a rewrite worked; with AI answers changing from run to run, you could spend weeks measuring noise. **I would measure 20 buyer prompts, fix a defined set of pages, then re-measure against an untouched control.** That is a smaller bet than a monitoring program and a better test than rewriting on instinct.

The question is not whether visibility matters. It is whether measurement tells you what to do soon enough to justify the time and money. I want to give both camps their best argument before I rule. One has a real triage problem to solve. The other has seen enough charts that never led to a fix.

## Position A: measure before you touch content

The measure-first case starts with a practical problem: say you have 60 pages you _could_ rewrite. Which one goes first? Google rankings alone will not tell you when a page ranks but [never gets cited in an AI answer](https://hginsights.com/blog/what-is-ai-visibility-monitoring/). You need to see the buyer question, the answer, whether your brand appears, whether your domain is cited and who gets the space when you do not.

A serious baseline does more than count screenshots. One [monitoring protocol](https://blog.hubspot.com/marketing/best-aeo-checkers) calls for 25 to 40 buyer questions across ChatGPT, Gemini, Perplexity, AI Overviews and AI Mode, repeated weekly for a month. That gives you a fixed panel rather than a rotating collection of prompts somebody remembered to try after lunch. Keep branded questions separate: a customer asking for you by name is not evidence that you win a category question.

The same discipline applies to share of voice. Freeze the wording, track named competitors pulled from the answers, and compare [mentions, position and citations together](https://dev.to/pineprompt/how-to-measure-share-of-voice-in-ai-search-3o9l). One worked example puts a brand at 15% of mentions but 10% of citations, with a 5% result on one engine and 25% on another. Roll that into a single visibility score and you hide the part you need to fix. I used to tell clients one number was enough. I was wrong.

Measurement also keeps the content team honest. If buyers ask a question and the answer repeatedly cites a competitor, that is a more useful assignment than “make this page more AI-friendly.” If your brand gets named but your page does not get cited, that is a different assignment again. **The strongest measure-first argument is triage:** establish where you lose, then spend your editing time there.

## Position B: fix pages before you buy a dashboard

Fix-first people are not afraid of numbers. They are afraid of treating unstable answers as settled evidence. In a [citation-volatility panel](https://parse.gl/research/ai-citation-volatility-by-industry), consecutive cited answers shared only 33.3% of hostnames on ChatGPT Search and 24.7% on Google AI Mode. An earlier panel showed 75 to 81% churn on ChatGPT and 63 to 72% on AI Overviews by industry. If you check once and announce a loss, you may be reacting to the sampling rather than the page.

Even the measurement guidance admits the cost of making that chart useful: [run each prompt at least three times in fresh chats, average the runs, and wait for a steady four-week change before calling it signal](https://dev.to/pineprompt/how-to-measure-share-of-voice-in-ai-search-3o9l). That is defensible research. It is also work a small team could have spent making its existing pages clearer.

Then there is the dashboard problem. A team can spot a false price or a lost recommendation and still have no process for fixing either. One [fix-first playbook](https://akii.com/blog/ai-monitoring-signal-to-action-system) calls this _dashboard paralysis_. The subscription keeps running while the page stays the same. [Listed entry tiers](https://blog.hubspot.com/marketing/best-aeo-checkers) include $29 a month for 15 prompts on Otterly Lite, about $95 for 50 on Peec Starter, $99 per domain for 25 on Semrush, and $199 per engine on Ahrefs; a daily 200-prompt setup can run $417 to $974 a month. Those are monitoring prices, not page fixes.

Say you spend $20k a month on search. The objection is not that a tracking bill will swallow that budget. It is that three months spent arranging charts are three months in which the pages an answer might quote remain unchanged. **The strongest fix-first argument is opportunity cost:** an answer block, a clear heading or a corrected fact can do useful work while a dashboard can only describe the gap.

## My ruling: take a baseline, then get out of the dashboard

I side with a narrow version of measure-first, and I switch to execution fast. **Measure 20 prompts, fix, re-measure.** Not 200 prompts checked daily. Not zero tracking because a page looks like it ought to work. Without a baseline and an untouched control group, I cannot tell whether a citation gain followed an edit or Tuesday’s model shuffle. That single consideration decides it for me.

The big monitoring program loses because its precision does not help if nobody acts on what it finds. Pure fix-first loses because a rewrite without a comparison tells you little about why the answer changed. Twenty prompts are enough to give the work a target while keeping the project small enough to move from observation to editing.

There are readers who should skip my version. If you have fewer than ten pages worth fixing, fix them rather than build a spreadsheet to rank them. If you manage forty client accounts, manual repeats will not scale; automate from day one. For the team in between, the point is to spend as little time measuring as you can without giving up the control that makes the next decision credible.

![A tiny paper checklist labeled 20 prompts balancing against a giant wall of charts](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/b0a02a3d-fcb1-4967-8080-05d6d4dd0d32.png)

## The 20-prompt test: what would change my mind

Here is the question I would put in the notebook: **does rewriting pages for quotability lift mentions and citations more than leaving comparable pages alone?** I expect the rewritten group to pull ahead because a direct answer block gives an AI answer something short and relevant to use. That is an expectation, not a result. The test has to leave room for the control group to win, or for neither group to move. The fuller holdout logic is in [the prompt-trigger test](https://groas.com/post/the-prompt-trigger-test-find-out-which-b).

### Setup: freeze the questions before looking at the answers

1. Choose 20 buyer prompts. Use ten category questions where you currently lose, five where you already appear and five where a named competitor wins. Keep branded prompts in a separate tab so they do not flatter your category visibility.
2. Freeze the wording. Check the same questions across ChatGPT, Gemini and AI Overviews; log when a surface does not produce an applicable answer rather than quietly swapping in an easier prompt.
3. Run each prompt three times in fresh chats per check, twice a week for two weeks. Save each answer as its own observation. Week one is your baseline; make the defined edits after that week, then run the same checks in week two.

The repeats matter because a single answer can change before your page does. The frozen wording matters because changing the question changes the test. Neither step is glamorous. Neither is keeping a control group in an ad-copy test, until somebody wants to claim a win.

![Lab notebook with frozen prompt list beside three AI answer screens showing different brand mentions](https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/19189103-d00d-4356-accc-2d4e1206ef71.png)

### Controls: change one group, leave the other alone

Map the losing prompts to the pages you could reasonably edit. Split those pages into a group A to fix and a group B to leave untouched. Keep the prompts, engines, repeat count and logging method the same for both. The control is not a page you forgot about; it is the comparison that tells you whether both sets moved with the answers.

For group A, address the observed failure. Put a quotable answer block near the top, use clearer headings that match the buyer’s question, and make an existing stat or price easy to find and repeat accurately. If the answer states a false fact, fix the fact; if a competitor keeps getting the citation, [look at what it cites before writing more content](https://akii.com/blog/ai-monitoring-signal-to-action-system). Leave group B alone. Do not redesign the site while you are testing a quoting surface. Change ten things and you will not know which one mattered.

### Measurements: keep mentions separate from citations

Log what happened in every run, then compare averages rather than declaring each prompt a win or loss. I would keep four measures visible:

- **Appearance rate:** the share of runs in which your brand appears. Two appearances in three runs are about 67% for that prompt, not an unconditional win.
- **Mention share:** your brand’s mentions divided by all brand mentions in the fixed panel, counting a brand once per answer.
- **Position-weighted share:** [three points for a first mention, two for a second and one for a later mention](https://dev.to/pineprompt/how-to-measure-share-of-voice-in-ai-search-3o9l). This keeps a passing mention from looking like the lead recommendation.
- **Citation share:** the portion of answers that actually cite your domain, kept alongside the competitor domains they cite instead.

A model can name your brand while quoting somebody else. That is why I will not substitute mention rate for citation rate, or turn both into one reassuring dial. My sheet would have a row for each prompt, a record for each run and a weekly average for each group. The sheet’s job is to show the comparison, not win a design award.

### Read week two as a direction, not a victory lap

Look for separation: group A gains mentions and citations while group B stays flat on the same frozen panel. If both groups rise or fall together, the model may have shifted underneath you. If neither moves, check whether the pages are accessible and whether they offer anything short enough to quote before repeating the same rewrite. If the fixed group moves, you have a pattern worth continuing to test.

Two weeks give you a quick read, not a final claim. The [volatility guidance](https://dev.to/pineprompt/how-to-measure-share-of-voice-in-ai-search-3o9l) treats small weekly changes as noise and calls for a steady four-week change before naming a signal. So if group A is up 15 points on citations after week two and group B is flat, I would keep testing that edit pattern, not publish a case study. If they tie, I would stop making the same prose edit and inspect what the answers cite instead: perhaps YouTube, Reddit or a competitor directory. Either result changes the next job. That is what a control group is for.

## The verdict on tools: pay for the work, not the chart

You do not need a large subscription to start counting. An [open-source tracker](https://github.com/stefanmaritz/free-llm-tracker) queries five models through Vercel AI Gateway at about $0.025 per prompt across all five, with a first five-prompt report costing about $0.12. Its builder states the catch plainly: there is no Search Console for AI answers, so you have to ask and count; Google AI Overviews and AI Mode need SerpAPI scraping at two searches per query. A [Google Sheets tracker for repeated buyer prompts](https://groas.com/post/build-your-own-ai-visibility-tracker-in) is another way to keep the first pass small. Neither option fixes a losing page for you.

Stop expanding the measurement project when it tells you what to change. If the untouched group is flat and the edited pages keep gaining citations, your next choice is a [capacity choice, not a tracker debate](https://groas.com/post/diy-script-vs-visibility-tracker-vs-done): keep doing the edits yourself or have the monitoring and execution handled together. That is where I put groas ahead of a stand-alone dashboard. It keeps the prompt panel running, rewrites losing pages, logs changes and their reasons, and checks the frozen prompts again instead of handing over another chart.

My ruling stays narrow. Measure 20 prompts until you can see a lever. Then spend on pulling it, not on a prettier picture of the same 20 prompts.

## Frequently Asked Questions

### Should I track my brand in AI answers before fixing my pages?

Yes, but narrowly: measure 20 buyer prompts as a baseline, fix a defined set of pages, then re-measure against an untouched control group. A baseline is what tells you whether a rewrite actually worked, while keeping the measurement small avoids spending weeks tracking unstable answers.

### How do I decide which pages to rewrite for AI visibility?

Use measurement to triage. Google rankings alone will not show you when a page ranks but never gets cited in an AI answer, so check the buyer questions, whether your brand appears, whether your domain is cited, and who gets the space when you do not. Then spend your editing time where you actually lose.

### Why not just skip AI monitoring tools and fix pages directly?

Pure fix-first works if you have fewer than ten pages worth fixing, but a rewrite without a baseline and control group tells you little about why an answer changed. Cited answers can share only a third or less of hostnames between consecutive runs, so you need some comparison to know whether a citation gain followed your edit or just a model shuffle.

### How do I run a 20-prompt test for AI visibility?

Choose 20 buyer prompts: ten category questions where you currently lose, five where you already appear and five where a named competitor wins. Freeze the wording, run each prompt three times in fresh chats per check twice a week for two weeks, and use week one as the baseline before making your edits in week two.

### Do I need a control group when testing AI visibility changes?

Yes. Split the pages you could edit into a group A to fix and a group B to leave untouched, keeping the prompts, engines, repeat count and logging method the same for both. If both groups rise or fall together, the model may have shifted underneath you rather than your edit working.

### What metrics should I track for AI search visibility?

Track four measures: appearance rate (the share of runs where your brand appears), mention share (your brand's mentions divided by all brand mentions in the fixed panel), position-weighted share (three points for a first mention, two for a second, one for later), and citation share (the portion of answers that actually cite your domain). Keep mentions and citations separate, because a model can name your brand while quoting somebody else.

### How long before an AI visibility change counts as a real signal?

Two weeks give you a quick read, not a final claim. Volatility guidance treats small weekly changes as noise and calls for a steady four-week change before naming a signal. So if the fixed group is up after two weeks while the control stays flat, keep testing that edit pattern rather than publishing a case study.

### Do I need an expensive AI visibility monitoring tool to start?

No. An open-source tracker queries five models at about $0.025 per prompt, with a first five-prompt report costing about $0.12, though Google AI Overviews and AI Mode need SerpAPI scraping at two searches per query. A Google Sheets tracker for repeated buyer prompts is another small way to start. Neither option fixes a losing page for you.

## Pay For Results, Not For Hours

Businesses buy the outcome, agencies resell it, and groas answers for it either way.

[See If You Qualify](https://groas.typeform.com/to/xC1bQNUT)

## Structured data

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://graph.groas.com/entity/groas","name":"groas","url":"https://groas.com/","logo":{"@type":"ImageObject","url":"https://groas.com/icon-512.png","width":512,"height":512},"sameAs":["https://www.linkedin.com/company/groas/"]},{"@type":"WebSite","@id":"https://groas.com/#website","name":"groas","url":"https://groas.com/","inLanguage":"en-US","publisher":{"@id":"https://graph.groas.com/entity/groas"}},{"@type":"WebPage","@id":"https://groas.com/post/measure-first-or-fix-first-the-real-argu#webpage","url":"https://groas.com/post/measure-first-or-fix-first-the-real-argu","name":"Measure First or Fix First? The 20-Prompt Test for AI Visibility","description":"Should you track your brand in AI answers before fixing pages? Here is the strongest case for each side, followed by a 20-prompt test with a baseline and a control.","inLanguage":"en-US","isPartOf":{"@id":"https://groas.com/#website"},"about":{"@id":"https://graph.groas.com/entity/groas"},"primaryImageOfPage":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/4b923078-1b62-4926-8867-95c814fb3909.png"},"breadcrumb":{"@id":"https://groas.com/post/measure-first-or-fix-first-the-real-argu#breadcrumb"},"dateModified":"2026-10-16T05:30:26.682Z","mainEntity":{"@id":"https://groas.com/post/measure-first-or-fix-first-the-real-argu#article"}},{"@type":"BreadcrumbList","@id":"https://groas.com/post/measure-first-or-fix-first-the-real-argu#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://groas.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://groas.com/blog"},{"@type":"ListItem","position":3,"name":"Measure First or Fix First? The 20-Prompt Test for AI Visibility","item":"https://groas.com/post/measure-first-or-fix-first-the-real-argu"}]},{"@type":"BlogPosting","@id":"https://groas.com/post/measure-first-or-fix-first-the-real-argu#article","url":"https://groas.com/post/measure-first-or-fix-first-the-real-argu","mainEntityOfPage":{"@id":"https://groas.com/post/measure-first-or-fix-first-the-real-argu#webpage"},"isPartOf":{"@id":"https://groas.com/blog#blog"},"headline":"Measure First or Fix First? The 20-Prompt Test for AI Visibility","description":"Should you track your brand in AI answers before fixing pages? Here is the strongest case for each side, followed by a 20-prompt test with a baseline and a control.","datePublished":"2026-10-16T05:30:26.619Z","dateModified":"2026-10-16T05:30:26.682Z","image":{"@type":"ImageObject","url":"https://pub-87da24ecbbfc4c3bad6875f3aa013712.r2.dev/generated-images/4b923078-1b62-4926-8867-95c814fb3909.png"},"author":{"@id":"https://groas.com/author/alexander-perelman#person"},"publisher":{"@id":"https://graph.groas.com/entity/groas"},"keywords":"first, group, test, fix, pages, 20-prompt test, answer, before","wordCount":1942,"inLanguage":"en-US"},{"@type":"Person","@id":"https://groas.com/author/alexander-perelman#person","name":"Alexander Perelman","jobTitle":"Head Of Product @ groas","description":"Ex Goldman Sachs and Ex Stanford Computer Science","image":"https://groas.com/media/blog/4f17cac3a81acc224cc1b2cfab8f94bbc295a3089d13e8421125771ef8d4064f.jpg","worksFor":{"@id":"https://graph.groas.com/entity/groas"},"sameAs":["https://www.linkedin.com/in/alexander-433793253/"]},{"@type":"FAQPage","@id":"https://groas.com/post/measure-first-or-fix-first-the-real-argu#faq","mainEntity":[{"@type":"Question","name":"Should I track my brand in AI answers before fixing my pages?","acceptedAnswer":{"@type":"Answer","text":"Yes, but narrowly: measure 20 buyer prompts as a baseline, fix a defined set of pages, then re-measure against an untouched control group. A baseline is what tells you whether a rewrite actually worked, while keeping the measurement small avoids spending weeks tracking unstable answers."}},{"@type":"Question","name":"How do I decide which pages to rewrite for AI visibility?","acceptedAnswer":{"@type":"Answer","text":"Use measurement to triage. Google rankings alone will not show you when a page ranks but never gets cited in an AI answer, so check the buyer questions, whether your brand appears, whether your domain is cited, and who gets the space when you do not. Then spend your editing time where you actually lose."}},{"@type":"Question","name":"Why not just skip AI monitoring tools and fix pages directly?","acceptedAnswer":{"@type":"Answer","text":"Pure fix-first works if you have fewer than ten pages worth fixing, but a rewrite without a baseline and control group tells you little about why an answer changed. Cited answers can share only a third or less of hostnames between consecutive runs, so you need some comparison to know whether a citation gain followed your edit or just a model shuffle."}},{"@type":"Question","name":"How do I run a 20-prompt test for AI visibility?","acceptedAnswer":{"@type":"Answer","text":"Choose 20 buyer prompts: ten category questions where you currently lose, five where you already appear and five where a named competitor wins. Freeze the wording, run each prompt three times in fresh chats per check twice a week for two weeks, and use week one as the baseline before making your edits in week two."}},{"@type":"Question","name":"Do I need a control group when testing AI visibility changes?","acceptedAnswer":{"@type":"Answer","text":"Yes. Split the pages you could edit into a group A to fix and a group B to leave untouched, keeping the prompts, engines, repeat count and logging method the same for both. If both groups rise or fall together, the model may have shifted underneath you rather than your edit working."}},{"@type":"Question","name":"What metrics should I track for AI search visibility?","acceptedAnswer":{"@type":"Answer","text":"Track four measures: appearance rate (the share of runs where your brand appears), mention share (your brand's mentions divided by all brand mentions in the fixed panel), position-weighted share (three points for a first mention, two for a second, one for later), and citation share (the portion of answers that actually cite your domain). Keep mentions and citations separate, because a model can name your brand while quoting somebody else."}},{"@type":"Question","name":"How long before an AI visibility change counts as a real signal?","acceptedAnswer":{"@type":"Answer","text":"Two weeks give you a quick read, not a final claim. Volatility guidance treats small weekly changes as noise and calls for a steady four-week change before naming a signal. So if the fixed group is up after two weeks while the control stays flat, keep testing that edit pattern rather than publishing a case study."}},{"@type":"Question","name":"Do I need an expensive AI visibility monitoring tool to start?","acceptedAnswer":{"@type":"Answer","text":"No. An open-source tracker queries five models at about $0.025 per prompt, with a first five-prompt report costing about $0.12, though Google AI Overviews and AI Mode need SerpAPI scraping at two searches per query. A Google Sheets tracker for repeated buyer prompts is another small way to start. Neither option fixes a losing page for you."}}]}]}
```