Prompt Tracking

The vendors’ own reports show little of the answer itself. Search Console counts the clicks and impressions of AI Overviews and AI Mode within the Web search type, together with the rest of Search^[1], and treats a follow-up question in AI Mode as a new query with data of its own^[2]: the numbers arrive without the answer that earned them.

Bing shows more than most. Its AI Performance report counts how often a site’s pages are cited across Copilot, Bing’s AI summaries and some partner integrations, with the grounding queries behind them, but says the citation count does not indicate placement or presentation within an answer^[3]. Which brand an answer recommends, and in what words, is seen only by asking.

Answers vary, so one reading is not a measurement. The same prompt can draw a different plan of searches and different sources from one run to the next, as described under query fan-out. Tracking fixes what can be fixed, the prompts, the surfaces, the schedule and the way each is asked, so that the change left is in the answers.

From our practice: each question is asked in a new chat, two or three times, because answers vary.

A fixed, stated set is what gives a rate its meaning. QueryRev’s study works the same way on another object: it starts from a dated list of 10,000 places, leaves out 2 that were not valid domains and 25 domains of companies that build language models, and says so beside its figures^[4]. A prompt set needs the same care: which prompts, why these, and what was left out.

Per prompt, surface and run, record the answer text, whether the brand is named, each cited URL in order, the competitors named and anything the answer gets wrong. From those rows come the AI citation rate, the mention rate and share of voice against competitors.

Group the prompts by topic and intent, and keep buying prompts apart from definitions, so a gain on easy questions cannot hide a loss on the ones that sell.

Report each rate with its denominator and its spread. Forty prompts on four surfaces over three runs are 480 answers, and a move of one or two of them can be noise.

Sample data: an invented site, to show the method.

A project-management app tracks 60 prompts on four AI surfaces, each asked three times in a fresh session every Monday: 720 answers a week. Fifteen of the prompts are about pricing, and for four weeks the app is named in about one in twenty of their answers.

In the fifth week the pricing prompts move. Reading the answers shows why: a review site has published a comparison table that names the app’s free plan, and the answers now cite it. The rise is real, and it was earned on someone else’s page.