How to track brand mentions in AI assistants, and tell a real change from noise

Why the answers move, and what one look can still tell you, is covered in how to find out what ChatGPT says about you. For tracking, it means each point on the line has to be a rate. A series you can keep by hand looks like this: a fixed list of buyer prompts, each asked several times of each assistant you follow, every answer logged as its own row. The figure for the period is the share of rows that named you and, separately, the share that recommended you, which are two parts of what AI visibility covers.

Keep each assistant as its own series. Of the companies named anywhere in that study, 75.7% were named by exactly one of the assistants, so what one assistant says is not a reliable guide to what the others say.

The fields for a single look are on how to find out what ChatGPT says about you. Three additions matter once looks are compared:

• Position. The recommendation, one of the first three names, or further down. A slide from first to third leaves a count of mentions exactly where it was.
• Steered away from. An answer that names you and then advises the buyer against you still counts as a mention in a simple tally. Keep it apart from the first day, for the reasons in mentioned is not recommended.
• The prompt, the date and the model. The exact wording, the day, and the model name where the product shows one. Without them, a change in how you asked and a change in the market look the same on the line.

Delphi keeps every answer word for word, with the exact prompt it sent and the model that replied. For each answer it reads, it records whether the brand was the top recommendation, named, named but advised against, or absent, its position, how it was described, the rivals named with their own positions, and the sources cited where the assistant returned them. The score is built only from the answers it has read.

The repeat figure above is the reason. A difference that shows up between two askings in one sitting cannot be read as a movement when it shows up between Monday and Tuesday, so a few prompts, each asked once and read as that day's result, will show changes on many days whether or not anything has moved.

The real choice is how to spend the askings. One large reading a month, with every prompt asked several times of each assistant, costs one sitting and carries its own spread, but it cannot say when within the month something moved. The same askings spread across the month and pooled into one monthly rate are just as valid, and can be split by week to show roughly when a change began, with a wider spread on each week. Either way, keep the prompts fixed, and take an extra reading after anything that could move an answer, such as a launch, a new comparison page or a model release.

Measure the noise before you measure the change. Take your baseline more than once, a few days apart, with nothing changed on your side: the same prompts, each asked several times. The spread across those readings is a first estimate of the noise, and it firms up as readings accumulate. A move from one month to the next that stays inside it is not a result, and reporting it as one sends a team to explain a change that did not happen.

Compare like with like: the same prompts, asked the same way, of the same assistants. A rate built from many askings and a rate built from a few are not equally reliable, even when they print the same percentage. The same holds when checking whether work on ChatGPT landed: the reading before the work and the reading after it have to be taken the same way.

Delphi uses a different spread for the same purpose: the band published with its score is how far the assistants disagreed with each other in that run, and its dashboard and history page report a movement smaller than the band as no confirmed change. How that band is computed is on how many questions a measurement needs.

• The prompts. A reworded prompt is a new question. Keep the old wording, or start a new series and do not join it to the old one.
• The session and the place. Ask from a clean session, from the same place, every time. OpenAI documents that, where Reference chat history is switched on, ChatGPT can use information from past conversations to personalise later responses, that a temporary chat set to Unpersonalized does not use existing memories, and that ChatGPT may use an approximate location from your IP address for local results. In ChatGPT, a tracker run from one signed-in account, or from wherever somebody happens to be that day, may be partly measuring that account and that place.
• The scoring rule. If what counts as a recommendation changes halfway through, the line changes with it. Write the rule down on the first day.
• The model. OpenAI's release notes record changes to the models in the ChatGPT app, including updates to a model that keeps its name and the retirement of older ones, and say which of them also apply to the API. A movement that lines up with one may come from the assistant rather than from last month's work, so check which model answered before crediting or blaming the work.

Delphi holds these still or records them. Each answer is requested as a single new question with no conversation before it. Editing a question set creates a new version rather than rewriting the old one. Every answer is stored with the exact model that produced it, and the download for each run carries that model beside every answer, so for a Delphi reading that stored model is the thing to check rather than an app's release notes. And every run records which version of Delphi's own reading rules labelled it: where the questions or those rules changed between two runs, the trend view says so above the chart, so a step made by the method can be told apart from one made by the market.

The runs each level includes are on the pricing page. Once a brand has its first measurement, it is measured again automatically about once a month, when its last run is at least four weeks old and the month's allowance has a run left, unless automatic runs are switched off for that brand. The automatic run is one a month; where a level includes more, the others are started by hand. A month that needs one more reading, after a launch for example, can have one for $49.

What goes into each run is the point. A question set holds up to 40 questions, every question is planned for every assistant more than once, and how far the assistants disagree becomes the band published with the score. A run that reaches fewer assistants than planned says so rather than filling the gap.