Do AI assistants agree on what to recommend?

This is the finding that governs every other number in the category. A language model does not look up an answer, it generates one, so two askings of one question are two samples of something that moves rather than two readings of a fixed fact. The measurement here is direct: one question, one assistant, asked more than once inside the same measurement, comparing the set of companies named each time.

Asking within a single measurement is what makes it a fair test. Pooling answers from measurements taken months apart would count a market that genuinely changed as an assistant failing to repeat itself, which is a different and much weaker claim.

The practical reading is that a single pass at a question set tells you what one assistant said once, and the useful question about any visibility score is how far it would move if the same measurement were taken again.

For each question we took the company each assistant named most often, then asked whether the assistants had landed on the same one. The average question produced 3.34 different top picks between the assistants that answered it.

So there is no single ranking to be at the top of. There are five, they move, and the company a buyer meets depends on which assistant that buyer happens to open.

Three companies in four appeared in front of one assistant and none of the others. Checking one assistant is a real reading of that assistant and a poor proxy for the rest.

How many companies each assistant will name at all is closer than the disagreement above suggests, at a spread of 1.4 between the widest and the narrowest. ChatGPT named 880 distinct companies across these questions and Perplexity named 645. So they name similar numbers of companies and largely different ones, which is the more useful way round to state it.

• ChatGPT: 880 distinct companies named
• Claude: 857 distinct companies named
• Gemini: 840 distinct companies named
• Grok: 788 distinct companies named
• Perplexity: 645 distinct companies named

There is no incumbent holding these answers. The most-named company appeared in about one naming in thirty, the top three together in about one in twelve, and 1,588 companies surfaced a single time across the whole study. That long tail is what a generated answer does: it reaches for something plausible, and plausible is a very wide set.

561 of the 3,116 answers, 18%, named no company at all. Those answers explained how to approach the problem instead of recommending anybody, which is worth knowing if you are waiting to be recommended on a question that is not being answered with names.

This is the practical use of the study, and it applies to any tool in this category including ours. Ask how many prompts a score is built from, ask how many samples per assistant, and ask what the same measurement returned last time. Those three answers turn a number into a measurement.

It is also why repeat sampling is the design rather than a feature. Every measurement here asked each assistant each question more than once, which is what made the 21.5% figure computable at all. A measurement that samples once cannot report its own stability, because it has nothing to compare with.

Three limits are worth stating plainly, because a study that hides them is a brochure. The panel is 4 companies, which clears our reporting floor and is still small. The questions are the ones those businesses chose to measure, drafted for their own categories rather than sampled from search demand. And these systems change, so a figure here is a reading of a period rather than a constant.

The base is also narrower than the raw measurements. Answers reach these counts once the labelling stage has read them, and unlabelled answers are excluded rather than counted as naming nobody. That distinction decides the headline: an unlabelled answer has no list recorded against it, so counting it as an answer that named nobody would make a repeat asking look like a changed answer.

One more thing is true and belongs here. Our own product was named in none of the 357 answers measured for it. The study is the market we measured, and it did not include us.