The 31 questions nobody answered

On 28 July 2026, David McSweeney published a list of 31 questions and addressed them to the companies selling AI visibility scores. Not a takedown, not a ranking, not a comparison table. A list of questions about method, of the kind you would expect any buyer to ask and any vendor to have ready.

By the following day, as far as we could establish, two companies had answered in public: Ternith and Spyglasses.

That is the story. An entire category was asked to explain how its numbers are produced, and the overwhelming majority said nothing.

Why silence is the interesting part

It is tempting to read a non-answer as an admission. It usually is not. Most of the silence is ordinary: nobody owns the question internally, legal wants to review it, the person who could answer it is shipping something else, and a public methodology document is a commitment you cannot quietly walk back.

But that is the point. A score is a claim about the world. If a company cannot afford to explain how a number is made, the number is not doing what the buyer thinks it is doing. It is doing marketing.

There is also a commercial reason the questions land hard. A visibility score is bought precisely because the buyer cannot check it themselves. Nobody buys a thermometer they can verify by touch. So the entire value of the product rests on trust in the method, and the method is the one thing the category has been reluctant to describe.

Why the questions are fair

You can tell a fair question from a hostile one by whether a good answer exists.

Every question on that list has one. Some of the good answers are unflattering — “we do not know”, “our sample is too small to say”, “that number moves on its own by more than we would like” — but they exist, and they are sayable. None of the questions demand a trade secret. They ask what is measured, from where, how often, with what variance, and what would count as being wrong.

That is the standard applied to any other measurement product a business buys:

• An analytics vendor tells you what a session is and when it expires.
• A survey firm tells you the sample, the field dates and the margin of error.
• A rank tracker tells you the location, the device and the personalisation settings behind the number.
• An accountant tells you which standard the accounts were prepared under.

AI visibility is the only measurement market we know of where “we have a proprietary algorithm” is still offered as a complete answer. It is not an answer. It is a refusal with better manners.

The three we think are sharpest

The full list is worth reading in the original. These three are the ones that decide whether everything else on the list matters. We are paraphrasing rather than quoting, because the wording is the author’s.

1. Who chooses the prompts, and does that choice determine the result?

An AI visibility score is a measurement of answers to questions. Which questions? Choose them and you have very largely chosen the outcome.

Ask an assistant a narrow, brand-shaped question and most brands look visible. Ask a broad, category-shaped question and most brands vanish. Both are real measurements. They are measurements of different things, and only one of them resembles what a customer actually types.

So the honest disclosures here are: who wrote the prompt set, whether the client can see it, whether it changes between runs, and whether the same prompt set is used for the competitor a client is being compared against. A score built on a prompt set the buyer never sees is not a measurement they can act on. It is a number they have to believe.

2. Has the metric been validated against something genuinely external?

This is the question the whole category struggles with, and it is not a gotcha.

Validation means comparing your number against evidence produced by something that is not you. Running your own system twice and observing that it agrees with itself is reliability, not validity. It tells you the instrument is stable. It tells you nothing about whether it measures the thing on the label.

The external evidence that would count is unglamorous and hard to get: server logs showing assistant-referred traffic, conversions attributable to an assistant recommendation, brand-tracking surveys, a client’s own inbound record. Any of those can disagree with a visibility score, which is exactly why they are worth checking against.

If nobody has ever tried the comparison, the score is an internally consistent artefact of the system that produced it.

3. What result would make you withdraw the metric?

The falsification question, and the one that separates a measurement from a story.

Every real metric has a failure condition its owner can describe in advance. If assistant-referred traffic moved in the opposite direction to the score across a large enough client base and a long enough window, that would be a problem. If the score changed materially on re-runs with no change to the underlying business, that would be a problem. If two of a vendor’s own clients in the same category with the same visibility got wildly different commercial outcomes, that would be a problem.

A vendor who cannot name any such condition has told you something important: there is no observation, in principle, that would count against their number. That is not a strength. It is the definition of an unfalsifiable claim, and it means the score can never be wrong — which also means it can never be right.

What we are doing about it

We sell an AI visibility score, so the questions are addressed to us as much as to anyone.

We are publishing our answers to all 31, on our own site, under our own name, including the ones where the honest answer is that we do not know yet and the ones where the honest answer makes our product look less impressive than the marketing page does.

We think that is the minimum. A number sold to a business that cannot independently check it should come with the method attached, the limits stated, and a description of what would make us withdraw it.

If our answers are wrong, we would rather be told. That is the other thing a published method buys you.

What to do if you are the buyer

You do not need to read all 31 to protect yourself. Three questions, asked before you sign, will tell you most of what you need to know:

• Show me the exact prompts behind my score, and tell me who wrote them.
• Show me one occasion where you compared this score against evidence from outside your own system.
• Tell me what result would make you withdraw the number.

A vendor who answers all three cheerfully is worth talking to, whatever their score says. A vendor who cannot answer any of them is selling you a feeling.