How we measure whether AI recommends you
No black box. This is exactly what happens when you run a check, how the score works, and where its limits are.
1. We work out what you sell, in buyers’ words
We read your public homepage (its title and description) and describe your product as the category a buyer would type, for example “LinkedIn post scheduler” rather than your brand name. We also note who it’s for. Products in the same category share one set of questions, so they’re compared fairly.
2. We ask the questions buyers ask
For each category we ask five questions in the shapes buyers use: what’s the best, recommend one for a specific audience, the top tools this year, which to consider and why, and alternatives to the most popular option.
Each question goes to each assistant we track (currently ChatGPT and Gemini) with live web search switched on, in a fresh session with no history. That matters: answers built from live search reflect what the web says today, not what a model memorised during training.
3. We record what each answer says
For every answer we store the full text, the products it names in the order they first appear, and the web pages it cites as sources. A separate model reads each answer to list the products; we then match them to your product by name and domain, and double-check the raw text in case the list missed a mention.
4. How the score works
Each answer earns points depending on where you appear:
- Named first: 1.0 point. Second: 0.8. Third: 0.67. Each place lower is worth a little less (1 ÷ (1 + 0.25 × (position − 1))).
- Your own site cited as a source: +0.15, up to a maximum of 1.0 per answer.
- Not named: 0.
Your score is the average across all answers, from 0 to 100. A score of 100 means you were named first in every answer; 0 means you weren’t named at all.
5. What the results can’t tell you
- Answers vary. Ask the same question twice and the answer can change. That’s why we score the pattern across many answers, and why a weekly re-check is more reliable than a single check.
- Real users see personalised answers. Memory, location and chat history change what a specific person sees. We measure the neutral, logged-out view.
- We use the assistants’ developer access. It’s the same models with their own web search, but the consumer apps can differ slightly in model version and settings.
- Visibility isn’t revenue. Being named more often should bring more of the right buyers, but we don’t measure your traffic or sales.
6. The leaderboard
The public Shortlist Index ranks startups whose revenue is verified by TrustMRR, using the same questions and scoring. It’s refreshed monthly. If you’re listed and think we’ve described your category wrongly, email hello@shortlistindex.com and we’ll correct it.