The same small coffee shop, nine times. Each version carries exactly one defect taken from WCAG 2.2 or Nielsen's usability heuristics — and one carries none at all. Nothing in the prompt mentions any of them.
So you do not have to take our word for it: you know what was planted, and you can read what the flock reported.
found 3 · missed 1 · nobody struggled 4 · 15 people per run
Nothing. This is the control — the same shop with no defect at all.
the only route from the shop to a product is an image link with no alt text, no aria-label and no accompanying text link
every product links through identical 'Read more' text; the names are not links
adding to the basket works but produces no confirmation, no count change and no navigation
the postcode format is enforced but never stated until it has been failed
the phone field rejects spaces and says only 'Invalid entry.' — no field named, no fix offered
checkout removes the navigation and offers no way back to the basket
checkout requires a reference code shown two pages earlier, with no reminder and no link back
the same container is called Basket, Cart and Bag on different pages and controls
Why fifteen people, and why that matters. A run finds a place several people independently struggle at. Fifteen will surface something that trips one person in three; it will not surface something that trips one in fifty. Population size is the resolution of the picture, not a vanity number.
Four of the eight defects changed nothing measurable at this size, and are marked as such rather than counted as failures — a defect that violates a standard is not automatically a defect that stops anybody buying coffee. One did stop people, badly, and the run still put the cluster on the wrong page. That one is the real miss, and it is published too.
A betting site, and the journey nobody demos: somebody trying to stop. The deposit limit and self-exclusion controls were renamed “RG Tools” — present, reachable, and labelled in the operator's vocabulary instead of the customer's.
Nothing. The deposit limit and self-exclusion links are named in the words a person would use.
The same controls, reachable, but named in the operator's vocabulary rather than the customer's.
Every one of them decoded the jargon. In their own words: “I can see 'RG Tools' in the menu, which likely stands for Responsible Gambling Tools — that's where betting limits would be.” The cost of the defect was one extra click: 5.6 steps against 6.6.
A language model trained on the internet knows what RG Tools means in a betting context. Somebody who has been betting more than they meant to, and has decided to stop, may well not. Synthetic people are better readers than real ones — so this method finds mechanical friction, and is blind to friction made of vocabulary.
Fifteen people, one fixture, one piece of jargon. That is a signal, not a proof. It is published because a tool that will not tell you where it stops working is not a measurement instrument.