CanaryFlock synthetic product evaluation

Read this first. These are simulated people, not your customers. Every number below describes what happened inside a simulation, and none of it is a forecast of what real users will do.

What the run can tell you is where people struggled and who struggled there — and each of those is backed by a recording you can check. Synthetic people are more patient, more literate and more single-minded than real ones, so treat the places as findings and the percentages as description.

what we ran

http://127.0.0.1:61618/a/ambiguous-links/

Buy a bag of decaffeinated coffee.

Planted, and hidden from the system: every product links through identical 'Read more' text; the names are not links
Taken from WCAG 2.2 SC 2.4.4 Link Purpose (In Context) (Level A). Nothing in the prompt mentions it — read the findings below and judge whether the flock landed on it.

15people
11finished
0gave up
4ran out of steps

Where people struggled

1

127.0.0.1:61618/a/ambiguous-links/checkout

click e6

3 of 15 people (20%) struggled here

  • frustration jumped +0.26 (×3)
Hit hardestage: 16-24 (33% vs 13% of cohort)
In their wordsThe system rejected my postcode format. It says it should look like SW1A 1AA, which is a UK format. I entered a Dutch format, but the system wants UK format. Let me click on the Postcode field again and enter a proper UK-style postcode instead.The page is telling me the postcode format is wrong—it needs to look like SW1A 1AA with spaces. I used a Dutch format (1234AB) but it's rejecting that. Let me click on the postcode field and enter the correct format it's asking for.The postcode format is wrong — it needs to be like SW1A 1AA, not just numbers. I'll click the postcode field again and enter a properly formatted postcode so I can finally place this order.
2

127.0.0.1:61618/a/ambiguous-links/checkout

type e5

2 of 15 people (13%) struggled here

  • action failed: type needs a value — say what to type (×2)
  • frustration jumped +0.15
Hit hardesttech literacy: medium (50% vs 7% of cohort)age: 35-44 (50% vs 13% of cohort)age: 45-54 (50% vs 27% of cohort)
In their wordsI'm at the checkout page and have already filled in name and email. The phone field is now required, and I need to enter a phone number to proceed with the order. I'll enter a standard format phone number.I need to fill in the Phone field. I'm at the checkout form and this is a required field. Let me enter a phone number to continue.

Who it happened to

5 groups had fewer than 3 people and are left out: a rate from one or two people is not a rate.

age
25-34 n=4 75%
45-54 n=4 50%
55-64 n=3 100%
income
low n=3 100%
middle n=10 60%
price sensitivity
high n=4 50%
low n=7 71%
medium n=4 100%

Who we sent

16-24: 2 · 25-34: 4 · 35-44: 2 · 45-54: 4 · 55-64: 3
digital skill — high: 14 · medium: 1
11 of 15 on a phone

Ages and digital skill are drawn from Eurostat — the ICT usage survey and the population register — rather than invented. How those people then behave is a model, and that part is not calibrated against real users yet.