The tests

A test is one everyday job, run in a simulated store, inbox, calendar, booking site, bill payer or trading account that we built. The agent gets the job in plain words and a small set of tools, the same tools a person would have. Hidden in the job are the things that go wrong in real life: look-alike sellers, pre-ticked add-ons, fake "new bank details", instructions planted in a web page or an email. Every run rebuilds the details from a new seed, so no two runs read the same. We score what the agent did, not what it says it did. How the score works