The short version
- 4 of 10 stores lost at least one browser agent before the order total. Two stopped both.
- The two browser agents disagreed on two stores. On Gymshark and Kylie Cosmetics the code-reading agent got through and the screen-reading one didn't. Which kind of AI shopper you are losing matters, because the fixes are different.
- No store lost an agent at checkout itself. The sales were lost earlier: at the front door, at search, and in the cart drawer.
- Every protocol checkout was handed back to the buyer. All ten stores answered Shopify's agent protocol with a checkout that needed the buyer to finish on the store's own page.
Every store, every shopper
Agent A reads the page's code (its accessibility tree, the same structure a screen reader uses). Agent B looks at screenshots. The protocol agent never loads a page and shops through Shopify's agent protocol (UCP), the way Google's AI Mode does.
| Store | Agent A, code | Agent B, screen | Protocol, total | What happened |
|---|---|---|---|---|
| Allbirds | Stuck | Stuck | Handed back | The cart drawer's Checkout buttons aren't in the page's code, so Agent A had nothing to press. Search came back blank for Agent B. |
| Bombas | Stuck | Stuck | Handed back | Turned away at the door: a bot checkpoint blocks cloud browsers before the store loads. |
| Gymshark | Got through | Stuck | Handed back | Search shows grey placeholders to cloud browsers. Agent A browsed to the leggings instead; Agent B gave up. |
| Kylie Cosmetics | Got through | Stuck | Handed back | Product grids stayed grey for Agent B. Agent A got through using the page's built-in tools for AI agents. |
| SKIMS | Got through | Got through | Handed back | Search never loaded from the cloud, so both agents found the T-shirt by browsing. |
| Chubbies | Got through | Got through | Handed back | A recommendations popup covered the page, and checkout asked for a login before offering guest checkout. |
| Rothy's | Got through | Got through | Handed back | A text sign-up popup got in the way of the first clicks. |
| Ruggable | Got through | Got through | Handed back | The size buttons stay hidden until the size menu is opened. |
| Brooklinen | Got through | Got through | Handed back | Address autocomplete swapped New York for a town in Kansas; Agent B noticed and fixed it. |
| Glossier | Got through | Got through | Handed back | Clean run for both browser agents. |
What stopped them
A bot wall at the front door
Bombas runs a bot checkpoint that turned both browser agents away before the store loaded. From a home connection the store opens normally. Hosted AI shoppers browse from cloud servers too, so a checkpoint tuned this tightly may be turning away real agents along with scrapers. In our earlier five-run test on 10 October it blocked the agents every time.
A Checkout button people can see and agents can't
On Allbirds, the cart drawer shows a large black Checkout button. In the page's accessibility tree there's no button there at all, only the express wallet buttons. The code-reading agent looked for a way to check out, found Shop Pay, PayPal and Google Pay, and gave up. In five runs on 10 October it failed at this exact spot every time, while the screen-reading agent got through all five. A screen-reader user would hit the same wall. The fix is a real <button> with a name, about an afternoon of a developer's time.
Search that never loads from the cloud
Gymshark's search showed grey placeholder tiles to our cloud browser, and Kylie Cosmetics' product grids stayed grey for the screen-reading agent. SKIMS' search never loaded at all. When the products are only visible to the code, the code-reading agent can still browse its way to them, which is how Agent A passed on Gymshark and Kylie. The screen-reading agent sees what a person would see, which here was nothing. A shopper at home may never notice. Hosted agents do.
Popups, login walls and hidden size buttons
These slowed agents down without stopping them. Chubbies opened a recommendations popup over the page, and its checkout asked for a login before offering guest checkout. Rothy's opened a text-message sign-up on the first clicks. Ruggable hides its size buttons until the size menu is opened. On Brooklinen, address autocomplete swapped New York for a town in Kansas; the screen-reading agent noticed and corrected it. Each is a coin flip a less careful agent might lose.
Every protocol checkout was handed back
The protocol agent searched each catalogue, chose the variant and asked for a checkout. All ten stores returned a shipping cost and a total, so the protocol route works for getting a price. None let the agent finish: every checkout came back with the status requires_escalation, Shopify's way of saying something in checkout needs the buyer, along with a continue_url to the store's own checkout page.
That isn't automatically bad. What matters is where the buyer lands. In a follow-up check, nine of the stores' hand-back links opened their checkout already filled in with the buyer's email, address, shipping method and total, so the buyer only has to pay. On Brooklinen the protocol search didn't find the product that day, so there was nothing to check. A hand-back that lands on an empty checkout, or not on a checkout at all, loses the buyer's details at the last step. One known reason a checkout needs the buyer is a checkout app or extension that asks them for something, like an age check or a referral code.
One run is a sample
Agents don't always do the same thing twice. On 10 October we ran the browser agents five times per store, and the five-run results held up some findings and complicated others:
- Allbirds: the code-reading agent reached checkout 0 of 5 times, stuck at the cart drawer each time; the screen-reading agent got there 5 of 5. On 13 October the screen-reading agent was stopped earlier, by a blank search, before it ever reached the drawer.
- Bombas: the bot checkpoint blocked the agent in 5 of 5 runs.
- Shipping cost: asked to find the standard shipping price before checkout, agents managed it 0 of 5 times on Allbirds, 0 of 5 on Chubbies and 1 of 5 on SKIMS.
- Six of the nine stores we could test reached checkout in all five runs.
This is why the free test calls itself a verdict on one run, and the full report runs each agent five times before calling something a wall.
How we ran it
- The job: the same one the free test gives. Find a specific product from the store's own catalogue, choose the size or colour, add it to the cart, check out as a guest and reach the order total.
- The agents: AI models driving a real Chromium browser, one through the accessibility tree and one through screenshots, plus a protocol agent calling the store's Shopify agent protocol endpoint.
- Where from: cloud servers, the same kind of network hosted AI shoppers use. Results that depend on that, like bot walls and search that won't load, are flagged as such.
- Test details: obviously fake ([email protected] and a public landmark's address). No agent entered payment details or placed an order, and every protocol checkout was cancelled.
- Runs: one per agent per store on 13 October, every browser run recorded on video. The five-run figures come from 10 October.
Stores change every week. These results describe what happened on those dates, not what the stores do today.
What to check on your own store
- Open your cart drawer with a screen reader, or inspect it, and make sure Checkout is a real button with a name.
- Ask whoever runs your bot protection whether it challenges traffic from cloud servers, and what that does to AI agents.
- Load your search and collection pages from a cloud browser, not just at home.
- Show the shipping cost before payment, and put shipping and returns where agents read them.
- Review checkout apps that ask the buyer for input, and see where a hand-back lands.
The agent experience (AX) guide explains each of these, or run the free test and let three agents find out for you.