WebsiteBenchWebsiteBench

Analysis

Why even the best agent passes only about one test in four

1Stops before the result 1Never sees the confirmation
1UnderexplorationAgents start nearly every workflow before coding, but finish few
2Right labels, wrong images 2?tab=coming-soon ignored 2Booking routes return 404 2Review fails after restart
2Failure to implement observed behaviorWhat the agent saw rarely reaches its code
3Stops at 1.0 h of 16 with 0% 3Declares done, bugs remain
3Early stoppingMost runs end well before the 16-hour limit
2,234.8 agent turns per task, on average, for the best model and it still passes only 26.2% of hidden tests

Three ways the rebuilds go wrong

1Underexploration

Agents visit the pages but seldom follow a workflow to its end

A human trace raised scores.

A checkout workflow, and where agents stop

  1. Browse products
  2. Add to cart
  3. Check out
  4. Order confirmation

2Failure to implement observed behavior

What agents observe rarely reaches their code

Most tool calls explored the reference or checked the rebuild. Few edited code.

Two traced Grok 4.6 runs: what the agent saw, and what it built

AMC Theatres

SawOpened /movies?tab=coming-soon and captured the Coming Soon list.

BuiltClicking the tab works, but the server ignores the tab query, so the URL opens the default list.

PetSmart

SawA homepage screenshot with matching artwork on the Fish, Bird and Small Pet cards.

BuiltThe cards keep their labels, but all three show dog artwork.

3Early stopping

Agents stop with most of the clock left

With Claude Code’s /goal hook rejecting the early exit, Kimi K3 on Lowe’s went from 0% to 31.3%.

Kimi K3 on Lowe’s, without and with the /goal hook

Tokens
10.6M to 87.9M

Human trace

Grok 4.6paired runs16-hour limit

Each website is rebuilt twice, with the same model, agent and time limit
RunWhat the agent has
Explores aloneThe reference website, through the browser.
With a human traceThe same browser access, plus a read-only log of a developer’s actions on the reference.