Offline reference accepted Agent experiment not started

Worked example 01 · shopping commerce

Amazon Shopping

A shopper discovers, configures, purchases, receives, and can recover or return an evidence-backed product in a bounded local Amazon shopping simulation. The freeze covers anonymous Singapore/USD source evidence plus local account journeys; it explicitly denies whole-marketplace or whole-site fidelity.

Compare screenshots Replay journeys

Amazon Shopping standard offline reference website — Standard offline reference serverDataset input

15Route familiesDiscovery through aftercare

79Interaction statesSuccess, failure, recovery

16Frozen checkpointsDesktop, tablet, mobile

454Local runtime assetsZero required remote calls

Stateful benchmark surface

The shopping mainline, end to end

Not a static homepage clone

⌕

DiscoverHome · Search · Deals

→

▣

DecidePDP · Variants · Compare

→

＋

CommitCart · Identity · Checkout

→

✓

FulfilOrder · Delivery · Review

→

↺

RecoverRetry · Return · Refund

Dataset calibration evidence

Original website evidence ↔ standard offline reference

These pairs validate the benchmark input. They are not an Agent score and will never populate the model leaderboard.

2 current-direct pairs

Checkpoint

acceptedEvidence: current-directComparison: source → offline reference

Original website evidence Original Amazon evidence at home.desktop.loaded

Standard offline reference Offline Amazon reference at home.desktop.loaded

Absolute visual difference Visual difference map at home.desktop.loaded

pixel similarity84.25%

acceptance threshold80.0%

Role in benchmarkInput calibration

Inspectable surface

Route & state explorer

15 route families

p0home/current direct

Discovery · fixture validated

loadedaccount hoverdepartment drawerautocompletenarrow native scroll

p0search/s and /s/ref=nb_sb_nosscurrent direct

Discovery And Decision · fixture validated

resultsfilteredsortedno resultsbounded card offer

p1departments/books, /home-kitchen, /toys, /computers, /beautycurrent direct

Cross Category Discovery · fixture validated

department card griddepartment filterbrowse only cardquoted card

p1deals/gp/goldbox/current direct

Promotion Discovery · fixture validated

default cardsdepartment filterprice filterdiscount filterno results

p0pdp/dp/<ASIN> and /<slug>/dp/<ASIN>current partial

Decision And Configuration · local tested

rich defaultrich valid variantdependency disabledgallery dialogsparse evidence boundarybuy now

p0cart/gp/cart/view.html and cart mutation endpointsinferred

Commit Staging · local tested

emptyactive linessibling variantsquantity updatesaved for latermerged after authrecommendations

p0auth/ap/signin, /ap/register, /ap/cvf/verify, /ap/forgotpasswordcurrent partial

Identity And Recovery · local tested

unknown email to registerpasswordregisterverify emailinvalid codepassword resetsigned insigned out

p0checkout/gp/buy/* and /gp/buy/place-orderinferred

Commit · local tested

addressdeliverypayment approved cardpayment declined cardpayment approved bankstale reconcileplaced

p0orders/gp/css/order-history and /gp/your-account/*inferred

Aftercare And Recovery · local tested

placedpreparingshippeddeliveredcancelledreturn requestedreturn receivedrefundedemail retry

p1reviews/product-reviews/<ASIN>current partial

Decision And Aftercare · local tested

source aggregateexplicit zerosource unavailablelocal create updatehelpfulverified purchase

p1compare/gp/compare and compare mutation endpointsinferred

Decision · local tested

emptysingle guidancesame family tablesibling variantsfamily rejectedlimit rejected

p1lists/hz/wishlist/*current partial

Saved Intent · local tested

public introaccount listcreaterenamemove to cart

p1shopping-adjacent/gift-cards/*, /sell/*, /registry/*, /gp/help/*inferred

Supporting Shopping · scope decision

gift card previewsell draftregistry create search detailcustomer service

p2prime-video-placeholder/Amazon-Video/*unavailable

Orthogonal · scope decision

explicit static placeholder

omitexternal-systemsreal SMTP, payments, fulfillment, telemetry, ads, and third-party seller systemsunavailable

External Boundary · scope decision

safe local adapter or omitted

Dynamic interaction evidence

Journey replay

Click any step

shopping.success

anonymous shopper becoming a verified local account

1 / 9

This is a structured replay of the frozen interaction contract. Per-agent browser traces and before/after frames will populate the same space after experiments begin.

Reference server acceptance

Calibration passed before any Agent is measured

ACCEPTED

Visual checks26 / 262 direct screenshot pairs

Browser journeys22 / 2241 pages visited

Asset closure454 / 4540 nonlocal requests

Local test suite330 / 330Frozen semantics

Assets gate · passedBackend gate · passedFrontend gate · passedRelease gate · passedSource gate · passed

Agent benchmark results

The scoring canvas is ready; the experiment data is not.

The future comparison changes here: standard offline reference ↔ Agent-generated clone.

0 model runs

Visual fidelity—

Not measured

Interaction fidelity—

Not measured

Journey completion—

Not measured

Robustness—

Not measured

Efficiency—

Not measured

Agent-generated clone

No build yet

A screenshot and live run link will appear here.

Screenshot comparison—

Interaction replay—

Terminal journey checks—

Runtime & cost—

Independent research benchmark; not affiliated with or endorsed by Amazon. Brand and source-origin media remain the property of their respective owners and are shown only as bounded evaluation evidence.

Dataset readiness and curator evidence

Manifest schema

present

Required public artifacts

Frozen purpose, routes, journeys, checkpoints, coverage, and asset closure

present

Clone artifact

Runnable isolated offline reference server

present

User journeys

3 frozen success, failure, and recovery journeys

present

Visual checkpoints

16 route-state-viewport checkpoints

present

Legacy verifier report

Acceptance report is current and all release gates passed

present

Visual evidence companion

2 sanitized source/reference pairs

present

Seed and reset

Managed by the offline-clone harness rather than the candidate scoring contract

not applicable

Controlled time

Deterministic local simulations are certified by acceptance evidence

not applicable

License and assets

Research evidence remains subject to source-owner rights and publication review

not applicable

Official candidate report

0 valid agent experiment runs

not applicable

Scoring contract

Agent scoring remains intentionally empty until experiments start

not applicable