Commerce platform · Case study
An agent that could turn an existing site into a storefront.
A custom, tool-using agent ingested an existing web presence, planned a commerce experience, and assembled local storefront and checkout pages. A purpose-built evaluation harness made the result inspectable instead of relying on a convincing demo.
- Work
- Agent architecture and evaluation
- System
- Custom tools, durable runs, rendered-page evidence
- Focus
- Website ingestion, generation, checkout, review
- 01IngestSite, brand, catalog
- 02PlanPages and commerce flow
- 03Use toolsBuild and revise
- 04RenderLocal experience
- 05EvaluateRules, evidence, review
The challenge
Generation was only half the problem.
A storefront builder has to do more than make a page that looks plausible. It must understand imperfect source material, preserve brand and product facts, choose an information hierarchy, use the right assets, construct a coherent checkout path, and stop when a human decision is required.
The hard part was giving an LLM enough agency to do useful work while keeping its actions observable, recoverable, and measurable across repeated runs.
What the work involved
An agent with tools, memory, and a quality bar.
The workflow treated the agent as one component inside a product system.
Turned broad goals into narrow actions.
Purpose-built tools handled browsing, content, assets, page structure, catalog data, and checkout work. The agent chose actions; the software kept each action’s contract explicit.
Made long-running work resumable.
Execution history and state survived beyond one model response. That created a basis for reconnecting to a run, inspecting its progress, and checking what happened before continuing.
Kept the workflow larger than one model.
Model-specific behavior sat behind runtime adapters. The core agent loop, tools, and stored evidence did not have to become provider-specific each time a candidate changed.
Scored the task the user actually asked for.
Fixtures checked required copy, sections, ordering, assets, commerce state, and handoff behavior. Repeated runs exposed variance; screenshots and traces stayed available for human judgment.
Evaluation shape
A good-looking page was evidence, not the score.
Required content, exact changes, ordering, assets, links, and commerce state.
Tool use, completion state, takeover requests, resume behavior, and unsafe actions.
Desktop and mobile captures retained for review without pretending pixel variance was objective quality.
Events, tool calls, phases, usage, scores, and artifacts kept together for replay and diagnosis.
What transferred
The agent is the visible part. The harness is what makes it a product.
Tool contracts, durable state, explicit handoffs, and repeatable evaluations turned a one-off generation trick into an engineering system that could be compared, debugged, and improved. The same structure applies wherever an agent moves through a multi-step task with real effects.