Report

WhatCopy Anchor Link

bun run bench:report reads the database and writes results/report.html, one file with no server and no network, then opens it in the browser.

The page has three parts:

1. Summary. One row per model and context: runs, mean design-system violations, mean quality violations, share of complete runs, cost. A row built from batches that ran on different design-system versions says so.
2. One section per task. The latest run of each model and context, side by side, in the order none → storybook-mcp → ds-mcp → ds-mcp-check → ds-mcp-check-full. Each card has:
   • the screenshot (the Light / Dark buttons at the top switch every card; click a screenshot to enlarge it, Escape closes it);
   • violation counts for design system and quality, whether the run was done, and the cost (with the gateway's charge); a "not counted" chip shows compiler errors that restate a design-system violation;
   • the tools the agent called, so you can see it read the guidelines before writing;
   • Violations by rule, Every violation (file, line, value, message) and the code written (page.tsx), each in a collapsible block.
3. Batches. The commit and design-system hash of every batch shown.

WhyCopy Anchor Link

• Numbers hide what changed. "39 violations versus 0" is a claim; two screenshots of the same task and the code behind them is the evidence. The report is where a person decides whether the benchmark measures what it says.
• Latest run per cell. Repeats feed the summary's means. Extra cards would bury the comparison.
• One file. It can be attached to a message or opened a year later. The screenshots are inlined as data, so it does not depend on results/.
• Everything from the model is escaped. Pages, error messages and violation values are untrusted text. The only script in the file is the page's own.
• Colour never carries the meaning alone. Every chip says its number or its word, and every screenshot has alt text naming the model, context, task and theme.

Try itCopy Anchor Link

bun run bench:report
bun run bench:report --label haiku-contexts
bun run bench:report --model anthropic/claude-haiku-5.5 --task statement
bun run bench:report --mocks --no-open

Tests: bun test packages/bench/src/report (rendering, with no database), bun run check:db (reading from a database) and bun run check:page (the page's own script in Chrome).

Extend itCopy Anchor Link

• Another section. Add a loader to report/data.ts, a field to ReportData and a function in report/render.ts.
• Another filter. Add it to HistoryFilter in history.ts and to the queries.
• The dashboard. loadReport() is the data layer. The dashboard in apps/web can call it and render the same cards with the design system.

Known gapsCopy Anchor Link

• Light only until you switch. The page starts with light screenshots. The choice is not remembered.
• Size. Each screenshot is about 100 KB inline, so a report of hundreds of cards is tens of megabytes. Filter it, or wait for the dashboard.
• No side-by-side diff of the code. The pages are listed, not compared line by line.
• Run history is not charted. Only the latest run per cell shows; trends over time come with the dashboard.
• No chat or search. It is a static page.