Skip to Content
Vektos
Guides

Dashboard

A Next.js app, built with the design system, that shows how contexts compare, every run, and the page an agent wrote, from the Postgres history.

What

apps/web is a dashboard on top of the database. It needs Postgres (bun run db:up) and runs on port 3000:

bun run web
PageShows
/ OverviewOne sentence and three stats on what the benchmark found (lib/findings.ts); a heatmap of contexts × models with a mean column, one metric at a time (?metric= design system, quality, done, cost), one domain (?domain=, banking by default; guide 24) and a batch (?label=); models with fewer than five runs hidden unless ?all=1; cost per run against violations on a log-log chart
/runsEvery real run, 50 to a page (?page=), filtered with five Selects (?domain=, ?model=, ?context=, ?task=, ?label=), sorted by a column head (?sort= ds, quality, steps, cost or started, ?dir=asc); every choice is in the address. ?batch= and ?rule= narrow it from other pages
/batchesOne row per benchmark call: batch label, model, context, design-system hash, mean violations per run, cost. A "new version" badge marks the first batch after the design-system hash changed
/runs/<id>One run: cost, tokens, batch and design-system hash with copy buttons; the screenshot next to the code, its lines numbered and those with a violation marked; violations in an accordion by rule, each linked to its line; every step; a "Compare with…" menu of the task's other runs
/runs/compare?a=&b=Two runs side by side: their numbers, their screenshots, and the rules each broke, the biggest difference first
/tasksWhich tasks are hardest (share done, quality and design-system violations per run, read on the series that ran every task: difficulty.ts), then the ten tasks: title, summary, runs, no help against the best context, the components a good answer uses
/tasks/<id>One task: the brief as the agent reads it, the texts the page must show, the best and worst runs, a heatmap of the task alone, the reference answer
/models/<provider>/<model>One model: runs, violations and cost per run, a row per context, the rules it breaks most; every model name in the app links here
/rulesThe rules broken most, by group (?group=), domain (?domain=) and model (?model=), with one message as an example
/methodologyHow it works: the five contexts, the three groups of rules, the tasks, cost, how the overview counts, what it does not say
/runs/<id>/screenshot/<theme>The PNG stored with the run
/chatAsk in plain language; answers come from tools over these same queries and from the docs (guide 17)

The queries are in packages/bench/src/dashboard.ts (overview, listBatches, listRuns, filterOptions, getRun, getScreenshot). They count like bench:history: violations marked ignored or covered are not counted (anecdote 08). The app imports @vektos/bench/dashboard, a subpath, so it does not bundle the runner, the AI SDK or Playwright.

Why

  • Built with the design system. Every component on the page is from @vektos/ds; the rest is css() with tokens. A test (dogfood.test.ts) grades apps/web/src with the benchmark's own design-system graders and fails on a single violation. A dashboard that breaks the rules it measures would not be credible, and writing it showed what the rules feel like from the agent's side (anecdote 09).
  • The chart answers the thesis. "Which context gives compliant code at the lowest cost?" is the bottom-left of one picture. Color is the context, shape is the model, and a table with the same numbers sits below it.
  • Server components, no client state. Pages read the database on the server. Filters are links. The only client code is the dark-mode switch and the header's current-page marker.
  • Dark mode from the start. The choice is remembered, and a small script in <head> restores it before the first paint so a dark page does not flash white.
  • Screenshots from the database. The bytea column is served by a route handler, so the dashboard does not need results/ on disk.
  • A broken database is a page, not a crash. error.tsx says to start Postgres.
  • Every name leads somewhere. A model, a context, a task, a batch label, a batch and a rule are links wherever they appear: the tables, the run's header and batch card, the chat's tool results. They all go to the run list with that filter, through one helper (runsHref in apps/web/src/lib/links.ts), so a name always leads to the same place. A count in the Overview's table opens the runs behind it. /runs takes ?batch= and ?rule= (a11y/color-contrast) as well, shown as a sentence with a way to remove them.
  • A link looks like one. Text links are the design system's Link, underlined in accent.text; the filter pills, the header and the Overview's badge links keep next/link with a class of their own. Router (components/router.tsx) gives the design system Next's router through RouterProvider, so Link, Breadcrumb and Pagination move without a page load and prefetch on hover. (Until chapter 38 a global rule styled every <a> without a class.)
  • A run's page starts with a Breadcrumb (Runs / task), its error is an Alert, and its screenshots are Tabs, light first. The run list pages with Pagination.

Try it

bun run db:up && bun run db:migrate
bun run web          # http://localhost:3000
bun run build        # a production build; pages read the database at request time
bun test packages/bench/src/dogfood     # the dashboard follows the design system
bun run check:db                        # the queries, against a real database
bun run check:app                       # with the app running: every page at 1280 and 390, light and dark, axe

Click a model's name anywhere, then a rule in one of its runs: /runs?rule=a11y%2Fcolor-contrast lists every run that broke it. Open /runs?model=moonshotai/kimi-k3.1-pro&context=none, pick a run, and compare its screenshot with the same task at ds-mcp-check-full. Open /batches to see which design-system version each batch used, and /?label=smoke to see only the smoke-test batches.

Extend it

  • A new page. A file in apps/web/src/app/, a server component, await connection() first so Next does not run the query at build time, and a query in dashboard.ts.
  • A token the page needs. Add it in packages/ds/panda.config.ts and run bun run --filter @vektos/ds prepare. Adding one changes the design-system hash of later benchmark batches, because the agent reads the same table.
  • Responsive layout. Use the breakpoints ({ base: "4", md: "8" }, mobile first). The totals row is a grid of two columns that becomes four from md; page padding grows the same way. Cards in a row use flexWrap with flexBasis: "60" and minW: "0", so a long line of code scrolls inside its block instead of widening the page.
  • Tasks are read from the repository at run time (lib/tasks.ts, Node's fs), not from the database. next.config.ts ships tasks/ with the server (outputFileTracingIncludes); a deploy that leaves the repository out needs them in the database instead.
  • Syntax colours in the code. The code view numbers and marks lines; colouring tokens would need a highlighter and colour tokens for it.

Known gaps

  • check:app runs in CI on demo runs (guide 20): fifteen addresses, a run's page and a model's page included, at two widths and two themes. Lighthouse is not run.

  • Task difficulty depends on one panel. Only the series that ran every task count; a new task run by one series leaves no panel until a series has run all of them. The design-system column mixes contexts, so it mostly reflects the runs without context.

  • The share preview is one picture for every page. opengraph-image.tsx draws the name and the question with @vektos/ds/raster; a run or a task does not get its own.

  • No auth, no write path. It reads the database; anyone who can reach port 3000 sees every run, including the code agents wrote. Fine on a laptop, not on the internet.

  • The chart is read-only. No hover table, no zoom; the points have a <title> and the table has everything. The chart is hidden from screen readers on purpose, because the table carries the same numbers.

  • No trend chart. /batches lists batches in time order and /regressions compares versions (guide 23), but nothing draws a line per model and context across batches.

  • The finding compares means of means. Each model counts once, whatever its number of runs; the cost is the median of each model's ratio, so one expensive model does not decide it. Both are said on the page, not argued.

  • The metric and batch controls need JavaScript. They change the address with the router; without scripts they do nothing (the links to "show all" and to the runs still work).

  • Filters do not know each other. On /runs?rule=… the selects list every model and context, including ones with no run that broke the rule, so a choice can lead to an empty list.

  • No sticky table head. The runs table scrolls with the page; its head goes with it. A sticky head needs the table's scroll box to scroll vertically, and TableScroll scrolls sideways.

  • Names in the chat's sentences are not links. Only the tool results are.

  • Tables scroll sideways on a phone, inside their cards; the chart is small there.

Report

A single HTML page that puts the saved runs side by side, with screenshots, violations and code, so you can see what changed between contexts.

Retrieval (RAG)

Chunk the project's Markdown, search it by words and by meaning, rerank the candidates, and measure each stage.

On this page

WhatWhyTry itExtend itKnown gaps