The CSS that was not there
The dashboard built clean and rendered with half its design-system styles missing, and writing it against our own rules hit the same walls the agents hit.
What happened
The dashboard type-checked, linted and built, and the first screenshot looked almost right.
Almost: card titles touched the card's left edge, and a heading sat flush against the table
under it. CardHeader has px: "6". In the browser its padding was 0px.
The class px_6 was on the element. The rule for it was not in the page's CSS. Panda's
PostCSS plugin ran from apps/web, and include in panda.config.ts is relative to where it
runs, so ./src/**/*.tsx meant the web app's files, not the design system's. Only the
classes the app wrote itself (and the benchmark tasks) were generated. A class used only
inside a component, like px_6, was silently missing.
What we learned
- Nothing red. The compiler cannot know that a class has CSS, and neither can the linter.
Only looking at the page, or reading the computed style, shows it. Neither the old
page.tsx(oneButton) nor the benchmark noticed: the page check generates CSS withpanda cssgen --cwd packages/ds, so it never had the problem. - The fix is one option:
cwd: "../../packages/ds"on the plugin, so Panda resolvesincludethe way Storybook and the benchmark do. - Writing against the design system teaches what agents feel. In an afternoon I met the
errors the agents meet:
gridTemplateColumns: { base: …, md: … }fails withTS2769: the design system defined no breakpoints. It is the uncoveredTS2769of anecdote 08.minW: "0"fails: sizes had no0. The spacing scale stopped at8; there was no12.fontFamily: "mono"was accepted and meant nothing, because there was no font token forstrictTokensto enforce. The wrong value passed silently.pb,pt,prare flagged by theno-physical-propertieslint rule (594 findings in the benchmark's runs), andpbedoes not exist as a shorthand; the longpaddingBlockEnddoes.- A scrollable table needs
tabIndex={0}for axe, and thejsx-a11ylint rule forbidstabIndexon a non-interactive element. Our own graders flag a// oxlint-disablecomment, so the answer was a lint override forapps/web, in the config, not a suppression in the code.
- The dashboard is a test subject. The rules were written for agents; applying them to our own app found places where they are unclear or incomplete.
What it means for Vektos
- Added
xltofontSizesand apagesize to the design system. Both change the design system's hash and the catalog the agent reads, so batches run after this commit are a new design-system version inbench:history. dogfood.test.tsgradesapps/web/srcwith the benchmark's graders. The dashboard has to stay at zero design-system violations.- The design system is missing things agents also reach for: breakpoints, a monospace font, bigger sizes. Whether to add them is a decision about the benchmark, not only the app.
Follow-ups
- A grader for token names that do not exist (anecdote 08) would catch
fontWeight: "bold"and anyfontFamilythat is not a font token. - Done: the gaps are closed (journal chapter 19): breakpoints
sm/md/lg/xl, font familiessansandmono, spacing 5, 10, 12 and 16, sizes0, 6, 12 and 16. The choice came from the 159 stored pages: agents wrotemd224 times,sm94,lg32 andxl8, andfontFamily: "mono"or"monospace"26 times. - A check that every class the build uses has a rule in the CSS, or a computed-style probe in the page check, would have caught this without a screenshot.
The compiler that counted twice
Most of the quality violations in runs without context were the design-system violations again, seen by the compiler. They are now counted once.
The hybrid that was worse than its best half
Merging full-text search with vector search scored below vector search alone, with four different embedding models. A reranker fixed it, and the reason says where each half helps.