Project plan

1. WhyCopy Anchor Link

AI coding agents write more and more of the code built with design systems. Vektos measures whether they follow one, how much help they need to, and what that help costs. It has to show hand-written RAG, generative UI, an agent with MCP tools, and evals, on a real question.

2. ThesisCopy Anchor Link

Tools like Supernova give agents the rules. Vektos checks, in CI, whether the agents followed them, and at what price.

A vendor-neutral benchmark with results over time, plus a chat that explains the results by citing design-system rules and regulations.

Compliance is the first metric. Cost per run is the second: every run records tokens in/out/cache and dollar cost per task, model and context level. This answers a question no design-system platform answers: which context gives compliant code at the lowest cost? It is a dimension of the benchmark, not a FinOps product.

Positioning vs. the marketCopy Anchor Link

Vendor MCPs become one of the context levels in the benchmark matrix, not competitors.

3. Why fintechCopy Anchor Link

Fintech is the domain of the tasks, not a separate project.

The bridge: every design-system rule has a regulatory rationale. Chat: "why is this a violation?" → retrieve the rule → cite the EAA article or WCAG criterion → render the correct component.

4. StackCopy Anchor Link

Stack compatibility notes (checked 2026-11-07)Copy Anchor Link

• All peer ranges resolve: React 19.3, Next 16.4, Ark 5.40, Panda 2.5, Vite 8.3, Storybook 10.6, AI SDK 7, zod 4.6, Drizzle 0.45 (1.0 is RC; revisit in week 2), Playwright 1.65 (Bun supported since 1.65 / Bun 1.4).
• Storybook defaults to react-docgen (Babel, no TS API). Never set reactDocgen: "react-docgen-typescript". Props must be declared in the component file for docgen and the MCP to see them.
• Publishing: Changesets does not rewrite workspace:*, bun publish has no trusted publishing. Flow: changeset version → bun pm pack (rewrites workspace:*) → npm publish *.tgz with OIDC. Keep published packages free of internal deps where possible.
• oxlint JS plugins are alpha. The Panda plugin binds one panda.config per run and resolves include from cwd, so the root oxlint-panda.mjs sets cwd to packages/ds. Its eslint peer is unused.
• styled-system/* imports rely on tsconfig paths. Works for tsc, Next and Bun; Storybook needs Vite tsconfigPaths. Consider subpath imports (#styled-system/*) before publishing @vektos/ds.

TypeScript 7 caveatCopy Anchor Link

TypeScript 7.0 (stable since 2026-07-08, typescript@7, binary tsc) ships the native compiler without the JS compiler API; the API returns in 7.1 with a different shape. Anything that does import "typescript" breaks: typescript-eslint, ts-jest, ts-node, TypeDoc. None of them is in this stack. Therefore:

• TS 7 for type checking (tsc --noEmit) in graders and CI. Pin typescript@~7.0.2; 7.1 is not needed.
• oxc-parser for AST analysis in graders.
• .d.ts emission for published packages via tsc -b / --emitDeclarationOnly. No bundler-generated declarations (they need the API).
• Next.js ≥ 16.3 runs the project-local tsc CLI in next build, so TS 7 works as is. The CLI checks the whole tsconfig project, tests included: exclude them or run tsc --noEmit in CI with typescript.ignoreBuildErrors.
• Panda CSS ≥ 2.0 has a native compiler and no longer depends on ts-morph.
• Editor: TypeScript 7 extension for VS Code. No tsserver, so the next tsconfig plugin may not load.

Fallback if a tool needs the API: side-by-side aliases ("typescript": "npm:@typescript/typescript6", "@typescript/native": "npm:typescript@^7"), never a downgrade. The plan does not change.

5. ArchitectureCopy Anchor Link

vektos/
  apps/web/           Next.js: dashboard + chat
  packages/
    ds/               Ark + Panda, 8 components, markdown doc per component
    ds-mcp/           MCP server: components, props, examples, tokens, rules
    bench/            result schema, tasks, AI SDK runner, graders
    pricing/          model price tables, cost per run (published as @vektos/pricing)
    rag/              ingest, chunking, embedding, hybrid retrieval, reranking
    db/               Drizzle schema, migrations, pgvector
  tasks/              5 fintech tasks + JSON fixtures
  corpus/             WCAG 2.2 and the EAA as Markdown, converted from the official HTML (guide 16)
  docs/PLAN.md

All packages are published under the @vektos npm scope.

rag and bench are separate packages: the two hardest parts, testable alone, publishable.

pricing is small on purpose: a price table per model (input, output, cache read/write, per million tokens), a cost(usage, model) function and a date-versioned table so old runs keep their original price. The runner records usage from the AI SDK response and stores both tokens and cost on each run. No dashboard, no forecasting, no billing: those belong to the paused FinOps product.

Chat: two pathsCopy Anchor Link

• Numbers (runs, scores, cost, trends): SQL tools such as query_runs, compare_runs, get_violations, cost_by_context. No number ever comes from the model's memory.
• Text (docs, rules, traces, PR comments): RAG, isolated per tenant.
• Read-only tools always available. Tools that modify rules only after explicit approval (human-in-the-loop).
• Mandatory citations: every claim points to the query or document it comes from.
• Tool results render as components (charts, tables, diffs, DS components), not as text.

Benchmark matrix (phase 2)Copy Anchor Link

2 stacks × 3 context levels, same effort on both sides:

Separates two questions: "does the stack matter?" and "does the context matter?". The second one brings clients. Each cell reports score and cost, so the answer is "this context gives +N points for +$X per task".

RunnersCopy Anchor Link

Interface Runner with adapters:

1. AI SDK agent + MCP first: full control, cheap, model/provider switch via flag, reproducible.
2. Claude Code / Codex headless later: credible, it is what teams actually use.

Both consume the same DS MCP server, so the "standard MCP" context is identical.

6. Demo scope (vertical slice)Copy Anchor Link

Only Ark + Panda, only the AI SDK runner.

1. DS: 8 components: Button, Input, Select, FormField, Card, Table, Badge, Dialog.
2. 5 fintech tasks with synthetic JSON fixtures.
3. 3 graders: tsc type check, AST token analysis with oxc, axe.
4. Cost per run via @vektos/pricing: tokens and dollars stored with every result.
5. Minimal dashboard: run list, run detail with violations, screenshots and cost; compliance per dollar across runs.
6. RAG + chat right after the first run: DS docs + 2 regulatory PDFs.
7. No CLI upload, no multi-tenant, no Tailwind until the demo.

7. PhasesCopy Anchor Link

Next steps (decided 2026-11-10)Copy Anchor Link

The design system had 33 governed components (46 after DS-6 and DS-7), the chat composes views (ch. 44). An audit of the app (ch. 45) found it dense and silent about its own result, so the app comes before any paid run. The order, all at $0:

Paid models are off the plan for now. Components take inspiration from @pyck/react for names and API shapes; the implementations are Vektos's own.

Week 0 outcome (2026-11-08): GOCopy Anchor Link

TS 7 alone is enough. Nothing in the stack needed the TypeScript 7 JS API or a TS 5 fallback.

Unplanned additions that stayed: Biome → oxc swap, Storybook, root check script, shared editor settings.

Phase 2, after the demo: Claude Code runner, Tailwind adapter, CLI with upload (Codecov-style GitHub Action), cost regression in CI (fail the PR if cost per task rises above a threshold), production backups (managed Postgres + nightly dump to a bucket + monthly restore drill: see guide 12), LLM-as-judge, visual diff (needs a pinned Chromium instead of the system Chrome the page checks use: see guide 11), multi-tenant, "why did it get worse?", rule assistant with human-in-the-loop, on-demand reports.

8. RisksCopy Anchor Link

1. Honest comparison between stacks, same effort on both sides.
2. Multi-tenant isolation in RAG. Must be tested explicitly, evals included.
3. GDPR. Plan for an EU-region model or the user's own API keys from the start.
4. Competition. Design-system platforms could add verification. Advantage: specialization and neutrality.
5. Cost scope creep. Cost is a metric of the benchmark, not a product: no budgets, forecasts or billing connectors.

9. NameCopy Anchor Link

The project was called "Aurora" for one commit. It was renamed to Vektos in November 2026: Aurora collides with Amazon Aurora, a Postgres-compatible database, which is confusing next to a Postgres stack.