Database

WhatCopy Anchor Link

Every real run is saved to Postgres, next to the files in results/. The files are for a person to open; the database is for comparing runs over weeks.

packages/db holds the schema (Drizzle), the migrations and the connection. Six tables for benchmark runs:

packages/bench writes to it: store.ts saves, import.ts loads old results/ folders, history.ts summarizes.

Three more tables hold text for retrieval (rag_sources, rag_chunks, rag_embeddings). packages/rag writes them; see guide 15. chat_usage holds what the chat has spent, one row per model call, and is what its spending caps read (guide 17). The auth_* tables hold who may use the chat, and chat_conversations and chat_messages what they asked (guide 18). They are all in the same backups.

WhyCopy Anchor Link

• results/ is not history. It is gitignored, has no backup and cannot be queried. A laptop change would lose every real run.
• A result is only comparable with what it ran on. Each batch records the git commit, whether the tree was dirty and a hash of the design system (tokens and component sources). Each run records a hash of its task (task.json, prompt.md, fixture.json). bun run bench:history flags a row built from more than one design-system version.
• Violations are rows, not a count. The dashboard and the chat need to ask "which rules does this context fix?", not only "how many".
• Files and screenshots are kept. A run can be reviewed after results/ is gone.
• Ignored violations are kept too, flagged ignored, so a report can show how often vektos-ignore was used.
• Postgres with pgvector from the start. Retrieval needs the extension (migration 0003 runs create extension vector), and changing image later would mean moving data.
• Port 5433. Many machines already run a Postgres on 5432.
• Real runs save by default, mocks do not. A mock run is a test of the pipeline; in the history it would be noise. --save saves one anyway, --no-save skips a real one.
• covered is decided when a run is saved. bun run bench:recount applies the rule again to every stored run, for runs saved before it existed or when the rule changes. It changes only the flag, never the violations, and is safe to repeat.
• A real run needs the database up. The CLI checks it before the first model call, so a paid run is not lost to a connection error.
• A database that stops answering is an error, not a hang. Every call the CLI makes has a timeout (30 s; VEKTOS_DB_TIMEOUT_MS changes it). If saving fails in the middle of a batch, the CLI stops, tells you the runs so far are in results/ and says to run bun run bench:import when the database is back. It stops because more runs would be paid for and not saved.

Try itCopy Anchor Link

bun run db:up          # starts Postgres 18 + pgvector on 127.0.0.1:5433
bun run db:migrate     # creates the tables
bun run bench:import   # loads the real runs already in results/
bun run bench:history

model                                 context        runs  ds      quality  done   cost
nvidia/nemotron-3.5-lightning-free    none           1     20.0    44.0     0%     $0.0000
openai/gpt-6-luna                     none           2     130.5   156.0    100%   $0.0075

ds and quality are the mean violations per run in each group (quality includes the completeness group: build failures and runtime errors); done is the share of checked runs that were complete. Violations marked ignored (silenced with vektos-ignore) or covered (a compiler error restating a design-system violation) are not counted. From now on a real run saves itself:

infisical run -- bun run bench --model openai/gpt-6-luna --label baseline
bun run bench:history

Look inside:

docker exec vektos-db-1 psql -U vektos -d vektos -c "select task_id, complete, cost_usd from runs"

Tests: bun run check:db creates a throwaway database, applies the migrations and drops it. The plain bun run check skips them.

BackupsCopy Anchor Link

The data lives in one Docker volume, so it has a copy outside it.

bun run db:backup            # a compressed pg_dump in ~/Backups/vektos, keeps the newest 14
bun run db:backups           # list them
bun run db:restore ~/Backups/vektos/vektos-2026-11-09T10-00-00.dump

The restore creates a new database (vektos_restore_<timestamp>, or --into name) and prints a DATABASE_URL for it. It never writes over the database in use. To make a restored copy the live one, point DATABASE_URL at it.

Backups also happen on their own:

• after every real batch of bun run bench (--no-backup skips it; mock batches never trigger one);
• before bun run db:migrate, when the database already has tables (--no-backup skips it).

Move the folder with VEKTOS_BACKUP_DIR. Put it in iCloud Drive or let Time Machine cover it, so the file also exists off the machine.

How it works: the local Postgres on a laptop is often older than the one in the container, and pg_dump must match the server, so for the compose database the tools run inside the container (docker exec). For any other DATABASE_URL it uses pg_dump from the PATH. A failed backup deletes its half-written file, and a file that does not start with Postgres's PGDMP header is rejected.

Backups in the finished productCopy Anchor Link

The commands stay the same; where they run changes.

Four rules for both: keep at least two copies in two places (the provider's backup and our dump in the bucket); restore once a month into a scratch database and check that the tables have rows, because a backup nobody restored is a hope; back up before every migration, which the deploy pipeline must do on its own; and when screenshots make the dump large, move them to the bucket and keep only the path in the table.

Extend itCopy Anchor Link

• A column or table. Edit packages/db/src/schema.ts, then bun run db:generate and commit the new SQL in packages/db/drizzle/. Never edit an applied migration.
• Another query. Add a function next to history(). The chat's SQL tools will be the same kind of function.
• Another database. Set DATABASE_URL. Without it the local compose database is used.
• Repeats. Pass the same --label to each batch of one experiment.
• A backup of a remote database. DATABASE_URL=<remote> bun run db:backup runs pg_dump in the compose container when it is up: its tools are Postgres 18, as on Neon. The laptop's own pg_dump (Homebrew's was 14) refuses a newer server; it is used only without the container.
• Another major version of Postgres. Its files do not open with the next one: back up, point the compose file at the new image and a new volume, start it, restore, compare every table's count. Postgres 18's images want the volume on /var/lib/postgresql, not …/data (ch. 71). To go back, point the compose file at the old image and volume again.

Known gapsCopy Anchor Link

• The Postgres 17 volume is still on disk (vektos_vektos-db), kept to go back to after the move to 18. docker volume rm vektos_vektos-db frees it once 18 has been used for a while.

• Imported runs have no provenance. The first three real runs were imported without a commit or a design-system hash (both are null).

• No views yet. history() is one query. The report shows runs side by side; trends over time come with the dashboard.

• Screenshots are stored in the table. Fine for hundreds of runs; thousands would want object storage.

• The volume can still be deleted. docker compose down -v and docker system prune --volumes remove vektos_vektos-db. Restore from a backup.

• Backups are local. They are as safe as the folder they are in. Off-machine copy is iCloud or Time Machine for now, and a bucket in production.

• No automatic restore test yet. The test suite proves a round trip; the monthly restore drill of the table above is not scheduled.

• ignored counts are saved, not shown. bench:history leaves them out.

• Local credentials. The compose file's password is vektos, for local use only.