CI

WhatCopy Anchor Link

.github/workflows/ci.yml runs one job on ubuntu-latest for every pull request to main and every push to main:

A run takes about three minutes. A new push to a pull request cancels the run of the previous one.

WhyCopy Anchor Link

• The checks of a block ran on one Mac. A file that only exists there (a generated type, a local .env) would pass every time, and nobody would know until a fresh clone.
• bun run check does not build the app. A design-system module that ran client code when it loaded broke every page and passed every check (anecdote 15). next build fails on it.
• The database tests need a database. They are a separate command so a laptop without Docker can still run check; CI has Docker, so it runs both.
• A page can build and still break. check:app opens every page of the built app, a run's page and a model's page included, and fails on a missing heading, a page wider than a phone or an axe violation. It needs runs to show: db:seed-demo writes two fake models (demo/alpha, demo/beta) at two contexts on three real tasks, under two grader versions, so even /regressions has a comparison, and one worse, to draw. The numbers are fixed in packages/bench/src/demo.ts; no model is called. The auth secret in the workflow is a placeholder: nobody signs in.
• No secrets, so no cost. The workflow has no AI Gateway key: nothing in it can call a model, create embeddings or open the chat. Benchmark runs stay a decision made by a person. GitHub's free plan includes 2,000 Linux minutes a month for a private repository and has a $0 spending limit by default; a public repository does not count minutes at all.

Try itCopy Anchor Link

Open a pull request: the run appears under the PR's checks. From a terminal:

gh pr checks --watch                               # wait for every check the PR lists now
gh pr checks | awk -F'\t' '$1=="check"{print $2}'  # CI's own verdict: it must print "pass"
gh run list --limit 5                              # the latest runs
gh run view <id> --log-failed                      # only the failing step's output

Since the deploy a pull request lists five checks: CI's job, named check, and four from Vercel (a preview of each project, and its comments). Only check says the code is right. Vercel's are green in about a minute, and --watch waits for the checks that exist: while a CI run is queued and has no job yet, it is not in the list, and the command ends with four passes and exit 0 (anecdote 29). A run that stays queued: gh run list shows it; closing and reopening the pull request starts another.

To reproduce a run on a laptop, from a fresh clone:

git clone <repo> /tmp/vektos && cd /tmp/vektos
bun install --frozen-lockfile
bun run check && bun run db:up && bun run check:db && bun run build && bun run docs:build

The app's pages on demo runs, on a laptop, without touching the real database:

docker exec vektos-db-1 psql -U vektos -d vektos -c "create database vektos_demo"
export DATABASE_URL=postgres://vektos:vektos@127.0.0.1:5433/vektos_demo
bun run db:migrate && bun run db:seed-demo     # "16 demo batches and 48 runs written."
bun run build && BETTER_AUTH_SECRET=local-placeholder-0123456789abcdef bun run --cwd apps/web start -- --port 3100
VEKTOS_APP_URL=http://localhost:3100 bun run check:app   # 4 pass

Extend itCopy Anchor Link

• The page check of the benchmark (bun run check:page) needs Chrome too, which the runner already has (check:app uses it); it also needs the design system's Storybook build and a scratch app, about a minute more.
• A new page goes in PAGES in apps/web/src/app.browser.test.ts. A page that needs an id is found from a link, as the run and model pages are.
• More demo data (a new metric, a run with screenshots) goes in demo.ts; its test in demo.test.ts pins what the pages can count on.
• Another check is one more step. Keep the expensive ones last, so a cheap failure stops the run early.
• Never add the gateway key to run a benchmark in CI. A scheduled benchmark is a decision about money, and belongs to a separate, manually started workflow with its own budget cap.

Known gapsCopy Anchor Link

• GitHub does not block a red merge. Required status checks need a paid plan for a private repository. The rule is in AGENTS.md: merge only when the run is green. Once the repository is public, a ruleset can require the check job.
• check:page (Chrome, 229 tests) runs only on a laptop.
• The demo runs have no screenshots, steps or chat history, so the parts of the pages that draw them are checked empty.
• ubuntu-latest moves to new Ubuntu releases on GitHub's schedule; pin ubuntu-24.04 if a move breaks something.
• CI does not ship, and does not stop a deploy. Vercel builds every push on its own, a red main included (guide 22).