Chat

WhatCopy Anchor Link

/chat in the dashboard answers questions in plain language: which context level is best and what it costs, what agents get wrong, why a violation matters, what the Accessibility Act requires. The model is Claude Haiku 5.5 through the AI Gateway. It answers by calling tools, and the page draws what the tools return as the dashboard's own tables and cards.

bun run db:up && bun run db:migrate
infisical run -- bun run web            # http://localhost:3000/chat
infisical run -- bun run chat:try "Why does low color contrast matter?" "And which criterion is it?"

The chat needs the AI Gateway key in the server's environment. Started without it (bun run web on its own), /chat says so above the form and disables it, and POST /api/chat answers 503 with "The chat needs AI_GATEWAY_API_KEY. Start the app with: infisical run -- bun run web". Nothing is called and nothing is spent.

chat:try asks the running app from the terminal and prints the tools used, the answer, and whether each citation names a source a tool returned. Each question after the first continues the conversation.

Views. When an answer is mostly numbers, the model composes a view (stats, a table, an alert) from the catalog of guide 21 and passes its root node to show_view (packages/chat/src/view.ts). The tool accepts it only if every part renders (validateSpec) and every number in it is one a data tool returned in the same answer (specNumbers, isGrounded). A refused view goes back to the model with what to fix (formatIssues); after a second refusal it answers in words. The page draws the view while it streams in, keeps only an accepted one, and shrinks the data tools' tables to one line next to it.

Actions. A view may end with a form that opens a page of this app (ch. 69). The app registers what forms may run (CHAT_ACTIONS, packages/chat/src/actions.ts): open-runs (the run list, filtered by model, context, task or label) and compare-runs (two runs side by side). show_view accepts a form only if it names one of them and names its fields after the action's (validateSpec(view, { actions }), guide 21); a form that runs nothing is refused. On submit the page builds the address itself from the values (actionHref in tool-view.tsx, with runsHref/compareHref) and opens it in a new tab, so the conversation stays. The model never writes an address, and nothing is sent anywhere. A select's options may be { label, value }, so the reader picks "beam-free · none · behaviour-v3" and the form sends the run's id (ch. 70).

The code is packages/chat (tools, citations, budget, the endpoint as a function) and, in apps/web, the route api/chat, the page chat and the components in components/chat/.

Sharing. The owner of a conversation can share it ("Share" above it, a Popover with a Switch and the link in a Clipboard). A shared conversation is read by anyone at /chat/shared/<id>, signed in or not; it is drawn by the same Transcript as the live chat, and nothing calls a model. chat_conversations.shared_at holds when it was shared (migration 0006_chat_share); shareConversation, getSharedConversation and listSharedConversations are in packages/chat/src/conversations.ts. Whoever cannot ask, on /chat, gets the shared ones as examples to read.

WhyCopy Anchor Link

• Numbers come from tools, not from the model. The system prompt says every count, cost and comparison must come from a tool, and the page shows the tool's table next to the words. A sentence about the benchmark can be checked against the table above it.
• No free SQL. The plan says "SQL tools" with named examples and does not say who writes the SQL. Here the model fills the parameters of queries that already exist (the dashboard's), so a prompt cannot make it read or change anything else. A request to "run DROP TABLE runs" is answered with a refusal, and the tool cannot do it anyway.
• A citation is a ref the tools returned. Each excerpt has a ref (eight characters of its chunk id) and the model cites [3f9a1b2c]. The page checks every ref against the tool results of that answer: a match is a numbered mark with the source below it; a ref that matches nothing is shown as "unverified source". A user who asks the model to cite [deadbeef] gets a refusal, not an invented source.
• The history comes from the database, not the browser. The browser sends one thing: the new question, as text, for a conversation id. The server reads the earlier messages, tool results included, from chat_messages, so a tool result the model sees is one the server produced. A browser could otherwise send a made-up one (a source that does not exist, a table with other numbers). The question is saved before the model answers and the answer when it ends, even if the reader closes the page. Conversations, sign-in and who may use the chat are in guide 18.
• A violation leads to its rule by lookup. explain_rule follows axe's tag to the criterion (guide 16), which was exact where search was not.
• The spending caps are a table. Every model step and every search is written to chat_usage with its cost, who asked and in which conversation. Before a request starts, the chat's total is compared with its cap (default $1, VEKTOS_CHAT_BUDGET_USD) and the person's with theirs (default $0.50, VEKTOS_CHAT_USER_BUDGET_USD); a request that could take either over is refused with a 429 before the model is called. A step that takes a total to its cap is the last of its request; its cost is counted before it is written, so the decision never waits on the database. The limits survive a restart and apply to every tab.
• The cost is never rounded down. A step is priced by @vektos/pricing and the gateway's own figure is stored beside it; the dearer counts. In 24 real steps the two were identical. A search has no usage report for its rerank, so it is charged an embedding plus a flat $0.0003, above what it can cost.
• Costs are quoted, not rounded. Every cost in a tool result has a text twin (costText, costPerRunText: "$0.0126"), and the prompt says to quote it and never call a non-zero cost "$0". Before this the model wrote "about $0" for a run of $0.0126.
• Names in a result are links. A model, a context, a run or a rule in a tool's table leads to the dashboard's run list with that filter; a rule from top_rules keeps the filter the tool used.
• The design system draws the results. Tables, badges, cards and the form are @vektos/ds; the dogfood test grades the chat's code with the benchmark's own graders, and found nothing.

Try itCopy Anchor Link

infisical run -- bun run chat:eval-views  # five questions on the real data: view or not, at which try (~$0.01)
infisical run -- bun run chat:examples -- --as <github login>  # asks three example questions and shares them (~$0.007)

bun test packages/chat                  # citations, budget math, tools (no database, no model)
bun run check:db                        # the tools, the cap and the endpoint against a database
infisical run -- bun run web            # then open /chat and click "Why color contrast matters"
curl http://localhost:3000/api/chat/budget   # 401 unless signed in

The endpoint's tests use a scripted model and a made-up user: no network, no cost. They check, in the order the endpoint does: no key (503), nobody signed in (401), someone off the allowlist (403), the body (400), someone else's conversation (404, as if it did not exist), a full conversation (409), both caps (429, before the model is called, and stopped in the middle of a request), the limit of six steps, that the history comes back from the database with its tool results, and that a forged tool result never reaches the model.

What a real trial looked like (Haiku 5.5, about eleven questions, 29 billed calls): $0.0143 in all, of which $0.0128 for 24 answer steps and $0.0015 for 5 searches. The questions included a comparison of contexts, a follow-up, a question outside the project (refused), an injection attempt (refused) and a request to cite an invented source (refused, with the real sources offered). Every citation in every answer was in a tool result, and every number I checked against the database (the context table, four incomplete runs out of 33, one run's violations and cost) was right.

Extend itCopy Anchor Link

• A new tool. Add it to createTools in tools.ts with a zod input schema and an output type, add the type to ChatTools, and a case to ToolView in the web app to draw it. A tool that returns corpus text must return sources so its citations can be checked.
• Another model. Pass modelId to createChatHandler. It must have an entry in the price table (@vektos/pricing), or the first step fails before any call is paid.
• A different cap. VEKTOS_CHAT_BUDGET_USD=0.25. To start over, delete the rows of chat_usage (after a backup).
• A new action. An entry in CHAT_ACTIONS (what it does, its fields, which are required) and a case in actionHref that builds its address from the values. The prompt lists it by itself.
• Tests of a new behaviour. scriptedModel in fake.ts plays a list of text and tool-call steps; seedRuns and seedCorpus give a database to ask about.

Known gapsCopy Anchor Link

• Actions only open pages. None changes data; one that would needs a confirmation, an owner check and a test of what it did, which navigation does not.

• A shared conversation shows what it showed then. Its numbers are the database's on the day it was answered; the page says so. Sharing again does not refresh them.

• No one can unshare someone else's conversation, not even an admin; there is no admin.

• A verified citation proves where a source came from, not that it supports the sentence. The check is that the ref is in a tool result. A model could cite a real passage for a claim the passage does not make. The reader has the passage under "sources" to check.

• Numbers in a sentence are not checked, only those in a view. A view's numbers must be ones a tool returned; nothing compares the model's sentence with the table. In a few cases the model garbled a name (it wrote "Claude Haiku 4.5" next to the id of 5.5) while the numbers were right.

• One model, one language of answer. Haiku 5.5, in English. Other models are a config away but have not been compared for this job.

• The retrieval cost is an estimate. Embedding tokens at the model's price plus a flat amount for the rerank. The real figure was not measured.

• The key is checked when a request arrives, not what it can do. A key that is set but wrong or out of credit still gives "The assistant could not answer. Try again." to the reader; the real error is in the server's log (chat: the model call failed: …).

• The conversation's length is limited: 40 messages, of 2,000 characters for a question, six model steps per answer, 3,000 output tokens per step (a view's spec needs more than a sentence). A full conversation asks to start a new one.

• A view may show a number rounded from a tool's, or as a percentage of it (0.94 for 0.9375, 94 for 0.94): isGrounded accepts those, so a number "derived" by rounding passes.

• Two views in one answer are both drawn, if both pass; the prompt asks for one.

• The system prompt is longer: about 9,600 characters with the catalog, sent with every step.

• The whole history goes to the model each time. A long conversation costs more per question, and nothing summarises it.