Skip to Content
Vektos
User guide

Writing a task

How to add your own task to the benchmark, with a prompt, data and a reference solution.

A task is a folder under tasks/. The runner and the tests pick it up on their own.

tasks/invoice/
  task.json       id, title, summary, components, mustShow
  prompt.md       what to build, like a product ticket
  fixture.json    synthetic data
  solution.tsx    your own answer, with zero violations
  1. Name the folder like the id, in kebab-case.
  2. prompt.md never mentions the design system. How much the agent knows about it is what the benchmark varies. End with "The data is in ./fixture.json."
  3. mustShow lists texts the finished page cannot avoid: names, identifiers, labels. An empty page has no violations, so this is what makes doing nothing fail. Avoid formatted numbers.
  4. solution.tsx default-exports the page. Run bun run check: it must pass graders, tsc and lint.
  5. Update the count in tasks/tasks.test.ts, then try it:
bun run bench --task invoice

More in guide 08.