Skip to Content
Vektos
Anecdotes

Four green checks, none of them ours

The rule said to merge when `gh pr checks --watch` is green. It came back green in under a minute, with four checks, all Vercel's: CI had not started.

Date: 2026-11-11. Context: the pull request of the deploy's docs, #32 (journal ch. 74).

What happened

The rule in AGENTS.md read: a pull request "is merged with a rebase merge once its CI run is green (gh pr checks --watch)". For twenty-eight pull requests the command and the condition were the same thing, because CI's job was the only check a pull request had.

The deploy of chapter 73 connected three Vercel projects to the repository. Since then every pull request also gets four checks from Vercel: a preview of each project, and its comments. On the first two (#29, #31) CI started at once, so the command still waited for it.

On the third, #32, GitHub Actions was in a major outage. By the clock:

Time (UTC)What
20:54:16The pull request is opened
20:54:20CI's run is made, and waits in a queue. It has no job yet, so it is not among the checks
20:55:08The last of Vercel's four checks passes. gh pr checks --watch ends: four passes, exit 0
20:55:38The assistant closes and reopens the pull request, which makes a second run
20:57:38The second run gets its job. The first never did
21:01:54CI's job, check, passes
21:05:49Merged

For two and a half minutes the pull request showed four green checks and nothing else, and the command the rule names had said yes. The assistant did not merge, because it read the list and saw no row named check. Nothing in the rule asked it to read the list.

The docs in that pull request were fine and CI passed. That is luck, not the rule working.

What we learned

  • A rule that names a command is obeyed as the command. The rule meant "CI passed". It said gh pr checks --watch, which means "every check listed now has finished and none failed". An agent scripted on the exit code would have merged.
  • Green speaks for the checks that reported. A check that is missing fails nowhere. Nothing in the output said that one was expected and absent.
  • An integration changed what an old command means. Nobody touched the rule or the workflow. Connecting Vercel added four checks, and with them a way to be green without CI.

What it means for Vektos

It is the fourth time in this project that "nothing reported" was read as "nothing wrong": a linter that was given no files (06), a valid spec that drew nothing (16), axe running no rule for a misspelt tag (19), and now a merge rule with no CI among its checks. Each time the fix was the same: the check must prove that it looked at something.

For the context a design system gives an agent, the lesson is in the wording. Say the condition, then the command that checks it. An instruction that is only a command is followed to the letter when the command stops meaning what it meant.

Follow-ups

  • Done: AGENTS.md and guide 20 name the job, check, with the line that prints its verdict.
  • When the repository is public: a ruleset that requires check, so GitHub refuses the merge itself.
  • A pr:merge script that merges only when check passed on the head commit.

On this page