Three defects the first sweep left behind, each caught by a rule the guard did not yet have. Statuses. RULES.md gives three; the tree had 25 values, and the free-text ones were carrying progress. "Implemented (wiring landed)" and "Ratified (rev. 2e). Rev. 2e adds the OpenTaskByPR lookup. Rev. 2d revised section 10..." were both changelogs living in the one field the other rules exempt. Every PRD now declares Draft, Approved or Finalised and nothing else; five that declared nothing now declare one. A revision number moves to **Revision:**, a pointer to the authoritative document to **Authority:**, and what a past revision changed to git. CLAUDE.md's citation of state-store-contract rev. 2e still resolves. Stale cross-references. "Supersedes the 2026-09-03 open question", "this bullet previously said", "an earlier draft of this document claimed" all point at text the reader cannot see, because it was replaced. A supersede marker survives only when it names a file, a section or a symbol that can be found. Temporal hedges. 100 uses of today, currently and right now across 31 files. Each dates its sentence to the day it was written: "[notify] today is one webhook URL, read in one place" stops being true the moment this design lands, while still reading as the premise it rests on. The header-field exemption was also too wide. "**Date:** 2026-09-03, Channel resolved 2026-09-22" smuggled a dated status into the one line the rules did not read; the exemption is now a Date holding only a date and a Status holding only one of the three.
77 lines
3.3 KiB
Markdown
77 lines
3.3 KiB
Markdown
# Review calibration -- decision
|
|
|
|
**Status:** Draft
|
|
**Beads issue:** `archie-core-h019.5` (parent `archie-core-h019`)
|
|
|
|
Decides the half of calibration `docs/prds/adversarial-self-review.md` left
|
|
open: "the adversarial-confirmation mechanism that keeps a finding from
|
|
surfacing on a single reviewer's say-so."
|
|
|
|
## The problem, precisely
|
|
|
|
The shipped contract (`internal/domain/workflow/review_findings.go`, h019.2)
|
|
already makes only a **confirmed** error-level finding block
|
|
(`Blocking() == Verdict == confirmed && Level == error`), and a zero-finding
|
|
review is already distinguishable from one that never ran (`ReviewStatus`).
|
|
What is missing is the other half: a reviewer can report *nothing*
|
|
and say only "I found nothing" in free text. The operator cannot tell "I
|
|
checked the five things that matter and they are clean" from "I did not look."
|
|
That is the failure the epic names -- a findings surface that consumes attention
|
|
without earning trust.
|
|
|
|
## Decisions
|
|
|
|
### 1. "Checked and cleared" becomes structured, not prose
|
|
|
|
`ReviewReport` gains `Checked []ReviewCheck`:
|
|
|
|
```go
|
|
type ReviewCheck struct {
|
|
Property string // what was verified clean, e.g. "nil-safety of Foo callers"
|
|
Evidence string // how it was verified, e.g. "read all 4 callers; each nil-checks"
|
|
}
|
|
```
|
|
|
|
`Evidence` is required, for the same reason `ReviewFinding.FailureScenario` is:
|
|
a check without evidence is an assertion, and the whole point is to separate a
|
|
verified property from an unexamined one. A zero-finding review is credible
|
|
only when it lists what it checked.
|
|
|
|
### 2. Report checks through a typed tool, like findings
|
|
|
|
The reviewer calls a new capture tool `record_checked` (mirroring
|
|
`record_finding`) once per property it verified clean. The same environmental
|
|
argument as findings applies: the reviewer cannot end its turn having silently
|
|
skipped the checklist. `ReviewReport.Summary` stays as the human overview.
|
|
|
|
### 3. Style nits are out of contract
|
|
|
|
A finding must describe wrong behaviour or a crash. Style, formatting, naming,
|
|
and preference nits are explicitly out of contract -- they are the padding that
|
|
trains the operator to skim. Stated in the reviewer prompt and in the
|
|
`ReviewFinding` type's own documentation. Not machine-enforced: deciding "is
|
|
this a style nit" from free text is unreliable, and a wrong rejection is worse
|
|
than a tolerated one. The category enum already has no style bucket; the rule
|
|
closes the gap that would otherwise let one hide under `other`.
|
|
|
|
### 4. Surfacing
|
|
|
|
The PR-body findings section lists the cleared properties alongside findings,
|
|
so "ran and found nothing" arrives with its evidence. (Park reasons keep
|
|
rendering only blocking findings -- what a human needs to act on, not the full
|
|
checklist.)
|
|
|
|
## What this does not decide
|
|
|
|
- Adding a second, independent reviewer pass. The verdict distinction plus the
|
|
recorded checks is the confirmation mechanism; a second model pass is a
|
|
separate, heavier decision and nothing in `h019.5` requires it.
|
|
- How many findings can survive before `MaxSteps` is exhausted (a budget
|
|
question, already satisfied by the per-stage budget).
|
|
|
|
## Packages this touches
|
|
|
|
- `internal/domain/workflow` -- `ReviewCheck`, `ReviewReport.Checked`, validation.
|
|
- `internal/app/agentworker` -- `record_checked` tool, prompt rules.
|
|
- `internal/domain/workflow/review.go` -- render cleared properties on the PR body.
|