[EPIC] Adversarial self-review before archie opens a PR #1348

Closed
opened 2026-10-27 13:40:02 +00:00 by sam · 0 comments
Owner

Archie enforces gates on every PR it opens, but it does not run the adversarial review CLAUDE.md makes mandatory for every change in this repo: a fresh reviewer that did not write the code, told to assume every line is wrong, working a defined checklist, where a surviving finding blocks the work. It is the one discipline the project requires of its contributors and exempts itself from.

Why it matters, with evidence rather than assertion: on 2026-08-20/21 the update-verification change (PR #530) passed task check at every checkpoint -- exit 0, golangci-lint 0 issues, gofumpt clean -- and three adversarial passes still found six real defects, two of them blocking. A 'dev' version placeholder would have reported healthy updates as failures, and an 'unknown' install claim was still being worded as confirmed success. The gates were green throughout and would have stayed green. This is CLAUDE.md's own recorded lesson: tests catch behaviour that breaks, not behaviour that quietly starts meaning something else.

Archie is unusually well placed to do this. It already runs each task in an isolated container and worktree, so it can spawn a reviewer that genuinely never saw the implementation -- not a second opinion from the same context pretending to be fresh, which is the failure mode of every single-context tool. The isolation boundary built for safety is what makes honest self-review possible.

What the operator gets: a PR that arrives with a findings section -- what was checked, what was found, what was fixed, what survived. A surviving finding parks the task as waiting_human instead of opening a PR that looks ready and is not. Review shifts from hunting for bugs to adjudicating findings.

The risk that kills the feature is a reviewer that cries wolf. Findings must be adversarially confirmed before they surface, and the reviewer must be able to report zero findings on properties it checked and cleared. Calibration is the feature, not the finding count.

Acceptance criteria

An implement-workflow task runs an adversarial review after gates pass and before the PR is opened; the reviewer provably cannot see the implementer's context; a surviving confirmed finding blocks PR creation and parks the task; findings and their disposition are visible on the PR and the dashboard.

Close reason

All 8 child issues shipped; adversarial self-review and operator-triggered PR review are production-reachable.


Imported from beads archie-core-h019 · GitHub #552 · type epic · priority P1 · created 2026-08-21 · closed 2026-09-16

Archie enforces gates on every PR it opens, but it does not run the adversarial review CLAUDE.md makes mandatory for every change in this repo: a fresh reviewer that did not write the code, told to assume every line is wrong, working a defined checklist, where a surviving finding blocks the work. It is the one discipline the project requires of its contributors and exempts itself from. Why it matters, with evidence rather than assertion: on 2026-08-20/21 the update-verification change (PR #530) passed task check at every checkpoint -- exit 0, golangci-lint 0 issues, gofumpt clean -- and three adversarial passes still found six real defects, two of them blocking. A 'dev' version placeholder would have reported healthy updates as failures, and an 'unknown' install claim was still being worded as confirmed success. The gates were green throughout and would have stayed green. This is CLAUDE.md's own recorded lesson: tests catch behaviour that breaks, not behaviour that quietly starts meaning something else. Archie is unusually well placed to do this. It already runs each task in an isolated container and worktree, so it can spawn a reviewer that genuinely never saw the implementation -- not a second opinion from the same context pretending to be fresh, which is the failure mode of every single-context tool. The isolation boundary built for safety is what makes honest self-review possible. What the operator gets: a PR that arrives with a findings section -- what was checked, what was found, what was fixed, what survived. A surviving finding parks the task as waiting_human instead of opening a PR that looks ready and is not. Review shifts from hunting for bugs to adjudicating findings. The risk that kills the feature is a reviewer that cries wolf. Findings must be adversarially confirmed before they surface, and the reviewer must be able to report zero findings on properties it checked and cleared. Calibration is the feature, not the finding count. ### Acceptance criteria An implement-workflow task runs an adversarial review after gates pass and before the PR is opened; the reviewer provably cannot see the implementer's context; a surviving confirmed finding blocks PR creation and parks the task; findings and their disposition are visible on the PR and the dashboard. ### Close reason All 8 child issues shipped; adversarial self-review and operator-triggered PR review are production-reachable. --- _Imported from beads `archie-core-h019` · GitHub #552 · type epic · priority P1 · created 2026-08-21 · closed 2026-09-16_
sam added the highepic labels 2026-10-27 13:40:02 +00:00
sam self-assigned this 2026-10-27 13:40:02 +00:00
sam closed this issue 2026-10-27 13:40:02 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sam/archie#1348