Merge pull request #871 from samcharles93/docs/ratify-pr-review-remediation

docs(prds): ratify PR review remediation with amendments
This commit is contained in:
sam authored and GitHub committed 2026-09-19 01:56:41 +10:00
commit 1829020398
1 file changed
+134 -29
+134 -29
View File
@@ -1,11 +1,11 @@
# PR review comment remediation -- decision
**Status:** Proposed
**Date:** 2026-09-10
**Status:** Decided, not yet implemented
**Date:** 2026-09-10, amended and ratified 2026-09-19
**Beads issue:** (filed as `archie-core-…`, parent: `archie-core-7d5u` event sources and typed reactions)
Answers: how archie reacts to a human's review comments on a PR it opened,
and remediates them. Built on two settled decisions — the adversarial review
Answers: how archie reacts to review comments on a PR it opened, from a human
or from another review bot, and remediates them. Built on two settled decisions — the adversarial review
epic's isolation contract (`docs/prds/adversarial-self-review.md`) and the
event-sources decision (`docs/prds/event-sources-and-reactions.md`) — plus the
webhook intake security contract (`docs/prds/webhook-intake-security.md`).
@@ -57,7 +57,12 @@ cursor for both poll and webhook dedup.
tasks calls `ListReviews`/`ListReviewComments` with a persisted `sinceID`
cursor per task. This is the Gitea path and the deployment-without-webhook
path; it must stay first-class, not a legacy fallback (the repo's own
webhook-intake rule).
webhook-intake rule). It is **one shared scan over all `pr_open` tasks**,
not a scan inside each identity's poll loop, and it calls the forge client
the scanned task's own identity owns. This is `reconcilePRs`'s shape
exactly (`internal/daemon/daemon.go`: `Store.OpenPRs` then `forgeFor(&t)`).
A per-identity scan would call one identity's forge client against another
identity's PR.
Both paths emit the **same** typed reaction, so `PublishUnique` dedups them
for free — the `TaskEnvelope.IdempotencyKey` pattern (7d5u decision 4) applied
@@ -65,17 +70,55 @@ to reviews.
### 3. Reaction contract: producer-only, resolves to an existing task
A new `workintake.ReviewCommentEnvelope{Owner, Repo, PRNumber, CommentID,
A new `workintake.ReviewCommentEnvelope{Owner, Repo, PRNumber, Kind, CommentID,
ReviewID, Author, State, Body, Path, Line}`. Idempotency key is
`owner/repo/pr-number/comment-id` — source-independent, so a webhook delivery
and a poll of the same comment collapse. This is a producer-only reaction
`owner/repo/pr-number/kind/id` — source-independent, so a webhook delivery
and a poll of the same comment collapse.
`Kind` (`review` or `comment`) is part of the key because a review ID and a
review-comment ID come from independent forge sequences. Keying on the bare id
lets a review and an inline comment on the same PR collide, silently dropping
one of the two. This is a producer-only reaction
(7d5u decision 3): it never vetoes or mutates in-flight work; the target task
is idle (`pr_open`).
The daemon resolves `PRNumber` → the task that owns it (store lookup by
owner/repo/pr_number). An unmatched PR (not an archie task) is dropped with a
counter — a public webhook must not be able to start an agent against an
arbitrary repository.
The daemon resolves `PRNumber` → the task that owns it. An unmatched PR (not
an archie task) is dropped with a counter — a public webhook must not be able
to start an agent against an arbitrary repository. A PR whose task is merged,
closed or archived is dropped through the same counter: the reaction is only
ever work against a PR archie itself opened *and still owns*.
That lookup does not exist. `pr_number` is written by
`internal/store/store.go` and never read in a `WHERE` clause, so this needs a
`TaskStore` method keyed on `(owner, repo, pr_number)`, a matching
`StateStoreService` RPC (a Go interface method on a store facade without one
is forbidden, `CLAUDE.md`), and an index on those three columns. It is the
authorization boundary, so it is sequenced as its own step below rather than
carried along with the envelope.
**Identity is derived from the resolved task, never carried in the reaction.**
The task row already stores `identity` (`internal/store/store.go`), and it is
the resolved task's identity that selects the forge client, worktree manager,
repo config and `bot_user` for the remediation run and the `ReplyToReview`
reply, through the existing `forgeFor`/`treesFor`/`repoFor` helpers. That is
what decision 5's "the identity that owns the task" means, and it is the only
place identity enters this path.
Identity is deliberately **not** added to the envelope, the idempotency key or
the index:
- A PR number is unique within an `owner/repo`, and the tasks table is keyed
`UNIQUE(owner, repo, issue_number)` with no host or identity column, so
archie already treats `owner/repo` as globally unique across identities. A
`(owner, repo, pr_number)` lookup cannot return a task other than the one
that owns that PR; there is no cross-identity resolution to guard against.
- The webhook producer has no identity to carry: webhook intake refuses to
start when `[[identities]]` is configured (`setupForgeWebhook`,
`internal/app/archied/bootstrap.go`).
- Keying on a field only one of the two producers can populate is precisely
the delivery-source-specific key `CLAUDE.md` forbids, and it would break the
webhook/poll dedup the key exists for. An index column that never appears in
the `WHERE` clause is dead weight.
### 4. Remediation: a `remediate` workflow on the existing branch
@@ -83,9 +126,9 @@ A new workflow that runs when a reaction resolves to an archie-owned PR. It is
not a new branch/PR — it reuses the task's prepared worktree, refreshes it
onto the PR branch (the `worktree.Manager.refresh` idempotency), and:
1. Runs a builder agent whose mission is the review comment (body, path,
line): "Address this review comment with the smallest change; run the
gate." The toolset is the normal builder toolset — edit + gate + (never
1. Runs a builder agent whose mission is the review's actionable comments
(each with body, path, line): "Address these review comments with the
smallest change; run the gate." One agent run per review, per decision 5. The toolset is the normal builder toolset — edit + gate + (never
git; the orchestrator commits/pushes) — not the reviewer's read-only set.
2. Runs the repo gate (`[[repos.gate]]`) and `diff_cap_lines` — the same
enforcement as any other work (webhook-intake-security point 4; no
@@ -100,15 +143,56 @@ recoverable and auditable.
### 5. Routing and lifecycle
- `requested_changes` review → queue one remediation run addressing its
actionable comments.
- `commented` review with actionable inline comments → queue a remediation run
per comment (deduped by comment ID).
A **review** is the unit of remediation, never an individual comment.
- `requested_changes` review → one remediation run addressing every actionable
comment in that review.
- `commented` review with actionable inline comments → one remediation run
addressing every actionable comment in that review.
- `approved` review → no remediation. Record the review state on the task;
do **not** auto-merge — merging stays an operator action (the dashboard/chat
`/approve` precedent). Auto-merge is a separate decision if ever wanted.
- Comments from the bot itself are excluded (the `RepliesAfter` `exclude`
precedent), so archie's own reply does not retrigger it.
One run per review, strictly. A review carrying eight inline comments arrives
as one `pull_request_review` event plus eight `pull_request_review_comment`
events; remediating each separately means eight builder runs, eight gate runs
and eight pushes to answer one review. The comment events are collected
against their parent `ReviewID` and remediated together. A standalone comment
with no parent review is its own unit of one.
Remediation runs for a single task are **serialised**, whatever
`[[repos]].allow_concurrent` says. Two runs against one PR branch would race
on the shared worktree and on the push; `allow_concurrent` is about unrelated
tasks in a repo, not about one task reacting to itself.
#### Whose comments count
Only archie's **own** comments are excluded, matched on the forge identity
that owns the task (`bot_user`, or the identity's `bot_user` under
`[[identities]]`), so its `ReplyToReview` replies do not retrigger it.
Every other author is actionable, including other bots. The external review
bot on this repository is the main source of findings on an archie-opened PR,
and discarding comments because their author is a bot would discard the review
this feature exists to act on. There is no "is a bot" test anywhere in this
path.
(The original draft justified a broader exclusion by citing a `RepliesAfter`
`exclude` precedent. No such mechanism exists in the tree; the citation is
withdrawn.)
#### Bounding the exchange
Because bot comments are actionable, the exchange can run away without a human
ever touching it: archie pushes a remediation, the review bot re-reviews the
new commit, comments, archie remediates again. Nothing in the reaction path
terminates that on its own.
A task therefore carries a remediation round count, bounded by the repo's
existing `max_retries`. On the cap, archie stops remediating, posts one
comment saying it has stopped and why, and parks the task for an operator.
This is a hard stop, not a backoff: the failure it guards against is
unbounded spend, so slowing it down does not address it.
### 6. Security posture
@@ -119,17 +203,38 @@ per-source rate limiting (add when the receiver gains a second event family),
and blast radius bounded by the existing gate/sandbox/worktree enforcement,
**plus** the new archie-owned-task check (decision 3), which is the actual
authorization boundary: a review event only ever causes work against a PR
archie itself opened.
archie itself opened and still owns.
Accepting other bots' comments (decision 5) does not widen that boundary: an
author is never a reason to act, only the owned-task check is. It does widen
the *spend* surface, which the round cap bounds.
### 7. Sequencing
1. `PullRequestReviewReader` + GitHub/Gitea implementations + `ReplyToReview`.
2. `ReviewCommentEnvelope` + idempotency + `PRNumber → task` resolution + the
archie-owned-task guard.
3. `remediate` workflow (worktree refresh + builder + gate + push + reply).
4. Webhook receiver extension (`pull_request_review` / `_comment` events).
5. Poll backstop with the persisted `sinceID` cursor.
6. Tests + `task check`.
1. **The reaction stream.** `ARCHIE_REACTIONS`, under the fan-out
`jetstream.LimitsPolicy` the parent decision requires
(`event-sources-and-reactions.md` §2). `Config.Retention` is already
parameterised for exactly this (`archie-core-7d5u.2`), but no reaction
stream exists, and `ARCHIE_TASKS` carries `archie.task.>` only. Until this
lands, every publish to `archie.reaction.*` fails against no matching
stream. This step is first because nothing after it can be exercised
without it.
2. `PullRequestReviewReader` + GitHub/Gitea implementations + `ReplyToReview`.
*(Landed.)*
3. `ReviewCommentEnvelope` + the kinded idempotency key.
4. **The owned-task guard**: `(owner, repo, pr_number)` lookup, its
`StateStoreService` RPC, its index, and the drop-with-counter path for an
unmatched, merged, closed or archived PR. Nothing may consume a reaction
before this exists.
5. `remediate` workflow: worktree refresh, one run per review, builder, gate,
push, reply, round cap.
6. Webhook receiver extension (`pull_request_review` / `_comment` events).
7. Poll backstop with the persisted `sinceID` cursor.
8. Tests + `task check`.
Steps 6 and 7 are the producers and come last on purpose. Building a producer
before its consumer and its authorization guard yields a path that publishes
into nothing, and becomes an unguarded intake the moment a stream appears.
## Out of scope