chore(beads): close h019.2 findings contract
This commit is contained in:
1 parent
7b35590cb8
commit
170290c284
2 files changed
+2
-1
No files matched your search
@@ -123,3 +123,4 @@
|
||||
{"id":"int-07bcd93041cf024404279b55497e4f4a","kind":"field_change","created_at":"2026-08-24T08:15:48.462601908Z","actor":"Sam Catlow","issue_id":"archie-core-1ysw.8","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Fixed: CommitAll now takes a live context and checks ctx.Err() before open, worktree, staging, status and commit (worktree.go:398-430), so a pre-cancelled context mutates neither index nor HEAD. Proven red against pre-fix code: 'CommitAll() error = nil, want context canceled' for both a cancelled context and an expired deadline. task check green; fresh adversarial review zero findings."}}
|
||||
{"id":"int-997fffb13822c36e05730e69110d7602","kind":"field_change","created_at":"2026-08-24T08:15:48.90789093Z","actor":"Sam Catlow","issue_id":"archie-core-1ysw.9","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Fixed: precedence is now title-first via parseTitlePrefix with a labels fallback and 'feat' default; the strings.TrimSuffix(prefix, \"\") no-op is gone; the dead body parameter is removed from archieBranch and its call site (worktree.go:131); branchSlug's doc now says 60, matching the implementation. go fix later modernised parseTitlePrefix to strings.Cut -- verified equivalent (Cut splits on the first separator; ok mirrors idx != -1). Verified by inspection rather than a red run, since the signature change prevents compiling the tests against pre-fix code. task check green; fresh adversarial review zero findings."}}
|
||||
{"id":"int-01f64138504555374b1349cc30e94c40","kind":"field_change","created_at":"2026-08-24T08:15:59.810530521Z","actor":"Sam Catlow","issue_id":"archie-core-1ysw","extra":{"field":"status","new_value":"closed","old_value":"open","reason":"All 9 children complete. The P0/P1 trust-boundary and correctness defects (.1 RPC task binding, .2 colliding directory keys, .3 untracked cleanup, .4 fail-closed base resolution, .5 three-dot merge-base semantics) were already fixed on main and reconciled 2026-08-24. The remaining four (.6 push result, .7 no-change worktree leak, .8 commit cancellation, .9 branch naming contract) were implemented this session, merged as 474c31d and its two merge commits, task check green, with an independent adversarial review reporting zero findings."}}
|
||||
{"id":"int-df896cababb907b14b094c56b928ebd3","kind":"field_change","created_at":"2026-08-24T09:15:55.715496686Z","actor":"Sam Catlow","issue_id":"archie-core-h019.2","extra":{"field":"status","new_value":"closed","old_value":"in_progress","reason":"Implemented: workflow.ReviewFinding + ReviewReport in internal/domain/workflow/review_findings.go, merged 53bb841. The domain owns the type; gate.Finding was used as a shape precedent only, not reused, so the review contract is not coupled to the Yaegi gate's evolution. Blocking requires Verdict==confirmed AND Level==error, so a PLAUSIBLE finding never blocks however severe it claims to be. ReviewStatus makes 'ran and found nothing' structurally distinct from 'never ran' -- Passed() is false for a zero-value report, and the distinction does not rest on nil-vs-empty-slice. JSON round-trip covered; Line 0 is preserved (no omitempty) for whole-file findings. Review found and fixed a duplicate package doc comment and a redundant ReviewBlocking/BlockingReviewFindings alias pair. task check green. Fresh adversarial review clean across 8 dimensions; I independently mutation-tested the blocking rule: dropping the verdict check fails 5 tests (incl. plausible_error_MUST_NOT_block) and dropping the level check fails 5 more, so the tests genuinely pin both halves."}}
|
||||
+1
-1
@@ -41,7 +41,7 @@
|
||||
{"_type":"issue","id":"archie-core-h019.3","title":"Reviewer runs in a context that provably never saw the implementation","description":"This is the whole feature. A reviewer that shares the implementer's context is a second opinion from the same mind, and it will rationalise the same mistakes -- which is exactly the failure mode CLAUDE.md's 'never self-certified by the implementer' rule exists to prevent.\n\nArchie already has the machinery: each task runs in an isolated container against its own worktree. Launch the reviewer the same way, seeded only with the diff and the originating issue. It must not receive the implementer's conversation, its notes, its reasoning, or its own claims about what it did.\n\nPer 'environmental enforcement over prompt rules', isolation must be a property of construction, not a sentence in the prompt. The agent cannot talk its way out of a context it was never given. Assert it in a test rather than assuming it: a test that would fail if implementer context were threaded through.","acceptance_criteria":"The reviewer is launched with only diff + issue; a test fails if implementer context reaches it; the reviewer's prompt instructs it to assume every line is wrong until proven correct; the mechanism reuses the existing container/worktree isolation rather than inventing a second one.","notes":"BLOCKED 2026-08-24: not implementable as specified. docs/prds/adversarial-self-review.md is marked 'Superseded; requires redesign before implementation' -- superseded by the full-task worker boundary landed in archie-core-q15y. Its mechanism for this bead was a daemon-side second container.Pool.Acquire plus an in-process agentexec path; q15y.2 and q15y.3 removed BOTH call sites, so the isolation mechanism this bead reuses no longer exists. Needs a design decision first: how a fresh reviewer receives a .git-stripped snapshot from INSIDE the existing task-scoped archie-agent worker, without introducing a host executor or a nested container lifecycle. Do not implement against the superseded PRD. The sibling bead h019.2 (findings contract) is unaffected and is being implemented, since it is a pure domain type with no execution dependency.","status":"open","priority":1,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:02:54Z","created_by":"Sam Catlow","updated_at":"2026-08-24T08:58:06Z","external_ref":"https://github.com/samcharles93/archie-core/issues/548","dependencies":[{"issue_id":"archie-core-h019.3","depends_on_id":"archie-core-h019","type":"parent-child","created_at":"2026-08-21T13:02:54Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-h019.3","depends_on_id":"archie-core-h019.1","type":"blocks","created_at":"2026-08-21T13:03:25Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":1,"dependent_count":1,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-h019.4","title":"Wire review as an implement-workflow stage: after gates, before PR open","description":"Sequence matters. The review runs after gates pass -- there is no point reviewing code that does not build or test -- and before the PR is opened, because the point is to not spend the operator's attention on a PR that is not ready.\n\nA surviving CONFIRMED finding must not open a PR. It parks the task as waiting_human with the findings attached, which is the existing lifecycle state for 'a human needs to look at this' and needs no new state. Per CLAUDE.md, a finding that survives blocks closing the work; it does not get waved through.\n\nScope to the implement workflow first. Routing to tdd/bootstrap/feasibility is a later decision -- feasibility in particular produces a document rather than a diff, so 'review the diff' may not even apply.","acceptance_criteria":"An implement task with a clean gate and no confirmed findings opens a PR as it does today; one with a surviving confirmed finding parks as waiting_human with findings recorded and opens no PR; the stage is observable in the task's running(workflow:stage) status; retry after operator action re-runs the review.","status":"open","priority":1,"issue_type":"feature","owner":"sam@catlow.tech","created_at":"2026-08-21T13:02:54Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:13Z","external_ref":"https://github.com/samcharles93/archie-core/issues/549","dependencies":[{"issue_id":"archie-core-h019.4","depends_on_id":"archie-core-h019","type":"parent-child","created_at":"2026-08-21T13:02:54Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-h019.4","depends_on_id":"archie-core-h019.1","type":"blocks","created_at":"2026-08-21T13:03:25Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-h019.4","depends_on_id":"archie-core-h019.2","type":"blocks","created_at":"2026-08-21T13:03:25Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-h019.4","depends_on_id":"archie-core-h019.3","type":"blocks","created_at":"2026-08-21T13:03:26Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":3,"dependent_count":1,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-h019.1","title":"Design: adversarial self-review stage (one-page decision in docs/prds/)","description":"No settled design exists for this in docs/architecture/, so under the scope-discipline rule merged as PR #533 the design comes first -- briefly and decisively -- and is not finished until a diff follows it.\n\nDecide and record, in one page under docs/prds/: where the reviewer runs (reuse the per-task container pool, or a separate short-lived container); what it is given (diff plus originating issue, and explicitly what it must NOT be given); the findings contract and who owns the type; what a surviving finding does to the task lifecycle; and how the reviewer's model/effort is configured.\n\nConstraints from the repo that the design must respect: dependency direction is cmd -\u003e app -\u003e {domain, infrastructure}, so the reviewer contract belongs to the workflow domain and the execution to infrastructure/app. Agent execution is a data boundary. The model never runs git. Prefer environmental enforcement over prompt rules -- 'the reviewer cannot see the implementation' must be a property of how it is launched, not an instruction in its prompt.\n\nDo not write an architecture document, migration plan or review campaign. One page that decides.","acceptance_criteria":"A one-page decision exists under docs/prds/ answering: reviewer isolation mechanism, inputs and forbidden inputs, findings contract owner, lifecycle effect of a surviving finding, and model/effort configuration. It decides rather than surveys, and cites the packages it will touch.","status":"closed","priority":1,"issue_type":"decision","owner":"sam@catlow.tech","created_at":"2026-08-21T13:02:39Z","created_by":"Sam Catlow","updated_at":"2026-08-24T07:12:05Z","started_at":"2026-08-22T09:30:19Z","closed_at":"2026-08-24T07:12:05Z","close_reason":"One-page decision written at docs/prds/adversarial-self-review.md: reviewer runs in a second workspace (container or in-process) pointed at a .git-stripped snapshot of the reviewed commit, never the implementer's worktree; findings contract is workflow.ReviewFinding, owned by internal/domain/workflow, reported via a structured capture tool; a surviving error-level finding sets StatusParked (not WaitingHuman -- epic wording named two different states, this resolves which); model/effort reuse the existing Models[role] map and per-stage budget override, gated by a new opt-in Repo.ReviewEnabled so h019.4 doesn't gate every repo on an uncalibrated reviewer before h019.5 lands. [Closure restored 2026-08-24: originally closed 2026-08-22, lost when the beads DB was rebuilt (.beads.backup-pre-recovery).]","external_ref":"https://github.com/samcharles93/archie-core/issues/550","metadata":null,"dependencies":[{"issue_id":"archie-core-h019.1","depends_on_id":"archie-core-h019","type":"parent-child","created_at":"2026-08-21T13:02:39Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":0,"dependent_count":5,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-h019.2","title":"Findings contract: a structured review result the workflow can act on","description":"The reviewer's output must be data the workflow can branch on, not prose a human has to read to find out whether the PR is safe to open. Prose is what makes a review advisory; structure is what lets a finding block.\n\nDefine the finding type in the workflow domain (the domain declares the contract it needs; it never names who implements it). At minimum a finding carries: file and line, a one-sentence defect, a concrete failure scenario (inputs/state -\u003e wrong output), a verdict distinguishing CONFIRMED from PLAUSIBLE, and a disposition once acted on (fixed / skipped / no_change_needed).\n\nThe CONFIRMED/PLAUSIBLE split is load-bearing and must not be collapsed into a single boolean: only confirmed findings may block, or the feature degrades into crying wolf and the operator learns to skim past it.","acceptance_criteria":"A finding type exists in internal/domain/workflow with file, line, summary, failure scenario, verdict and disposition; it is serialisable across the agent-execution data boundary; a review result with zero findings is representable and distinct from a review that did not run.","status":"in_progress","priority":1,"issue_type":"feature","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-21T13:02:39Z","created_by":"Sam Catlow","updated_at":"2026-08-24T08:57:20Z","started_at":"2026-08-24T08:57:20Z","lease_expires_at":"2026-08-24T09:02:20Z","heartbeat_at":"2026-08-24T08:57:20Z","external_ref":"https://github.com/samcharles93/archie-core/issues/551","dependencies":[{"issue_id":"archie-core-h019.2","depends_on_id":"archie-core-h019","type":"parent-child","created_at":"2026-08-21T13:02:39Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-h019.2","depends_on_id":"archie-core-h019.1","type":"blocks","created_at":"2026-08-21T13:03:24Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":1,"dependent_count":3,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-h019.2","title":"Findings contract: a structured review result the workflow can act on","description":"The reviewer's output must be data the workflow can branch on, not prose a human has to read to find out whether the PR is safe to open. Prose is what makes a review advisory; structure is what lets a finding block.\n\nDefine the finding type in the workflow domain (the domain declares the contract it needs; it never names who implements it). At minimum a finding carries: file and line, a one-sentence defect, a concrete failure scenario (inputs/state -\u003e wrong output), a verdict distinguishing CONFIRMED from PLAUSIBLE, and a disposition once acted on (fixed / skipped / no_change_needed).\n\nThe CONFIRMED/PLAUSIBLE split is load-bearing and must not be collapsed into a single boolean: only confirmed findings may block, or the feature degrades into crying wolf and the operator learns to skim past it.","acceptance_criteria":"A finding type exists in internal/domain/workflow with file, line, summary, failure scenario, verdict and disposition; it is serialisable across the agent-execution data boundary; a review result with zero findings is representable and distinct from a review that did not run.","status":"closed","priority":1,"issue_type":"feature","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-21T13:02:39Z","created_by":"Sam Catlow","updated_at":"2026-08-24T09:15:56Z","started_at":"2026-08-24T08:57:20Z","closed_at":"2026-08-24T09:15:56Z","close_reason":"Implemented: workflow.ReviewFinding + ReviewReport in internal/domain/workflow/review_findings.go, merged 53bb841. The domain owns the type; gate.Finding was used as a shape precedent only, not reused, so the review contract is not coupled to the Yaegi gate's evolution. Blocking requires Verdict==confirmed AND Level==error, so a PLAUSIBLE finding never blocks however severe it claims to be. ReviewStatus makes 'ran and found nothing' structurally distinct from 'never ran' -- Passed() is false for a zero-value report, and the distinction does not rest on nil-vs-empty-slice. JSON round-trip covered; Line 0 is preserved (no omitempty) for whole-file findings. Review found and fixed a duplicate package doc comment and a redundant ReviewBlocking/BlockingReviewFindings alias pair. task check green. Fresh adversarial review clean across 8 dimensions; I independently mutation-tested the blocking rule: dropping the verdict check fails 5 tests (incl. plausible_error_MUST_NOT_block) and dropping the level check fails 5 more, so the tests genuinely pin both halves.","external_ref":"https://github.com/samcharles93/archie-core/issues/551","dependencies":[{"issue_id":"archie-core-h019.2","depends_on_id":"archie-core-h019","type":"parent-child","created_at":"2026-08-21T13:02:39Z","created_by":"Sam Catlow","metadata":"{}"},{"issue_id":"archie-core-h019.2","depends_on_id":"archie-core-h019.1","type":"blocks","created_at":"2026-08-21T13:03:24Z","created_by":"Sam Catlow","metadata":"{}"}],"dependency_count":1,"dependent_count":3,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-h019","title":"[EPIC] Adversarial self-review before archie opens a PR","description":"Archie enforces gates on every PR it opens, but it does not run the adversarial review CLAUDE.md makes mandatory for every change in this repo: a fresh reviewer that did not write the code, told to assume every line is wrong, working a defined checklist, where a surviving finding blocks the work. It is the one discipline the project requires of its contributors and exempts itself from.\n\nWhy it matters, with evidence rather than assertion: on 2026-08-20/21 the update-verification change (PR #530) passed task check at every checkpoint -- exit 0, golangci-lint 0 issues, gofumpt clean -- and three adversarial passes still found six real defects, two of them blocking. A 'dev' version placeholder would have reported healthy updates as failures, and an 'unknown' install claim was still being worded as confirmed success. The gates were green throughout and would have stayed green. This is CLAUDE.md's own recorded lesson: tests catch behaviour that breaks, not behaviour that quietly starts meaning something else.\n\nArchie is unusually well placed to do this. It already runs each task in an isolated container and worktree, so it can spawn a reviewer that genuinely never saw the implementation -- not a second opinion from the same context pretending to be fresh, which is the failure mode of every single-context tool. The isolation boundary built for safety is what makes honest self-review possible.\n\nWhat the operator gets: a PR that arrives with a findings section -- what was checked, what was found, what was fixed, what survived. A surviving finding parks the task as waiting_human instead of opening a PR that looks ready and is not. Review shifts from hunting for bugs to adjudicating findings.\n\nThe risk that kills the feature is a reviewer that cries wolf. Findings must be adversarially confirmed before they surface, and the reviewer must be able to report zero findings on properties it checked and cleared. Calibration is the feature, not the finding count.","acceptance_criteria":"An implement-workflow task runs an adversarial review after gates pass and before the PR is opened; the reviewer provably cannot see the implementer's context; a surviving confirmed finding blocks PR creation and parks the task; findings and their disposition are visible on the PR and the dashboard.","status":"open","priority":1,"issue_type":"epic","owner":"sam@catlow.tech","created_at":"2026-08-21T13:02:17Z","created_by":"Sam Catlow","updated_at":"2026-08-22T09:25:15Z","external_ref":"https://github.com/samcharles93/archie-core/issues/552","dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-3jag","title":"Chat/task agent persona is overconfident: presents unverified claims as verified, fabricates infrastructure details","description":"The chat assistant (archie, running via the Telegram/webui gateway) repeatedly\npresents unverified or fabricated information with false confidence, and only\nretracts it when directly challenged. This is a persona/system-prompt problem,\nnot a model quality problem -- reproduced on more than one underlying model.\n\nTwo concrete instances from a live Telegram transcript (2026-08-15):\n\n1. Fabricated infrastructure: asked \"are logs going anywhere else?\", the\n assistant invented an entire second log-shipping subsystem (\"Hermes\n gateway\") with specific fake file paths\n (/home/sam/.hermes/profiles/local-agent/logs/) that do not exist anywhere\n in archie-core, and stated it as verified fact (\"I confirmed the web UI\n token and task lifecycle messages are there\"). When challenged (\"Is there\n something in the codebase which tells you you're Hermes?\"), it admitted\n the paths were invented and carried forward from unrelated, unverified\n context.\n\n2. Overclaimed verification: asked \"Why did that last task fail?\", the\n assistant answered with a bulleted list explicitly labelled \"Verified\n details\" (attempt count, iteration count, token count, exact model name).\n When pressed with \"You checked it?\", it admitted the model name and\n change count were \"overstated\" -- i.e. it labelled unverified specifics as\n verified the first time, and only corrected itself under direct\n challenge.\n\nThe common shape: confident, specific claims presented as fact/verified\nfirst; a walked-back, hedged correction only appears after the user\nexplicitly challenges it. This erodes trust in every claim the assistant\nmakes about system state, since the default posture is confident assertion\nrather than \"I have/haven't actually checked this.\"","notes":"Root cause is believed to be the persona/system-prompt layer (internal/gateway/ BuildSystemPrompt + persona templates), not model selection -- reproduced across models per the reporter. Needs a system-prompt-level fix enforcing 'never label something verified unless it was actually checked this turn' and 'never state unconfirmed system details as fact.' Sam is reviewing the persona/prompt separately; this bead tracks the finding so it isn't lost.","status":"closed","priority":1,"issue_type":"bug","assignee":"Sam Catlow","owner":"sam@catlow.tech","created_at":"2026-08-15T02:25:37Z","created_by":"Sam Catlow","updated_at":"2026-08-20T12:55:18Z","started_at":"2026-08-20T12:50:34Z","closed_at":"2026-08-20T12:55:18Z","close_reason":"Fixed in fix/persona-overconfidence (fba80e3): reworded core rule 1 and added rules 8/9 to the chat agent system prompt enforcing this-turn verification before labelling claims verified/confirmed.","external_ref":"https://github.com/samcharles93/archie-core/issues/524","dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||
{"_type":"issue","id":"archie-core-1786756970855-1-72466b33","title":"worktree: retry fails on a dirty worktree left by an interrupted prior attempt","description":"## Problem\n\n`Manager.refresh` (`internal/worktree/worktree.go`) checks out the task's branch without `Force` before hard-resetting to base. If a prior attempt left uncommitted local changes in the worktree — a killed container, or a stage that edits files before ever reaching a commit — go-git's default checkout (`MergeReset` mode) can refuse to reconcile them, even when HEAD is already on the correct branch and the target commit hasn't moved.\n\nThat failure was then compounded by the fallback: the code assumed any checkout failure meant \"the branch doesn't exist yet in this worktree\" and retried with `Create: true`, which fails differently (and much more confusingly) since the branch *does* exist:\n\n```\ncheckout branch feat/1000000000000000-fix-github-issue-514-telegram-response-formatting-concatenat:\na branch named \"refs/heads/feat/1000000000000000-fix-github-issue-514-telegram-response-formatting-concatenat\" already exists\n```\n\n## Evidence\n\nReproduced directly against the actual broken worktree on a live deployment: `git status` showed a modified tracked file and an untracked file left over from an interrupted attempt, HEAD already correctly on the task branch. Retrying the task hit exactly the error above and parked again with the same unhelpful message, on every retry, since nothing about the state changes between attempts.\n\n## Fix\n\nAdd `Force: true` to both checkout attempts in `refresh`. This is safe unconditionally: `refresh` hard-resets to the base commit a few lines later regardless of what checkout leaves behind, so there is no scenario where discarding the working tree's local state here is wrong.\n\nRegression test: `TestPrepareRecoversFromADirtyWorktreeOnRetry` reproduces the exact production error (modified tracked file + untracked file, branch already checked out) and confirms retry now succeeds and correctly restores the tracked file to its base content.","status":"closed","priority":1,"issue_type":"bug","created_at":"2026-08-15T01:22:16Z","updated_at":"2026-08-15T01:22:38Z","closed_at":"2026-08-15T01:22:39Z","external_ref":"https://github.com/samcharles93/archie-core/issues/523","source_system":"github:https://github.com/samcharles93/archie-core/issues/523:523","metadata":null,"labels":["bug"],"dependency_count":0,"dependent_count":0,"comment_count":0}
|
||||
|
||||
Reference in new issue
Block a user