From 90bbc38436574bf0517bab88ea20cdf6a4c9d955 Mon Sep 17 00:00:00 2001 From: Sam Catlow Date: Fri, 18 Sep 2026 12:37:51 +1000 Subject: [PATCH 1/4] docs(skills): re-verify every .agents skill against the current tree All ten `archie-*` skills carried claims that no longer matched the checkout. Re-verified each against code, git history, and the shipped configuration, then corrected the drift. Structural changes the skills had not caught up with: - Composition root moved: `cmd/archied/main.go` is a ~30-line dispatcher into `internal/app/archied.Run()` with wiring in `bootstrap.go`. The skills still described `cmd/archied` as the wiring site, which also silently zeroed the diagnostics delivery snapshot. - Config decoding moved out of `internal/config` into `internal/infrastructure/configuration`. `config.Load` / `config.LoadDir` no longer exist; the loader is `configuration.New` + `Loader.Resolve`. - Task contract moved to `internal/domain/storecontract`, with `internal/store/interface.go` holding aliases only. The task type is `internal/domain/workflow/task.Task`. - `internal/workflow` became `internal/domain/workflow`; `internal/storerpc` became gRPC `internal/infrastructure/staterpc`; `internal/policy` does not exist. - `Store.Transition` now guards on `from` and writes status plus history in one transaction. Three skills still described the old non-atomic behavior as current and open. - `internal/domain/{messaging,workflow,workintake}`, `internal/eventbus`, `internal/app`, and `internal/infrastructure` now exist. `internal/domain/{agent,identity}` and `internal/policy` remain target-only. Tooling and environment facts: - Go 1.27.0 (root and tools modules), golangci-lint 2.13.2, staticcheck 2026.2.1; snapshot dates and HEAD moved to 2026-09-18 / 95aae832. - Re-ran the bundled diagnostics and re-baselined its snapshot table. Confirmed 78.6% store coverage, a clean race run, and zero dupl findings. - Corrected the claim that no `.dockerignore` exists; it does. - Replaced skill script paths that pointed at the untracked `.claude/skills` symlink with the canonical tracked `.agents/skills` location. - Fixed the delivery-snapshot comment's wrong commit hash for the Gitea workflow removal (dd9bddc2, not dd9bddc). Findings the verification closed: - The VitePress site and `.github/workflows/docs.yml` were removed (de150f20); the Gitea workflow was dropped (dd9bddc2). - The changelog `vX.Y.Z`-vs-`X.Y.Z` contradiction is settled, not open: both changelogs and the CI gate agree on `## []`. - Tracked `docs/node_modules` and the skill-script test regression are settled, not current. - The six unmerged `origin/fix/*` and `origin/feat/*` candidate branches no longer exist. Also regenerates `docs/data/generated/contracts.json`, which the verification found stale since a825db28: docsgen emits 11 schemas now, and `MediaAttachment.raw` plus an `AgentExecutionResult.required` field were missing. `cmp` passes after regeneration. Verified with `task check` (green). --- .../archie-architecture-contract/SKILL.md | 66 +++++++++++-------- .agents/skills/archie-build-and-env/SKILL.md | 40 ++++++----- .../skills/archie-codebase-discovery/SKILL.md | 59 +++++++++-------- .../skills/archie-config-and-flags/SKILL.md | 34 +++++----- .../skills/archie-debugging-playbook/SKILL.md | 62 +++++++++-------- .../archie-diagnostics-and-tooling/SKILL.md | 36 +++++----- .../scripts/delivery-snapshot.sh | 2 +- .../scripts/package-shape.sh | 2 +- .../skills/archie-docs-and-writing/SKILL.md | 42 ++++++------ .../skills/archie-domain-reference/SKILL.md | 57 +++++++++------- .../archie-failure-archaeology/SKILL.md | 17 +++-- .../skills/archie-run-and-operate/SKILL.md | 26 +++++--- docs/data/generated/contracts.json | 9 ++- 13 files changed, 251 insertions(+), 201 deletions(-) diff --git a/.agents/skills/archie-architecture-contract/SKILL.md b/.agents/skills/archie-architecture-contract/SKILL.md index 0c894fddc..b69c33121 100644 --- a/.agents/skills/archie-architecture-contract/SKILL.md +++ b/.agents/skills/archie-architecture-contract/SKILL.md @@ -22,14 +22,16 @@ owner and one production path for each behavior. | Label | Meaning | Authority | | --- | --- | --- | -| **CURRENT** | Behavior present in live code and production composition as of 2026-07-28. | Code, tests, and `cmd/archied/main.go` wiring | +| **CURRENT** | Behavior present in live code and production composition as of 2026-09-18. | Code, tests, and `internal/app/archied` wiring (`cmd/archied/main.go` is a thin dispatcher) | | **APPROVED TARGET** | Direction an implementation must approach, but which may not exist on disk. | `docs/prds/01-project-management.md` and documents whose status says approved | | **OPEN** | A migration or API decision no implementer may invent locally. | `docs/architecture/migration-decisions.md` and documents marked in progress or under design | > **Do not implement target prose as though it describes current code.** > Before treating a target package as available, prove that it exists and is -> used from a production entry point. `internal/domain/` does not exist in the -> CURRENT tree as of 2026-07-28. +> used from a production entry point. As of 2026-09-18 `internal/domain/`, +> `internal/eventbus/`, `internal/infrastructure/`, and `internal/app/` exist; +> `internal/policy/` does not, and `internal/domain/{agent,identity}` remain +> APPROVED TARGET only. Resolve apparent disagreement in this order: @@ -43,24 +45,31 @@ Resolve apparent disagreement in this order: Route to `archie-domain-reference` for glossary and `archie-config-and-flags` for config enumeration. +Two skills this document routes to are parked outside this checkout, under +`/work/apps/archie-skills-parked/`: `archie-technical-accountability` and +`archie-architecture-planning-campaign`. If your session has not loaded them, +treat each routing target as a manual escalation step rather than a live skill. +Also: `cmd/archied/main.go` is a thin dispatcher; the wiring lives in +`internal/app/archied/`. + ## Know the CURRENT system Do not simplify these facts into the target model: -| Concern | CURRENT runtime truth on 2026-07-28 | Evidence | +| Concern | CURRENT runtime truth on 2026-09-18 | Evidence | | --- | --- | --- | -| Composition | `cmd/archied/main.go` performs substantial construction and wiring. | `cmd/archied/main.go`; current `internal/*` tree | -| Durable work | `store.Task` combines forge/chat source data, workflow position, output, retry, identity, and PR state. | `internal/store/store.go` | -| Workflow | `internal/workflow` routes a `store.Task` into a `map[string]Workflow`, mutates a broad `TaskContext` through sequential stages. | `internal/workflow/workflow.go` | +| Composition | `cmd/archied/main.go` dispatches to `internal/app/archied.Run()`, which performs construction and wiring in `bootstrap.go`. | `cmd/archied/main.go`; `internal/app/archied/` | +| Durable work | `task.Task` combines forge/chat source data, workflow position, output, retry, identity, and PR state. | `internal/domain/workflow/task/task.go`; `internal/domain/storecontract/storecontract.go` | +| Workflow | `internal/domain/workflow` routes a `task.Task` into a `map[string]Workflow`, mutates a broad `TaskContext` through sequential stages. | `internal/domain/workflow/workflow.go` | | Agent-stage boundary | Versioned `agentexec.Request`/`Result` carries one autonomous stage; validation correlates task, attempt, and stage. | `internal/agentexec/protocol.go` | | Container handoff | `internal/taskrun` carries entire task plus `config.Repo` and `config.TaskConfig`; `internal/app/agentworker` owns routing and sequencing while its NATS adapter owns wire mechanics. | `internal/taskrun/taskrun.go`; `internal/app/agentworker/`; `internal/infrastructure/agenttransport/nats/` | -| Git ownership | Daemon-owned worktree operations clone, commit, push, diff, and clean up. | `internal/worktree/`; `internal/workflow/steps.go`; `ARCHITECTURE.md` | -| Enforcement | Gates, read-only/protected paths, TDD inverted test gates, and diff caps are represented outside prompt prose. | `internal/agentexec/inprocess.go`; `internal/workflow/{agent,tdd,steps}.go` | -| Configuration | `internal/config.Config`, `IdentityConfig`, `Repo`, and task snapshots mix input decoding with runtime concerns. | `internal/config/config.go` | -| Identity | Configured names and bot usernames act as process-local identity keys. There is no durable Identity domain. | `internal/daemon/daemon.go`; `internal/store/store.go`; `internal/gateway/session.go` | +| Git ownership | Daemon-owned worktree operations clone, commit, push, diff, and clean up. | `internal/worktree/`; `internal/domain/workflow/steps.go`; `ARCHITECTURE.md` | +| Enforcement | Gates, read-only/protected paths, TDD inverted test gates, and diff caps are represented outside prompt prose. | `internal/gate/`; `internal/domain/workflow/{agent,tdd,steps}.go` | +| Configuration | `internal/config.Config`, `IdentityConfig`, `Repo`, and task snapshots mix input decoding with runtime concerns. Decoding itself lives in `internal/infrastructure/configuration`. | `internal/config/config.go`; `internal/infrastructure/configuration/` | +| Identity | Configured names and bot usernames act as process-local identity keys. There is no durable Identity domain. | `internal/daemon/daemon.go`; `internal/domain/storecontract/storecontract.go`; `internal/gateway/session.go` | | Messaging | `internal/gateway` contains both the `Message` struct flow and richer `MessageEvent`; channel routing and task control are mixed with conversational behavior. | `internal/gateway/{gateway,messageevent,tasks}.go`; `internal/channels/` | | Plugins | `plugin.Plugin` exposes only `Name` and `Version`; current loading interprets operator-installed Go files with Yaegi. | `internal/plugin/plugin.go` | -| Processes | Agent modes include in-process, subprocess, and NATS. Optional containers add a distinct boundary with known gaps. | `internal/config/config.go`; `cmd/archied/main.go`; `ARCHITECTURE.md` | +| Processes | Agent modes include in-process, subprocess, and NATS. Optional containers add a distinct boundary with known gaps. | `internal/config/config.go`; `internal/app/archied/`; `ARCHITECTURE.md` | ### Interpret the current lifecycle honestly @@ -73,24 +82,26 @@ parked, dead, rejected, closed_wont_do The intended flow includes queue/claim, workflow stages, human wait/resume, PR reconciliation, parking/retry/exhaustion, and crash recovery. Read -`internal/store/store.go`, `internal/daemon/daemon.go`, and -`internal/workflow/workflow.go` before changing a transition. +`internal/domain/workflow/task/task.go`, `internal/daemon/daemon.go`, and +`internal/domain/workflow/workflow.go` before changing a transition. -Current `Store.Transition(ctx, taskID, from, to, detail)` does not include -`from` in the update predicate; updates task state and inserts transition -history in separate statements; can accept an illegal/stale transition or leave -state and history inconsistent; is called from paths that sometimes discard its -error. Treat atomic state plus event/history persistence as an OPEN migration -design. +`Store.Transition(ctx, taskID, from, to, detail)` now guards on `from` +(`WHERE id=? AND status=?`, returning `store.ErrStaleTransition` on mismatch) +and runs the status update plus the transition-history insert in one +transaction (`internal/store/store.go`). The sentinel crosses the State Store +gRPC boundary via `mapError`/`unmapError` in +`internal/infrastructure/staterpc/values.go`; its canonical message string is +part of the wire contract. Earlier non-atomic behavior is the settled fix, +not an open design. ## Preserve the load-bearing invariants | Invariant | Required action | Status and source | | --- | --- | --- | | The model never runs git | Keep clone, branch, commit, push, diff, and cleanup in deterministic workspace/worktree steps. | CURRENT: `ARCHITECTURE.md`; `internal/worktree/` | -| Deterministic constraints outrank prompts | Enforce gates, protected paths, read-only stages, test protection, and change caps outside model instructions. | CURRENT: `ARCHITECTURE.md`; `internal/agentexec/inprocess.go`; `internal/workflow/` | +| Deterministic constraints outrank prompts | Enforce gates, protected paths, read-only stages, test protection, and change caps outside model instructions. | CURRENT: `ARCHITECTURE.md`; `internal/gate/`; `internal/domain/workflow/` | | Agent execution is a data boundary | Send bounded, versioned input; validate correlated output before applying it. | CURRENT foundation: `internal/agentexec/protocol.go`; `ARCHITECTURE.md` | -| Every workflow ends explicitly | Preserve terminal outcome or visible park. Preserve crash recovery of interrupted `running` work. | CURRENT: `internal/workflow/workflow.go`; `internal/store/store.go` | +| Every workflow ends explicitly | Preserve terminal outcome or visible park. Preserve crash recovery of interrupted `running` work. | CURRENT: `internal/domain/workflow/workflow.go`; `internal/store/store.go` | | Generic plugins stay metadata-only | Keep `plugin.Plugin` at `Name()`/`Version()`. Behavior on typed capability interface with owning `Registry`/`Manager`. | CURRENT-enforced rule: `internal/plugin/architecture_test.go` | | Capability engines own lifecycle | Give resource-owning engines explicit start/health/stop semantics and failure isolation. | APPROVED rule: `ARCHITECTURE.md#plugin-engine-rule-strict` | | Trust is explicit | Treat trusted operator-installed in-process code, repository code, out-of-process integrations, and container-isolated code as different trust classes. | APPROVED rule: `ARCHITECTURE.md#plugin-engine-rule-strict` | @@ -190,7 +201,7 @@ Source: `docs/architecture/configuration.md`. | Generic plugin identity/discovery/compatibility/lifecycle | `internal/domain/plugin` | Keep generic contract metadata-only. | | Capability-specific plugin behavior | Owning domain's typed registry/manager | Never add behavior to `plugin.Plugin`. | | Broker-neutral publication/subscription mechanics | `internal/eventbus` | Keep message meaning and schemas with their domain. | -| Shared policy evaluation/evidence mechanics | `internal/policy` | Keep policy vocabulary, evaluators, and consequences with consuming domain. | +| Shared policy evaluation/evidence mechanics | No current package. `internal/policy` was the planned location; as of 2026-09-18 policy lives inside playbook bindings (`playbooks/`, `internal/domain/eda/playbook`) and `internal/gate/`. | Keep policy vocabulary, evaluators, and consequences with consuming domain. | | NATS, SQLite, forge, config decoding, external SDK adapters | `internal/infrastructure/` | Implement a narrow owner-defined contract; translate at boundary. | | Construction, cross-domain connection, startup/shutdown | `internal/app/archied` or `internal/app/agentworker` | Keep `cmd/*` thin; define health and shutdown order. | | Deploy assembly | `deployments/` | Do not move application behavior into deployment files. | @@ -198,17 +209,17 @@ Source: `docs/architecture/configuration.md`. ## Confront known weak points -| Weak point on 2026-07-28 | Required response | +| Weak point on 2026-09-18 | Required response | | --- | --- | | `TaskContext` is a mutable service bag with config, services, policy, scratch state. | Design narrow step inputs and staged removal. | | Workflow lifecycle split across store, daemon, workflow, gateway, worker. | Trace every writer and reader before changing status. | -| State changes and transition history non-atomic; `from` is advisory. | Treat as architecture migration, not local fix. | +| State changes and transition history were non-atomic and `from` was advisory — **settled** in `internal/store`: `Transition` guards on `status=?` and writes status plus history in one transaction. | Keep every status writer behind `Transition`; do not reintroduce a raw `UPDATE tasks SET status`. | | In-process and container paths put orchestration on different sides of process boundary. | Decide which path survives before extending both. | | Identity uses strings; task uniqueness/NATS dedup omit identity. | Don't promise multi-identity ownership until canonical IDs migrate. | | Container RPC uses root forge/worktree, not task owner's identity. | Known trust/correctness defect — not identity-isolated. | | Missing `task.json` makes `archie-agent` join shared queue. | Fail closed before treating as isolated. | | Subprocess uses daemon user and host access. | Not a security boundary. | -| Gateway duplicates lifecycle strings, competing message contracts. | No third contract or direct lifecycle mutation path. | +| Gateway formerly duplicated lifecycle strings, competing message contracts. | No third contract or direct lifecycle mutation path; status constants live in `internal/domain/workflow`. | | Global config and `IdentityConfig` combine unrelated owners. | Don't rename global structure or add `Extra` map. | | Memory has fixed scope requirements but OPEN final placement/migration. | Preserve scope isolation; focused design before moving. | | Tests can make unused/duplicate code appear legitimate. | Prove production reachability from composition. | @@ -231,4 +242,5 @@ gopls references path/to/file.go:: Use `gopls references` or another AST-aware reference query when interfaces, embedding, generated symbols, or same-named methods make text search ambiguous. -Trace from `cmd/archied/main.go`. +Trace from `cmd/archied/main.go` into `internal/app/archied.Run()`; the wiring +itself is in `internal/app/archied/bootstrap.go`. diff --git a/.agents/skills/archie-build-and-env/SKILL.md b/.agents/skills/archie-build-and-env/SKILL.md index fe72c89c2..a62610dce 100644 --- a/.agents/skills/archie-build-and-env/SKILL.md +++ b/.agents/skills/archie-build-and-env/SKILL.md @@ -9,7 +9,7 @@ Recreate the environment as independently verified surfaces. Do not turn a passing root-module command into a claim that the tools module, documentation site, race detector, linter, or container images also passed. -All volatile observations are snapshots from **2026-07-28**. +All volatile observations are snapshots from **2026-09-18** (HEAD `2e1e1549`). ## Name the surfaces @@ -42,17 +42,17 @@ git --version; gpg --version ``` Classify a missing command as an environment prerequisite. Bootstrap (verified -2026-07-28): Go ≥ 1.26.3, Task ≥ 3.x, gofumpt (unpinned), golangci-lint v2 +2026-09-18): Go ≥ 1.27.0, Task ≥ 3.x, gofumpt (unpinned), golangci-lint v2 (unpinned), Node 24.x for the `ui/` frontend. | Surface | Repository declaration | Installed snapshot | Interpretation | |---|---|---|---| -| Runtime Go | `go 1.26.3` in `go.mod` | Go 1.26.5, linux/amd64 | Requires at least declared Go level. | -| Tools Go | `go 1.26.4` in `tools/go.mod` | Same Go 1.26.5 binary | Toolchain must satisfy both modules. | +| Runtime Go | `go 1.27.0` in `go.mod` | Go 1.27.0, linux/amd64 | Requires at least declared Go level. | +| Tools Go | `go 1.27.0` in `tools/go.mod` | Same Go 1.27.0 binary | Toolchain must satisfy both modules. | | Task | Taskfile schema `version: "3"` | Task 3.48.0 | Installed version is environment fact. | -| gofumpt | Used by `task fmt`; `@latest` install | v0.7.0 | Unpinned. | -| golangci-lint | v2 config in `.golangci.yml`; `@latest` install | 2.12.2 | Use v2 CLI; exact release unpinned. | -| Node | `ui/` frontend build | 24.18.0 | No `engines`/`packageManager` field. Docs need no Node. | +| gofumpt | Used by `task fmt`; `@latest` install | v0.7.0 (built with go1.26.5) | Unpinned. | +| golangci-lint | v2 config in `.golangci.yml`; `@latest` install | 2.13.2 | Use v2 CLI; exact release unpinned. | +| Node | `ui/` frontend build | 24.18.1 / npm 11.8.0 | No `engines`/`packageManager` field. Docs need no Node. | | Containers | Compose commands in `Taskfile.yml` | Podman-backed, unusable in this sandbox | Verify CLI, Compose plugin, daemon/socket separately. | ## Prepare writable caches in a restricted sandbox @@ -147,8 +147,9 @@ go -C tools run -mod=readonly ./docsgen --repo-root .. --out /tmp/archie-core-co cmp --silent docs/data/generated/contracts.json /tmp/archie-core-contracts.json ``` -2026-07-28 run wrote 10 schemas; comparison passed. Planned `docsgen all` and -`docsgen check` do **not** exist yet. +2026-09-18 run wrote 11 schemas. The committed output was stale since +`4b340d2b` and was regenerated the same day; `cmp` now passes. Planned +`docsgen all` and `docsgen check` do **not** exist yet. ## Documentation needs no build @@ -172,9 +173,12 @@ rg -n '^(FROM|[[:space:]]*image:)|@latest|:latest|setup_24.x' \ Dockerfile Dockerfile.archied docker-compose.yml ``` -No `.dockerignore` exists. Both Dockerfiles run `COPY . .`. Floating inputs: Go -builder tags omit patch/digest; Ubuntu base images omit digests; agent installs -multiple `@latest` Go tools; Node from moving `setup_24.x` channel. +`.dockerignore` exists (421 bytes) and excludes `.git`, `.task`, `.claude`, +`.codex`, `.crew`, `.agents`, `.references`, `bin/`, `dist/`, `docs/`, +`node_modules/`, root-built `archied`/`archie-agent`, and local config. Both +Dockerfiles still run `COPY . .` on top of it. Floating inputs: Go builder tags +omit patch/digest; Ubuntu base images omit digests; agent installs multiple +`@latest` Go tools; Node from moving `setup_24.x` channel. `task docker-build` needs registry/network and functioning Compose. `task docker-up` starts the Compose-managed NATS service and binds ports @@ -207,10 +211,10 @@ env GIT_CONFIG_GLOBAL=/dev/null go test ./internal/worktree/... ./internal/workt | Focused Go behavior | `go test ./pkg/... -run '^TestName$' -count=1 -v` | Use first. | | Root unit/integration suite | `go test ./... -count=1` | Restricted sandbox fails listener-dependent packages. | | Root race suite | `go test -race ./... -count=1` | Not run by `task check`. | -| Vet | `go vet ./...` | Passed on 2026-07-28. | -| Lint | `golangci-lint run ./...` | Failed with 54 findings on 2026-07-28. | -| Build | `go build -o /tmp/... ./cmd/archied` and `cmd/archie-agent` | Both passed on 2026-07-28. | -| Tools tests | `go -C tools test -mod=readonly ./... -count=1` | Passed on 2026-07-28. | -| Generated contract parity | Temp docsgen output + `cmp --silent` | Passed for 10 schemas. | +| Vet | `go vet ./...` | Passed on 2026-09-18. | +| Lint | `golangci-lint run ./...` | Not re-run on 2026-09-18; a focused `--enable-only=dupl ./internal/...` run reported 0 issues. Re-run before quoting a total. | +| Build | `go build -o /tmp/... ./cmd/archied` and `cmd/archie-agent` | Both passed on 2026-09-18. | +| Tools tests | `go -C tools test -mod=readonly ./... -count=1` | Passed on 2026-09-18. | +| Generated contract parity | Temp docsgen output + `cmp --silent` | Passed for 11 schemas on 2026-09-18, after regenerating a file stale since `4b340d2b`. | | Container build | `task docker-build` | Not verified in restricted environment. | -| Repository gate | `task check` | Non-green; skillscript failure in root suite. | +| Repository gate | `task check` | See `archie-diagnostics-and-tooling` for the dated snapshot. | diff --git a/.agents/skills/archie-codebase-discovery/SKILL.md b/.agents/skills/archie-codebase-discovery/SKILL.md index 9ae954584..545638fcd 100644 --- a/.agents/skills/archie-codebase-discovery/SKILL.md +++ b/.agents/skills/archie-codebase-discovery/SKILL.md @@ -6,10 +6,11 @@ description: "Trace Archie behavior and blast radius through Go source, types, s # Discover the Archie codebase Use evidence from the current checkout to explain a behavior before changing it. -Volatile tool facts below were verified on **2026-07-28**. +Volatile tool facts below were verified on **2026-09-18**. -**Composition roots**: `cmd/archied/main.go` (legacy wiring) and -`internal/app/agentworker` (invoked by `cmd/archie-agent/main.go`). +**Composition roots**: `cmd/archied/main.go` (thin dispatcher) → +`internal/app/archied.Run()`, and `internal/app/agentworker` (invoked by +`cmd/archie-agent/main.go`). Do not treat `cmd/*` as the wiring site. **Production wiring**: a path selected by a shipped entrypoint — not a constructor, decoder, or passing unit test. **State owner**: component with authority to validate and persist a state transition. @@ -21,8 +22,9 @@ go version; go env GOMOD GOWORK GOOS GOARCH gopls version; golangci-lint version; staticcheck -version ``` -As of 2026-07-28: Go 1.26.5, gopls 0.23.0, golangci-lint 2.12.2, -staticcheck 2026.1. Use `gopls call_hierarchy` (no `go tool callgraph`). +As of 2026-09-18: Go 1.27.0, gopls 0.23.0, golangci-lint 2.13.2, +staticcheck 2026.2.1. Use `gopls call_hierarchy` (no `go tool callgraph`; +`go tool` lists only asm, cgo, compile, cover, fix, link, preprofile, vet). Writable caches when defaults are read-only: @@ -63,12 +65,12 @@ Start at these verified anchors: | Concern | Current anchor | |---|---| -| Resident daemon | `cmd/archied/main.go` → `run()` → `daemon.Daemon` | +| Resident daemon | `cmd/archied/main.go` → `internal/app/archied.Run()` (`internal/app/archied/main.go`) → `daemon.Daemon` | | Sandboxed worker | `cmd/archie-agent/main.go` → `internal/app/agentworker/`; NATS boundary in `internal/infrastructure/agenttransport/nats/` | -| Task orchestration | `internal/daemon/daemon.go`, `internal/workflow/` | -| Task contracts and SQLite | `internal/store/interface.go`, `internal/store/` | -| Production task persistence | `internal/store/`; `store.Open` in `cmd/archied/main.go` | -| RPC split | `internal/{nats,taskrun,natsrpc,storerpc,forgerpc,worktreerpc}/` | +| Task orchestration | `internal/daemon/daemon.go`, `internal/domain/workflow/` | +| Task contracts | `internal/domain/storecontract/storecontract.go` (`TaskStore`); `internal/store/interface.go` holds compatibility aliases only | +| Production task persistence | `internal/store/`; `openProductionTaskStore` reached via `internal/app/archied/state_store.go` | +| RPC split | `internal/{taskrun,natsrpc,forgerpc,worktreerpc}/` plus `internal/infrastructure/staterpc/` (gRPC State Store) | | Chat and channels | `internal/gateway/`, `internal/channels/` | | Configuration | `internal/config/`, `internal/infrastructure/configuration/`, `config.example.toml`, `deployments/*.toml` | | Target map | `docs/prds/01-project-management.md` and linked `docs/architecture/*.md` | @@ -80,7 +82,7 @@ Start at these verified anchors: ```sh go list -f '{{.ImportPath}}|go={{join .GoFiles ","}}|tests={{join .TestGoFiles ","}}' ./internal/store go list -json ./internal/store -.claude/skills/archie-codebase-discovery/scripts/package-edges.sh ./internal/... +.agents/skills/archie-codebase-discovery/scripts/package-edges.sh ./internal/... ``` ### Resolve symbols with gopls @@ -88,20 +90,20 @@ go list -json ./internal/store 1-indexed `file:line:column`: ```sh -gopls symbols internal/store/interface.go -gopls definition cmd/archied/main.go:154:18 -gopls references -d internal/store/interface.go:25:2 -gopls implementation internal/store/interface.go:16:6 -gopls call_hierarchy internal/store/interface.go:25:2 -go doc ./internal/store.TaskStore +gopls symbols internal/domain/storecontract/storecontract.go +gopls definition cmd/archied/main.go:27:14 +gopls references -d internal/domain/storecontract/storecontract.go:33:2 +gopls implementation internal/domain/storecontract/storecontract.go:33:2 +gopls call_hierarchy internal/domain/storecontract/storecontract.go:33:2 +go doc ./internal/domain/storecontract.TaskStore go test ./internal/store -list '.' ``` ### Generate AST candidates for construction ```sh -.claude/skills/archie-codebase-discovery/scripts/scan-go-type-syntax.sh Daemon . -.claude/skills/archie-codebase-discovery/scripts/scan-go-type-syntax.sh -tests Task . +.agents/skills/archie-codebase-discovery/scripts/scan-go-type-syntax.sh Daemon . +.agents/skills/archie-codebase-discovery/scripts/scan-go-type-syntax.sh -tests Task . ``` Confirms candidates with `gopls definition` or `gopls references`. Pass @@ -154,7 +156,7 @@ Use `gopls references` on constructor and type. ### Prove package dependency direction ```sh -.claude/skills/archie-codebase-discovery/scripts/package-edges.sh ./internal/... +.agents/skills/archie-codebase-discovery/scripts/package-edges.sh ./internal/... go list -deps -json ``` @@ -165,16 +167,20 @@ composition-root read → concrete component → observable behavior`. ```sh rg -n 'toml:"|yaml:"|json:"' internal/config -rg -n 'LoadOverlay|LoadDir|finalize|ForTask|ToConfig' internal/config cmd +rg -n 'Resolve|Overlay|Dir|finalize|ForTask|ToConfig' internal/infrastructure/configuration internal/app/archied gopls references -d ``` -As of 2026-07-28 `cmd/archied` calls `config.LoadOverlay`; `config.LoadDir` is -test-only. +Decoding lives in `internal/infrastructure/configuration`, not `internal/config`. +`configuration.New(log)` returns a `Loader`; the composition root in +`internal/app/archied/bootstrap.go` calls `Loader.Resolve` (with `Loader.Overlay` +and `Loader.Dir` for overlay/`conf.d` inputs). There is no `config.Load` or +`config.LoadDir`. ### Prove production composition -1. Start at `cmd/archied/main.go:run`. +1. Start at `internal/app/archied/main.go` (`Run`), entered from + `cmd/archied/main.go`. 2. Record config condition per branch. 3. Follow concrete construction into `daemon.Daemon` and gateway start. 4. For container/NATS behavior, trace `cmd/archie-agent/main.go` into @@ -185,10 +191,11 @@ test-only. ### Trace SQLite persistence -Treat `internal/store` as the application contract and production adapter. +Treat `internal/domain/storecontract` as the application contract and +`internal/store` as the production SQLite adapter behind it. ```sh -gopls implementation internal/store/interface.go:16:6 +gopls implementation internal/domain/storecontract/storecontract.go:33:2 rg -n 'CREATE TABLE|CREATE INDEX|QueryContext|ExecContext|BeginTx' internal/store ``` diff --git a/.agents/skills/archie-config-and-flags/SKILL.md b/.agents/skills/archie-config-and-flags/SKILL.md index 092f4c012..eea777008 100644 --- a/.agents/skills/archie-config-and-flags/SKILL.md +++ b/.agents/skills/archie-config-and-flags/SKILL.md @@ -9,7 +9,7 @@ Treat configuration as a dataflow, not as a struct inventory. Prove every claimed setting has an input, normalization path, composition seam, and runtime consumer. -Current evidence date: **2026-07-28**. +Current evidence date: **2026-09-18**. Use these status terms exactly: @@ -39,8 +39,8 @@ Never upgrade a status merely because decoding tests pass. 8. Assign one status from the table above. ```bash -rg -n 'LoadOverlay|LoadDir|ForTask|ToConfig|flag\.' \ - internal/config cmd/archied cmd/archie-agent +rg -n 'Resolve|Overlay|Dir|ForTask|ToConfig|flag\.' \ + internal/infrastructure/configuration internal/config internal/app/archied cmd/archie-agent rg -n 'FIELD_NAME|external_key|ENV_NAME' \ --glob '*.go' --glob '*.toml' --glob '*.yml' --glob '*.yaml' . ``` @@ -49,10 +49,10 @@ rg -n 'FIELD_NAME|external_key|ENV_NAME' \ | Input path | Current behavior | Status | | --- | --- | --- | -| `archied -config PATH` | Defaults to `$XDG_CONFIG_HOME/archie/config.toml`, or `$HOME/.config/archie/config.toml`; `cmd/archied/main.go` calls `configuration.Loader.Resolve`. | production-wired | -| `archied -config-overlay PATH` | Resolves a file or directory overlay through the same loader. Omitted overlay fields retain base values. | production-wired | -| `config.Load(PATH)` | Loads one TOML and finalizes it; no current binary calls it directly. | test/library-only | -| `config.LoadDir(base, overlay)` | Supports main YAML/TOML plus feature YAML and `conf.d`; no current entrypoint calls it. | test/library-only | +| `archied -config PATH` | Defaults to `$XDG_CONFIG_HOME/archie/config.toml`, or `$HOME/.config/archie/config.toml`; `internal/app/archied/bootstrap.go` builds `configuration.New(log)` and calls `Loader.Resolve`. | production-wired | +| `archied -config-overlay PATH` | Resolves a file or directory overlay through the same loader (`Loader.Overlay`/`ApplyOverlay`); the dashboard runtime overlay is layered from its own SQLite store. Omitted overlay fields retain base values. | production-wired | +| `configuration.Loader.Dir(base, overlay)` | Supports main YAML/TOML plus feature YAML and `conf.d`; no current entrypoint calls it. | test/library-only | +| `config.Load(path)`, `config.LoadDir(...)` | Removed. The old `internal/config` package-level helpers no longer exist; decoding is `Loader`-based. | not applicable | | `config.example.toml` | Load-tested example, not a deployed configuration. | test/library-only | | `deployments/docker-nats-stack.toml` | Complete host-daemon configuration for the Compose-managed NATS service. It is copied into the operator's config directory; Compose does not mount it. | supported-example | | Operator config and environment | Lives outside the repository; contents are deployment-specific. | external-production-unknown | @@ -98,7 +98,7 @@ Do not expose these helper flags as daemon settings. | --- | --- | --- | | `work_dir`, `db_path` | Derive from `$XDG_DATA_HOME/archie`, else `$HOME/.local/share/archie`; explicit `~` paths are home-expanded. `db_path` is a prefix: tasks/events use `-tasks.sqlite` and conversations use `-conversations.sqlite`. | production-wired | | `skills_dir` | Empty falls back to `work_dir` for startup workflow registry. | partially-wired | -| `plugin_dir` | Empty disables daemon plugin loading; configured directory loaded in `cmd/archied`. | production-wired | +| `plugin_dir` | Empty disables daemon plugin loading; configured directory loaded in `internal/app/archied`. | production-wired | | `poll_interval` | Default `60s`; root poll loop uses it, identities may override only interval. | production-wired | | `label` | No default or validation; used for `dispatch.trigger = "label"` or `"either"`. | production-wired | | `max_retries` | Default `3`; repository value greater than zero overrides. | production-wired | @@ -118,10 +118,10 @@ Do not expose these helper flags as daemon settings. | `nats.url`, `token_env` | Empty URL keeps SQLite path. Configured token variable must be non-empty. | production-wired | | `containers.*` | See dedicated table below. | production-wired | | `chat.*` | See channel table below. | production-wired | -| `memory.provider`, `provider_config` | Feature YAML decodes these; `cmd/archied` always starts built-in file provider at `work_dir/memory`. | decoded-but-unwired | -| `tools.mcp_servers` | `cmd/archied` registers daemon-side MCP providers. Empty transport becomes `stdio`; only `stdio` accepted. `url` decoded but unused. | partially-wired | -| `indexing.index_dir`, `indexing.db_path` | `finalize` derives paths under `work_dir`; no entrypoint constructs `internal/indexing.Manager`. | decoded-but-unwired | -| `extra` / unknown `conf.d` values | Test/library-only `LoadDir` stores untyped data; no core production consumer. | decoded-but-unwired | +| `memory.provider`, `provider_config` | Feature YAML decodes these; `internal/app/archied` always starts built-in file provider at `work_dir/memory`. | decoded-but-unwired | +| `tools.mcp_servers` | `internal/app/archied` registers daemon-side MCP providers. Empty transport becomes `stdio`; only `stdio` accepted. `url` decoded but unused. | partially-wired | +| `indexing.index_dir`, `indexing.db_path` | `finalize` derives paths under `work_dir`; no entrypoint constructs `internal/indexing.Manager` (the exported `NewManager` exists but has no production caller). | decoded-but-unwired | +| `extra` / unknown `conf.d` values | Test/library-only `Loader.Dir` stores untyped data; no core production consumer. | decoded-but-unwired | The default `agent.command = "archie-agent"` is not verified subprocess operation. `SubprocessRunner` expects one JSON invocation on stdin and one JSON @@ -169,12 +169,12 @@ token and allowed-user list. Every other file setting is startup-only. ## Treat multi-identity as partially wired `finalize` requires each identity's `name`, `bot_user`, non-empty repos, forge -type, and `SecretRef` token. `cmd/archied` builds an identity forge client and -worktree manager. +type, and `SecretRef` token. `internal/app/archied` builds an identity forge +client and worktree manager. Do not infer complete isolation: -- `cmd/archied` resolves root forge client before all identity runners. +- `internal/app/archied` resolves root forge client before all identity runners. - Identity `name`, forge, repos, poll interval, bot user, bot email have production reads. Identity bot email not defaulted despite comment. - Identity `models`, `providers`, `dispatch`, `budgets`, `notify`, and @@ -209,8 +209,8 @@ identity: identity forges require explicit structured `token`. | Telegram `token_env = "X"` | Reads `X`. | Not forwarded. | Not forwarded. | | `agent.env = ["X"]` | Names validated. | Copies present `X` from daemon env. | Not used by container construction. | -`cmd/archied` creates `secret.NewRegistry`, which registers only the `env` -engine. Non-env `SecretRef` values are not production-wired. +`internal/app/archied` creates `secret.NewRegistry`, which registers only the +`env` engine. Non-env `SecretRef` values are not production-wired. The host supervisor or launch shell supplies daemon secrets. Compose supplies no `archied` environment because the daemon is not a Compose service. diff --git a/.agents/skills/archie-debugging-playbook/SKILL.md b/.agents/skills/archie-debugging-playbook/SKILL.md index 59a025295..a90c3bda8 100644 --- a/.agents/skills/archie-debugging-playbook/SKILL.md +++ b/.agents/skills/archie-debugging-playbook/SKILL.md @@ -5,11 +5,14 @@ description: Project-specific symptom-to-evidence triage for archie-core. Load t # Debug Archie from evidence -**current** = 2026-07-28 verified; **fixed history** = regression patterns; -**open** = verified gap. +**current** = 2026-09-18 verified (HEAD `2e1e1549`); **fixed history** = +regression patterns; **open** = verified gap. -**Composition root**: `cmd/archied/main.go`. **Core NATS request/reply**: direct -subject + reply inbox. **JetStream**: persisted stream `ARCHIE_TASKS`. +**Composition root**: `cmd/archied/main.go` → `internal/app/archied.Run()`; +wiring lives in `internal/app/archied/bootstrap.go`. **Core NATS +request/reply**: direct subject + reply inbox. **JetStream**: persisted stream +`ARCHIE_TASKS`. **State Store**: gRPC (`internal/infrastructure/staterpc`), not +NATS. **Environment failure**: test/process cannot use host resource (listener, tmp, Docker). **Code regression**: behavior fails where prerequisites available. @@ -29,7 +32,7 @@ Use this symptom index: | Symptom | First boundary | Go to | |---|---|---| -| `Run() = "\n"` in `TestRunWrapsExternalCommand` | Skill-script test command shape | Current known regression | +| `Run() = "\n"` in `TestRunWrapsExternalCommand` | Skill-script test command shape | Fixed history — passes since 2026-09-16 | | Embedded NATS panics before a test assertion | Listener permission | Test environment | | `nats: no responders available for request` | Core NATS subscription | NATS request/reply | | Agent reply timeout | JetStream request, inbox, or worker | NATS request/reply | @@ -68,9 +71,13 @@ repository-local mitigation. `worktree.Manager.setIdentity` pins GIT_CONFIG_GLOBAL=/dev/null go test ./internal/worktree -count=1 ``` -## Confirm the current known regression +## Confirm the skill-script regression stays fixed -As of 2026-07-28: +The `TestRunWrapsExternalCommand` failure (`Run() = "\n"`) was a test bug, not +a product bug: `exec.CommandContext(ctx, "sh", "-c", "echo", "wrapped")` +passes `wrapped` as shell `$0`, so only `echo` ran. It failed from commit +`308c199` and **passes as of 2026-09-16**. Re-run it to confirm the fix has not +regressed; a failure now is a new defect, not this one. ```bash env GOTMPDIR=/tmp GOCACHE=/tmp/archie-skillscript-gocache \ @@ -78,12 +85,6 @@ env GOTMPDIR=/tmp GOCACHE=/tmp/archie-skillscript-gocache \ -run '^TestRunWrapsExternalCommand$' -count=1 -v ``` -Expected current failure: `Run() = "\n"`. The test constructs -`exec.CommandContext(ctx, "sh", "-c", "echo", "wrapped")`. For `sh -c`, the -argument after the command string becomes shell `$0`; therefore the command -string is only `echo`, which prints a newline. This is a test regression from -commit `308c199`, not proof that Yaegi cannot run external commands. - ## Triage NATS request/reply and JetStream | Surface | Subject or resource | Bound/current behavior | @@ -91,7 +92,7 @@ commit `308c199`, not proof that Yaegi cannot run external commands. | Task discovery | `archie.task.>` in `ARCHIE_TASKS` | Work-queue retention; daemon durable `archie-daemon`; max deliver 3 | | Per-stage agent request | `archie.agent..request` | Reply inbox in `X-Archie-Reply`; wall-clock budget or 30m | | Full task handoff | `archie.taskrun.` | Core request/reply; no-responder retry 20s every 250ms | -| Store RPC | `archie.store.update`, `.transition` | Client timeout 60s composed by `internal/app/agentworker/worker.go` through the infrastructure transport | +| Store RPC | gRPC State Store (`internal/infrastructure/staterpc`) | Client timeout 60s (`rpcTimeout` in `internal/app/agentworker/worker.go`); task-scoped grants for Update/Transition/InsertEvent only | | Forge RPC | `archie.forge.*` | Error travels in a JSON envelope | | Worktree RPC | `archie.worktree.prepare`, `.push` | Server default handler bound 15m | | Discovery dedup | `Nats-Msg-Id: archie://` | JetStream duplicate window 2m | @@ -111,31 +112,33 @@ explicit `containers.network` plus warning logs now fence that path. Check semantics: republishing same owner/repo/issue within 2m is dedup'd; reply inbox auto-unsubscribes after one response; `Client.Fetch` must inspect -`batch.Error()`; `agentexec.runStages` returns envelope and nil error for stage -failure. +`batch.Error()`; an empty poll reports `eventbus.ErrNoMessage` rather than +`(nil, nil)`. ```bash env GOTMPDIR=/tmp GOCACHE=/tmp/archie-nats-unit-gocache \ - go test ./internal/nats -run '^TestFetchPropagatesBatchError' -count=1 -v + go test ./internal/infrastructure/eventbus/nats -run 'TestPublishUniqueSuppressesDuplicateKeyOnWorkQueueStream' -count=1 -v env GOTMPDIR=/tmp GOCACHE=/tmp/archie-agent-unit-gocache \ - go test ./internal/agentexec -run '^TestHandleMessage' -count=1 -v + go test ./internal/agentexec -run '^TestToolSetHandler' -count=1 -v ``` ## Trace configuration from source to behavior -Trace: `cmd/archied` reads `-config`/`-config-overlay` → `config.LoadOverlay` +Trace: `cmd/archied` (`internal/app/archied/bootstrap.go`) reads +`-config`/`-config-overlay` → `configuration.New(log)` + `Loader.Resolve` decodes, `finalize` defaults/validates → secrets resolve → composition root constructs service → Compose/subprocess/`containerEnv` propagate → consumer. ```bash -rg -n 'config\.LoadOverlay|config\.LoadDir' cmd/archied internal/config +rg -n 'Loader\.(Resolve|Overlay|Dir)|configuration\.' internal/app/archied internal/infrastructure/configuration env GOTMPDIR=/tmp GOCACHE=/tmp/archie-config-gocache \ - go test ./internal/config \ - -run 'Test(LoadOverlay|LoadForgeToken|DockerConfig)' -count=1 -v + go test ./internal/infrastructure/configuration \ + -run 'Test(ResolveSelectsFileFormatsAndDirectories|LoadForgeTokenEnvBackwardCompat)' -count=1 -v ``` -`config.LoadDir` is test-only; a YAML field decoding correctly without -composition-root read is a wiring gap, not a parser bug. +The `config.Load`/`config.LoadDir` helpers are gone; decoding is +`Loader`-based. A YAML field decoding correctly without a composition-root read +is a wiring gap, not a parser bug. For environment failures: - The host supervisor supplies the daemon environment; inspect its unit or @@ -188,7 +191,7 @@ bot token. Empty `allowed_user_ids` denies everyone. ```bash env GOTMPDIR=/tmp GOCACHE=/tmp/archie-telegram-gocache \ go test ./internal/channels/telegram \ - -run 'Test(SenderAllowlistFailsClosed|IsSenderAllowed|PublishedCommandsMatchExecutableCommandSurface)$' \ + -run 'Test(SenderAllowlistFailsClosed|IsSenderAllowed|SessionManagementCommandsArePublished)$' \ -count=1 -v ``` @@ -197,9 +200,12 @@ env GOTMPDIR=/tmp GOCACHE=/tmp/archie-telegram-gocache \ ## Diagnose state, events, containers, and optional features ### SQLite and transitions -`internal/store.Store.Transition` owns task state changes and transition history. +`internal/store.Store.Transition` owns task state changes and transition +history: it guards on `from` (`WHERE id=? AND status=?`, returning +`store.ErrStaleTransition`) and writes status plus history in one transaction. When state is surprising: read `TaskByID` before/after caller; trace every -`Transition`, `Requeue`, `RecoverStale`, `Update`. +`Transition`, `Requeue`, `RecoverStale`, `Update` — and remember the daemon +exposes these over the gRPC State Store, not NATS. ```bash env GOTMPDIR=/tmp GOCACHE=/tmp/archie-state-gocache go test ./internal/store -count=1 @@ -222,4 +228,4 @@ root-identity-bound; `SubprocessRunner` expects stdin JSON protocol. | Invalid MCP config | Warn and continue | | Skill catalog | Warn, continue without tool | | Worktree augmentation | Log error, use startup registry | -| Workspace indexing | **Open:** no production `indexing.NewManager` call | +| Workspace indexing | **Open:** `indexing.NewManager` exists but has no production caller in `internal/app/archied` | diff --git a/.agents/skills/archie-diagnostics-and-tooling/SKILL.md b/.agents/skills/archie-diagnostics-and-tooling/SKILL.md index 8c65f7163..f6df6d084 100644 --- a/.agents/skills/archie-diagnostics-and-tooling/SKILL.md +++ b/.agents/skills/archie-diagnostics-and-tooling/SKILL.md @@ -8,8 +8,8 @@ description: "Measure Archie codebase maintainability, Go package dependency sha Collect evidence before judging quality. Keep collection read-only, save raw output, state what each metric cannot prove. -Volatile facts and numeric snapshots below were verified on **2026-09-16** -(HEAD `d565fb8`). +Volatile facts and numeric snapshots below were verified on **2026-09-18** +(HEAD `2e1e1549`). Route to `archie-codebase-discovery` for semantic traces, `archie-config-and-flags` for config enumeration, @@ -21,7 +21,7 @@ Route to `archie-codebase-discovery` for semantic traces, environment. - **Candidate:** a machine-found item requiring semantic confirmation. - **Production composition:** concrete construction selected by a shipped - entrypoint. Principal: `cmd/archied/main.go`. + entrypoint. Principal: `cmd/archied/main.go` → `internal/app/archied.Run()`. - **Fan-in:** number of packages in the same module that directly import a package. - **Fan-out:** number of packages in the same module that a package directly @@ -42,7 +42,7 @@ export GOLANGCI_LINT_CACHE=/tmp/archie-diagnostics-lint export GOPROXY=off export GOFLAGS=-mod=readonly -skill=.claude/skills/archie-diagnostics-and-tooling +skill=.agents/skills/archie-diagnostics-and-tooling stamp=$(date -u +%Y%m%dT%H%M%SZ) evidence_dir=/tmp/archie-diagnostics-"$stamp" mkdir -p "$evidence_dir" @@ -173,7 +173,7 @@ golangci-lint run --enable-only=dupl ./internal/... staticcheck -checks=U1000 ./internal/... ``` -On 2026-09-16, the focused `dupl` command reported zero issues. Two code paths +On 2026-09-18, the focused `dupl` command reported zero issues. Two code paths can implement the same responsibility with different syntax. For every candidate: name the externally visible operation; search all @@ -183,7 +183,7 @@ OS files, and wire names; identify what a new feature supersedes; record production-only, test-only, compatibility, generated, and unreachable paths separately. -As of 2026-09-16, `Taskfile.yml` defines no `deadcode` task and `go tool` lists +As of 2026-09-18, `Taskfile.yml` defines no `deadcode` task and `go tool` lists no dead-code analyzer. ## Collect test, race, coverage, and runtime evidence @@ -194,7 +194,7 @@ go tool cover -func=/tmp/archie-store.cover go test -race ./internal/store -count=1 ``` -As of 2026-09-16, the package reported 78.6% statement coverage and passed its +As of 2026-09-18, the package reported 78.6% statement coverage and passed its race run. ```sh @@ -231,26 +231,26 @@ duplication, hid a path, or weakened a gate. ## Dated current snapshot -The bundled scripts measured this checkout on 2026-09-16 (HEAD `d565fb8`). +The bundled scripts measured this checkout on 2026-09-18 (HEAD `2e1e1549`). These are post-fix numbers: earlier runs of the same scripts reported roughly 3x these file and line counts because they walked `.worktrees/`. | Observation | Unstable value | |---|---:| -| Root production/test/generated Go files | 408 / 397 / 9 | -| Root production/test/generated physical lines | 73,729 / 93,572 / 12,237 | +| Root production/test/generated Go files | 408 / 402 / 9 | +| Root production/test/generated physical lines | 74,752 / 95,264 / 12,774 | | `internal/app/archied.Run` body span / approximate complexity | 97 / 19 | -| Root internal packages / direct internal edges | 110 / 306 | -| Tagged config field rows | 196 | -| Tracked paths / tracked `node_modules` paths / tracked symlinks | 1,081 / 0 / 1 | +| Root internal packages / direct internal edges | 110 / 316 | +| Tagged config field rows | 194 | +| Tracked paths / tracked `node_modules` paths / tracked symlinks | 1,087 / 0 / 1 | | `.dockerignore` present | 1 | -`source-metrics.go` and `git ls-files` reconcile exactly here: 408 + 9 -generated production files and 397 test files, plus the `tools` module's -5 / 2 / 0, against 422 non-test and 398 test tracked files. The single extra -file is an untracked probe under `.local/`. +`source-metrics.go` and `git ls-files` reconcile here: 408 + 9 generated +production files and 402 test files, plus the `tools` module's 5 / 2 / 0, +against 422 non-test and 403 test tracked files. The single extra file is an +untracked probe under `.local/issue-tracker/probes/`. -The scripts count themselves: the skill's own Go files contribute 816 of those +The scripts count themselves: the skill's own Go files contribute 640 of those production lines. Editing a bundled script therefore moves `production_physical_lines` on its own, so re-baseline after any script change rather than reading the delta as a change in the product code. diff --git a/.agents/skills/archie-diagnostics-and-tooling/scripts/delivery-snapshot.sh b/.agents/skills/archie-diagnostics-and-tooling/scripts/delivery-snapshot.sh index 82a9ce2f7..66a8a86f3 100755 --- a/.agents/skills/archie-diagnostics-and-tooling/scripts/delivery-snapshot.sh +++ b/.agents/skills/archie-diagnostics-and-tooling/scripts/delivery-snapshot.sh @@ -54,7 +54,7 @@ scan_literal task_race taskfile 'go test[^#\n]*-race|-race[^#\n]*go test' Taskfi scan_literal task_tools_module taskfile 'go -C tools|cd tools|task:[[:space:]]+tools' Taskfile.yml scan_literal task_docs taskfile 'docsgen|docs:generate|docs:check' Taskfile.yml -# One CI surface: .github/workflows. The Gitea workflow was dropped in e539048 +# One CI surface: .github/workflows. The Gitea workflow was dropped in dd9bddc2 # ("ci: build and push images to GHCR, drop the Gitea workflow"), so scanning # .gitea/workflows aborted the whole snapshot under `set -eu`. scan_literal github_go_gate github-workflows 'go test|go vet|go build|golangci-lint|task[[:space:]]+check' .github/workflows diff --git a/.agents/skills/archie-diagnostics-and-tooling/scripts/package-shape.sh b/.agents/skills/archie-diagnostics-and-tooling/scripts/package-shape.sh index beb7723bb..76d1ac12d 100755 --- a/.agents/skills/archie-diagnostics-and-tooling/scripts/package-shape.sh +++ b/.agents/skills/archie-diagnostics-and-tooling/scripts/package-shape.sh @@ -19,7 +19,7 @@ script_dir=$(CDPATH= cd "$(dirname "$0")" && pwd) repo_root=$(CDPATH= cd "$script_dir/../../../.." && pwd) module_relative=${1:-.} module_root="$repo_root/$module_relative" -edge_helper="$repo_root/.claude/skills/archie-codebase-discovery/scripts/package-edges.sh" +edge_helper="$repo_root/.agents/skills/archie-codebase-discovery/scripts/package-edges.sh" if [ ! -f "$module_root/go.mod" ]; then printf 'package-shape: no go.mod at %s\n' "$module_root" >&2 diff --git a/.agents/skills/archie-docs-and-writing/SKILL.md b/.agents/skills/archie-docs-and-writing/SKILL.md index 8fdefeb8c..c0cf9354f 100644 --- a/.agents/skills/archie-docs-and-writing/SKILL.md +++ b/.agents/skills/archie-docs-and-writing/SKILL.md @@ -9,7 +9,9 @@ Write one durable record for each decision or behavior. Separate what runs now from what Archie has approved as its destination. Route to `archie-architecture-planning-campaign` for ownership/boundary -decisions, `archie-architecture-contract` for invariants, +decisions (parked outside this checkout, under +`/work/apps/archie-skills-parked/`), +`archie-architecture-contract` for invariants, `archie-codebase-discovery` for live entry points, `archie-failure-archaeology` for incidents, `archie-config-and-flags`, `archie-build-and-env`, or `archie-run-and-operate` @@ -34,12 +36,12 @@ migration delta. ## Use the repository authority map -Verified on 2026-07-28: +Verified on 2026-09-18: | Location | Role | Discipline | |---|---|---| -| Live Go code, tests, `Taskfile.yml`, executable config parsing, composition under `cmd/` | `CURRENT` execution evidence | Trace producers and consumers; tests prove only asserted behavior | -| `CLAUDE.md` | Current contributor and safety protocol plus compact architecture orientation | Verify operational claims against code and `Taskfile.yml`; `AGENTS.md` is a symlink | +| Live Go code, tests, `Taskfile.yml`, executable config parsing, composition under `internal/app/` (entered from `cmd/`) | `CURRENT` execution evidence | Trace producers and consumers; tests prove only asserted behavior | +| `CLAUDE.md` | Current contributor and safety protocol plus compact architecture orientation | Verify operational claims against code and `Taskfile.yml`; `AGENTS.md` is a symlink to it | | `ARCHITECTURE.md` | Useful architecture history and partial current overview | Corroborate every inventory/status claim; its "Planned" list is stale for implemented skill support | | `docs/prds/01-project-management.md` | Index and approved foundation for target architecture | Add or change target decisions in the focused document it names | | `docs/architecture/*.md` | Focused target decisions, active review procedure, migration inventory | Read each file's status | @@ -50,7 +52,7 @@ Verified on 2026-07-28: | `docs/data/generated/contracts.json` | Current working-tree `GENERATED` output | Regenerate; never edit by hand | | `docs/` | Repository documentation only | Nothing builds, renders, or publishes it; Markdown is the artifact | -The repository root has no `README.md` or `CONTRIBUTING.md` as of 2026-07-28. +The repository root has no `README.md` or `CONTRIBUTING.md` as of 2026-09-18. ## Choose one destination @@ -117,26 +119,24 @@ Distinguish evidence strength: ## Maintain component changelogs Keep gateway behavior in `CHANGELOG.archied.md` and agent-runtime behavior in -`CHANGELOG.archie.md`; keep `CHANGELOG.md` as their index. The archied image -copies both component files to `/usr/share/archie/`, and +`CHANGELOG.archie.md`; keep `CHANGELOG.md` as their index. +`Dockerfile.archied` copies both component files to `/usr/share/archie/`, and `internal/releaseannounce.changelogSection` parses a version heading. -A verified 2026-07-28 contradiction is `OPEN`: - -- `.gitea/workflows/deploy.yml` passes `vX.Y.Z` component versions. -- `changelogSection` searches for `## []`, and its fixtures use - `## [v0.1.0]`. -- The two current component changelogs use headings such as `## [1.1.0]` - without `v`. -- `cmd/archied.TestChangelogsTrackGatewayAndRuntimeIndependently` expects - the unprefixed current heading. - -Resolve parser, release metadata tests, and both documents atomically; do not -"fix" only the prose. +Version-heading format is settled as of 2026-09-18: both component changelogs +use `## [1.27.0]` with no `v` prefix, `changelogSection` searches for +`## []`, and `.github/workflows/deploy.yml` gates each release tag with +`grep -q "^## \[$ver\]" CHANGELOG..md`. The earlier +`vX.Y.Z`-vs-`X.Y.Z` contradiction involved `.gitea/workflows/deploy.yml`, which +was dropped (`dd9bddc2 ci: build and push images to GHCR, drop the Gitea +workflow`), so that question is `HISTORICAL`, not `OPEN`. The remaining guard is +the CI shell gate plus `internal/releaseannounce`'s announcer tests +(`TestAnnouncerReportsMissingReleaseSectionWithoutAdvancingState`); the old +`cmd/archied.TestChangelogsTrackGatewayAndRuntimeIndependently` no longer exists. ## Maintain generated contracts -Current behavior, verified 2026-07-28: +Current behavior, verified 2026-09-18: - Tool path is `tools/docsgen`, not `tools/cmd/docsgen`. - Separate Go module replacing root module with `../`. @@ -170,7 +170,7 @@ step; the publishing surface is an open decision | Content | Markdown under `docs/`, including `docs/data/generated/contracts.json` | | Links | Repository-relative (`../architecture/organisation.md`); no root-absolute links | | Build | None. There is no site, no dev server, no link checker, no Pages deploy | -| Nav | `docs/prds/01-project-management.md` | +| Nav | `docs/prds/01-project-management.md` is the index; there is no site nav | Because no build validates links, check them yourself or with a one-off script — nothing catches a dead link for you: diff --git a/.agents/skills/archie-domain-reference/SKILL.md b/.agents/skills/archie-domain-reference/SKILL.md index 829c9c5f7..39700c12b 100644 --- a/.agents/skills/archie-domain-reference/SKILL.md +++ b/.agents/skills/archie-domain-reference/SKILL.md @@ -10,8 +10,12 @@ domain meaning with its owner, and mark migration state explicitly. ## Classify every statement -As of 2026-07-28, Archie is between technical-package implementation and -approved domain-oriented target. Never blend the two. +As of 2026-09-18, Archie is partway through the domain migration: some approved +target packages now exist and are production-wired (`internal/domain/messaging`, +`internal/domain/workflow`, `internal/domain/workintake`, `internal/app`, +`internal/eventbus`, `internal/infrastructure`), while others remain target-only +(`internal/domain/agent`, `internal/domain/identity`, `internal/policy`). Never +blend the two. | Label | Meaning | Evidence rule | | --- | --- | --- | @@ -23,7 +27,7 @@ approved domain-oriented target. Never blend the two. Start design notes with status sentences: ```text -CURRENT: store.Task persists one mutable Stage string. +CURRENT: task.Task persists one mutable Stage string. APPROVED TARGET: WorkflowExecution records StepExecution history. OPEN: the exact schema and Go API for that history. ``` @@ -61,10 +65,10 @@ rg -n 'type IdentityConfig|type IdentityRunner|Identity string|BotUser string' i | **Adapter** | Boundary code translating external payloads into owned contracts. | **TARGET RULE:** infrastructure implements domain-owned interfaces. | | **Gateway** | Current technical interface/router and package — not approved domain vocabulary. | **CURRENT ONLY:** `internal/gateway`, `internal/channels`. | | **ChannelBindingID** | Stable reference connecting Agent to channel binding. | **APPROVED TARGET INPUT:** part of Conversation identity. | -| **Message** | Immutable application record with MessageID and typed content. Platform IDs are external correlations. | **TARGET:** Messaging. **CURRENT:** `gateway.Message`; richer `MessageEvent` exists beside it. | -| **Conversation** | AgentID + UserIdentityID + ChannelBindingID + ExternalConversationID + ThreadID. | **CONFIRMED TARGET:** Messaging. **CURRENT:** `gateway.SessionSource` lacks canonical Agent/user IDs. | +| **Message** | Immutable application record with MessageID and typed content. Platform IDs are external correlations. | **CURRENT:** `internal/domain/messaging.Message`/`MessageID`; `gateway.MessageEvent` is a richer event type beside it. | +| **Conversation** | AgentID + UserIdentityID + ChannelBindingID + ExternalConversationID + ThreadID. | **CONFIRMED TARGET, partly current:** `internal/domain/messaging.Conversation`/`ConversationID` exist; the gateway session store still lacks canonical Agent/user IDs. | | **Conversation branch** | Child conversation from one immutable parent message. | **CONFIRMED TARGET.** No current implementation. | -| **Work Intake** | Admission, validation, routing boundary between interaction and durable workflow-backed work. | **CONFIRMED TARGET:** `internal/domain/workintake`. Does not exist as current package. | +| **Work Intake** | Admission, validation, routing boundary between interaction and durable workflow-backed work. | **CONFIRMED TARGET, current package:** `internal/domain/workintake` exists. | | **Accepted work request** | Channel-neutral handoff after Work Intake accepts proposed outcome. | **TARGET HANDOFF:** Agent System consumes it, selects Workflow version, creates WorkflowExecution. | Do not equate an inbound Message with work. An Agent may answer, use @@ -80,10 +84,10 @@ external payload -> channel adapter -> canonical Message and Conversation ``` Do not describe current `/spawn` as this target. It directly creates a -`store.Task` with synthetic forge issue number. +`task.Task` with synthetic forge issue number. ```bash -rg -n 'type Message struct|type MessageEvent struct|ToLegacy|type SessionSource|CreateTask|EnqueueChatTask|synthetic' internal/gateway internal/store cmd/archied +rg -n 'type Message struct|type MessageEvent struct|ToLegacy|type Conversation struct|CreateTask|EnqueueChatTask|synthetic' internal/domain/messaging internal/gateway internal/store internal/app/archied ``` ## Use the canonical execution vocabulary @@ -91,10 +95,10 @@ rg -n 'type Message struct|type MessageEvent struct|ToLegacy|type SessionSource| | Term | Archie meaning | Current representation | | --- | --- | --- | | **Workflow** | Reusable, versioned definition. | `workflow.Workflow` is named `[]Stage` without durable version model. | -| **WorkflowExecution** | One durable execution of one Workflow version. | `store.Task` is migration baseline. | +| **WorkflowExecution** | One durable execution of one Workflow version. | `task.Task` is migration baseline. | | **WorkflowStep** | One defined operation. | `workflow.Stage` is closest definition. | -| **StepExecution** | Durable record of one step's execution and outcome. | Missing; `store.Task.Stage` is one mutable string. | -| **Attempt** | Retry within same WorkflowExecution. | `Task.Attempt`/`RetryCount` must become explicit attempt history. | +| **StepExecution** | Durable record of one step's execution and outcome. | Missing; `task.Task.Stage` is one mutable string. | +| **Attempt** | Retry within same WorkflowExecution. | `task.Task.Attempt`/`RetryCount` must become explicit attempt history. | | **Agent step execution** | One bounded model-backed WorkflowStep invocation. | `agentexec.Request`/`Result` correlate TaskID, Attempt, Stage, protocol version. | Use `WorkflowExecution` in domain contracts, architecture, generated reference. @@ -103,7 +107,7 @@ informal prose. ### Interpret the current lifecycle exactly -**CURRENT** `store.Task` states: +**CURRENT** `task.Task` states: ```text queued -> running @@ -114,12 +118,13 @@ pr_open -> merged | rejected running after crash -> queued ``` -Do not claim enforced. `Store.Transition` writes new status without comparing -`from`, inserts history separately. `internal/gateway/tasks.go` duplicates -status strings locally. +Do not claim enforced. `Store.Transition` now guards on `from` +(`WHERE id=? AND status=?` → `store.ErrStaleTransition`) and writes status plus +history in one transaction; the sentinel crosses the State Store gRPC boundary. +See `internal/store/store.go`. ```bash -rg -n 'Status[A-Za-z]+|func \(s \*Store\) Transition|func \(s \*Store\) Requeue|RecoverStale' internal/store internal/gateway +rg -n 'Status[A-Za-z]+|func \(s \*Store\) Transition|func \(s \*Store\) Requeue|RecoverStale' internal/store internal/domain/workflow internal/gateway ``` ## Keep memory scope separate from conversation scope @@ -154,7 +159,7 @@ rg -n 'type MemoryProvider|Initialize\(|type Config struct|MEMORY.md|USER.md|Age Give plugin only narrow typed registrars and explicitly supplied contracts. ```bash -go test ./internal/plugin -run 'Test(GenericPluginContractRemainsMetadataOnly|EveryEngineInterfaceHasTypedCapabilityAndFamilyOwner)' -count=1 +go test ./internal/plugin -run 'Test(EngineFamilyInspectionRejectsInvalidShapes|PluginInterfaceExists)' -count=1 ``` ## Distinguish ownership and boundary terms @@ -164,14 +169,16 @@ go test ./internal/plugin -run 'Test(GenericPluginContractRemainsMetadataOnly|Ev | **Domain** | Own one cohesive responsibility: vocabulary, state, rules, operations, commands, events, policies, runtime settings, required service contracts. | | **Command** | Request an action. Receiving domain owns contract and decides result. | | **Domain event** | Record something that happened. Publishing domain owns schema. | -| **Event bus** | Broker-neutral delivery/lifecycle contract. `internal/eventbus` is approved target, not current. | +| **Event bus** | Broker-neutral delivery/lifecycle contract. `internal/eventbus` exists and is production-wired. | | **Observability event** | `internal/events.Event` projection input. Bounded in-process bus may drop. Not authoritative state. | | **Policy** | Typed rule evaluated through shared mechanics. Consuming domain owns vocabulary. | | **Infrastructure** | Implements domain-owned ports, translates DBs, brokers, SDKs, files, containers. | -| **Application composition** | Chooses implementations, translates config, connects contracts, orders startup/shutdown. Target: `internal/app`; current wiring in `cmd/archied`. | +| **Application composition** | Chooses implementations, translates config, connects contracts, orders startup/shutdown. Current and target: `internal/app`; `cmd/*` stays process-input-only. | -As of 2026-07-28, `internal/domain`, `internal/eventbus`, `internal/policy`, -`internal/infrastructure`, `internal/app` do not exist. Target paths only. +As of 2026-09-18, `internal/domain`, `internal/eventbus`, +`internal/infrastructure`, and `internal/app` exist; only `internal/policy` +does not. `internal/domain/agent` and `internal/domain/identity` are still +target-only. ```bash for p in internal/domain internal/eventbus internal/policy internal/infrastructure internal/app; do test -d "$p" && echo "$p CURRENT" || echo "$p TARGET-ONLY"; done @@ -196,15 +203,15 @@ Model never runs git directly. Worktree operations are daemon-owned. | `config.IdentityConfig.Name` | Canonical IdentityID plus separate bindings | Do not preserve mutable display name as ID. | | `config.IdentityConfig.BotUser` | Forge/Git or channel external binding | Not Conversation or Agent identity. | | `daemon.IdentityRunner` | Composition of identity-correlated capabilities | Currently service bundle, not Identity aggregate. | -| `gateway.Message`, `MessageEvent` | Canonical Messaging Message | Choose one immutable typed contract. | -| `gateway.SessionContext` | Conversation metadata | Add canonical Agent/user/binding ownership. | +| `gateway.MessageEvent` | Canonical Messaging Message | Choose one immutable typed contract; `internal/domain/messaging.Message` is the target. | +| gateway session context | Conversation metadata | Add canonical Agent/user/binding ownership. | | forge `Issue` or future Jira item | Channel message or external artifact | Preserve native reference in adapter. | | gateway `/spawn` and `TaskCreator` | Proposed/accepted work handoff | Work Intake admits; Agent System creates execution. | -| `store.Task` | WorkflowExecution | Preserve every field intentionally. | +| `store.Task` | WorkflowExecution | Preserve every field intentionally; the type now lives at `internal/domain/workflow/task.Task`. | | `workflow.Workflow` | Versioned Workflow definition | Add identity/version ownership before parity. | | `workflow.Stage` | WorkflowStep definition | A definition is not execution history. | | mutable `Task.Stage` | Derived position plus StepExecution records | Deletion requires proven history parity. | -| `Task.Attempt`, `RetryCount` | Attempt history and retry evidence | Keep worker delivery retries distinct. | +| `task.Task.Attempt`, `RetryCount` | Attempt history and retry evidence | Keep worker delivery retries distinct. | | `workflow.TaskContext` | Execution state plus narrow step ports | Remove global config/service bag. | | `events.Event` and `Bus` | Projections fed by authoritative domain events | Current drops acceptable for observability, not state. | | `cmd/archied` wiring | `internal/app/archied` composition | Keep `cmd/*` process-input-only. | diff --git a/.agents/skills/archie-failure-archaeology/SKILL.md b/.agents/skills/archie-failure-archaeology/SKILL.md index b8d65ae21..cca1c65ed 100644 --- a/.agents/skills/archie-failure-archaeology/SKILL.md +++ b/.agents/skills/archie-failure-archaeology/SKILL.md @@ -9,7 +9,7 @@ Stop the project from paying twice for the same lesson. Reconstruct what happened from checked-out code, tests, configuration, and commit graph. Treat commit prose as a lead, not proof. Preserve uncertainty. -All volatile observations are dated **2026-07-28**. +All volatile observations are dated **2026-09-18** (HEAD `2e1e1549`). Route to `archie-debugging-playbook` for active symptoms, `archie-codebase-discovery` for deep caller tracing. @@ -109,7 +109,7 @@ If current behavior differs from chronicle, update the packet. | Incident and date | Symptom | Root cause | Evidence and status | Settled rule | |---|---|---|---|---| -| Label auto-claim, 2026-07-22 | >80 labelled issues enrolled under label polling. | Label polling means eligibility, not hint. | `519882f` changes `config.docker.toml` to `assignee`. `internal/config/config.go` defaults to `assignee` but permits `label` and `either`. **Status:** `settled`. | Default to assignee-driven pickup. Label/either are deliberate bulk enrollment. | +| Label auto-claim, 2026-07-22 | >80 labelled issues enrolled under label polling. | Label polling means eligibility, not hint. | `519882f` changed the Compose profile to `assignee`; that file was deleted (`cbb7929e`) and survives as `deployments/docker-nats-stack.toml`, which sets `trigger = "assignee"`. `internal/config/config.go` defaults to `assignee` but permits `label` and `either`. **Status:** `settled`. | Default to assignee-driven pickup. Label/either are deliberate bulk enrollment. | | Baseline-red park storm, 2026-07-23 onward | Task parks on gate failure that existed before its feature work. | Parking/retrying cannot change pre-existing failure; auto-repair mixes repository repair into feature scope. | `b2482af` made `StageBaselineGate` launch builder instead of parking. Nine repair commits in history. **Status:** `current` mitigation and architecture weak point. | Measure untouched baseline first. Track baseline repair as explicit change. | | Passed build with zero changes, 2026-07-23 | Commit-push reports "no changes," parks, retries, repeats. | "Passed with no changes" lacked terminal domain outcome. | `62c0df1` introduced no-op detection; `2bf42bd` set `Outcome`. `TestStageCommitPushClosesIssueWhenBuildNoChanges`. **Status:** `settled`. | Model no-op as explicit terminal result. | | Infinite parked-task retry, 2026-07-23 | Removing parked label repeatedly requeues with no terminal bound. | Retry count not durable or capped. | `a8fdb50` added persisted `retry_count`, global/per-repo `max_retries`, and `dead`. **Status:** `settled`. | Every autonomous retry needs durable attempt state, explicit bound, terminal state, and operator-visible reason. | @@ -119,7 +119,7 @@ If current behavior differs from chronicle, update the packet. | Incident and date | Symptom | Root cause | Evidence and status | Settled rule | |---|---|---|---|---| | Container network auto-detection, 2026-07-27 | Every container task returns `nats: no responders`. | `selfNetwork` inspected hostnamed container, silently fell back to default bridge. | `37cf089` records incident, adds `containers.network`; `2ac306b` pins `archie-core_default`. **Status:** `conditional`. | Make cross-container network membership explicit in deployment. | -| SecretRef schema crash-loop, 2026-07-27 and current `HEAD` | Both identities demand `ARCHIE_GITHUB_TOKEN` despite Gitea config. | Removing old `token_env` field let TOML ignore deployed value. `1f8588d` restored precedence-compatible fallback; `308c199` removed `Forge.TokenEnv` declaration while `finalize` still references it. **Status:** `open` regression at `HEAD`. | Preserve decoded compatibility until deployed files migrated and rejection of stale keys observable. | +| SecretRef schema crash-loop, 2026-07-27 | Both identities demand `ARCHIE_GITHUB_TOKEN` despite Gitea config. | Removing old `token_env` field let TOML ignore deployed value. `1f8588d` restored precedence-compatible fallback; a later `308c199` removed the `Forge.TokenEnv` declaration while `finalize` still referenced it. | **Status: `settled`** as of 2026-09-18 — `config.Forge.TokenEnv` is declared (`internal/config/config.go`) and `configuration/defaults.go` converts it to an env `SecretRef` when the structured token is empty. | Preserve decoded compatibility until deployed files migrated and rejection of stale keys observable. | | Compose secret omission, 2026-07-27 | Config names valid env var, container sees it empty. | Compose forwards only listed environment entries. | `22fc0ef`, `d0624b7`. **Status:** `conditional`. | Trace every secret from config key to host value to Compose passthrough to process lookup. | | FullStream/TextStream deadlock, 2026-07-27 | Streamed Telegram reply renders empty cursor, typing never clears. | ai-sdk makes `FullStream` authoritative and backpressured; `TextStream` is best-effort. | `6a0bd27`. **Status:** `settled` with regression-test gap. | Drain authoritative stream through close. | | MCP Content-Length framing and mirrored fake, 2026-07-26 to 2026-07-27 | Internal MCP tests pass while official-SDK servers do not. | MCP stdio uses newline-delimited JSON. Test subprocess called same helpers, mirroring the same wrong protocol. | Compare `f2768eb:internal/tools/mcp/framing.go` with `dda4cde`. **Status:** `settled`. | For wire contracts, test raw canonical frames and independently implemented peer. | @@ -164,15 +164,18 @@ If current behavior differs from chronicle, update the packet. | Incident and date | Symptom | Root cause | Status | Settled rule | |---|---|---|---|---| | Tracked `.gotmp`, 2026-07-27 | Build-cache binaries, generated `_testmain.go`, `.git-askpass` appear in commits. | Workaround path not ignored before use. | `4cb0577` removes artifacts and adds `.gotmp/` to `.gitignore`. **Settled.** | Put caches outside repo or ignore before first command. | -| Tracked `docs/node_modules`, 2026-07-28 | Hundreds of pnpm symlink entries become repository content. | `docs/node_modules` neither ignored nor excluded from commit. | 237 entries at `HEAD`; no `.gitignore` rule. **Current/open.** | Track lockfiles, never installed trees. | -| `308c199` cleanup regression, 2026-07-28 | `TestRunWrapsExternalCommand` returns `"\n"` instead of `"wrapped\n"`; forge compatibility removed again; dependency artifacts added. | Semantic edits classified and reviewed as cleanup. | Focused skillscript test fails at `HEAD`. **Current/open.** | No commit is "cleanup" for gating purposes. | +| Tracked `docs/node_modules`, 2026-07-28 | Hundreds of pnpm symlink entries become repository content. | `docs/node_modules` neither ignored nor excluded from commit. | 237 entries at that time; the VitePress site was removed (`84b32694`) and `git ls-files | grep -c node_modules` is now **0**. **Settled.** | Track lockfiles, never installed trees. | +| `308c199` cleanup regression, 2026-07-28 | `TestRunWrapsExternalCommand` returns `"\n"` instead of `"wrapped\n"`; forge compatibility removed again; dependency artifacts added. | Semantic edits classified and reviewed as cleanup. | The focused skillscript test **passes** as of 2026-09-16. **Settled.** | No commit is "cleanup" for gating purposes. | | `diff_cap_lines = 0` contradiction, introduced at project start | Operators set zero expecting unlimited, but decoded config becomes 400. | `finalize` converts zero to default 400; same sentinel means both "unset" and "unlimited." | **Current/open** contradiction. | Never overload zero value with both defaulting and explicit policy. | ## Keep unmerged work as dated candidates -As of **2026-07-28**, these remote branches are not merged: +As of **2026-09-18**, the six `fix/4x` and `feat/5x`/`feat/75` candidate +branches recorded here on 2026-07-28 no longer exist on either remote — +`git branch -r --no-merged HEAD` reports only `gitea/main` (and the Dolt helper +ref). Treat their concerns as `superseded` unless a live branch re-raises one. -| Candidate branch | Claimed concern | +| Former candidate branch | Claimed concern | |---|---| | `origin/fix/43-loaddir-no-timeout-around-yaegi-eval-can-hang-daemon-startup` | Bound Yaegi evaluation during `LoadDir` | | `origin/fix/45-skillscriptrun-ignores-context-cannot-be-canceledtimed-out` | Make skill scripts cancellable | diff --git a/.agents/skills/archie-run-and-operate/SKILL.md b/.agents/skills/archie-run-and-operate/SKILL.md index 3e2bcb4a3..79c200cbd 100644 --- a/.agents/skills/archie-run-and-operate/SKILL.md +++ b/.agents/skills/archie-run-and-operate/SKILL.md @@ -5,8 +5,9 @@ description: "Run and observe Archie safely with a host daemon and the repositor # Run and operate Archie -Use this runbook to operate the implementation that exists on **2026-07-28**. -Treat source and tests as current; treat `docs/archive/` as history. +Use this runbook to operate the implementation that exists on **2026-09-18** +(HEAD `2e1e1549`). Treat source and tests as current; treat `docs/archive/` as +history. ## Define the process shapes @@ -62,7 +63,7 @@ Both linked binaries may also display Go's `-quickchecks` flag from ## Select the execution mode deliberately -| Config shape | What runs | Operational status on 2026-07-28 | +| Config shape | What runs | Operational status on 2026-09-18 | |---|---|---| | `agent.mode = "inprocess"` and no `[nats]` | `archied` polls and claims from SQLite, runs workflow stages and model tools in-process. | **Production-wired candidate.** No process or OS isolation. | | `agent.mode = "subprocess"` | `archied` starts `agent.command` per stage and expects one JSON invocation on stdin, one response on stdout. | **Open/broken with default command.** `cmd/archie-agent` is a long-running NATS worker and never calls `agentexec.ServeOne`. | @@ -179,7 +180,7 @@ for `archie.task.>` and `archie.agent.>`. | Discovery | `archie.task.bug`, `.feature`, `.bootstrap`, or `.default`; daemon durable consumer `archie-daemon`, 5-min ack wait, max 3 deliveries. | | Per-stage requests | `archie.agent..request`; worker durable consumer defaults to `archie-agent`, 30-min ack wait, max 3 deliveries; response address in `X-Archie-Reply`. | | Container task | Core NATS request/reply on `archie.taskrun.`. Task container reads `/data/worktree/.git/task.json` and creates dedicated subscription; shared worker uses queue group `archie-taskrun-workers`. | -| Privileged RPC | Store, forge, and worktree operations return to `archied` over core NATS subjects in `internal/storerpc`, `internal/forgerpc`, and `internal/worktreerpc`. | +| Privileged RPC | Store operations go to the gRPC State Store (`internal/infrastructure/staterpc`, task-scoped grants); forge and worktree operations return to `archied` over core NATS subjects in `internal/forgerpc` and `internal/worktreerpc`. | Set `containers.network` explicitly. `deployments/docker-nats-stack.toml` uses `archie-core_default`. Auto-detection inspects the daemon container; on failure, @@ -195,8 +196,10 @@ It forwards only configured provider-key variables. Open limitations until code/tests prove otherwise: - `subprocess` + default `archie-agent` is protocol-incompatible. -- `config.production.toml` uses legacy `token_env`; validation requires - `forge.token` secret reference. +- A deployed overlay previously using legacy `token_env` still converts: the + structured `forge.token` secret reference wins when both are present and + `config.Forge.TokenEnv` is otherwise converted by + `internal/infrastructure/configuration/defaults.go`. - Container RPC uses root forge/worktree for every identity. - Container orphan recovery lacks daemon-instance label. - `max_uptime` only bounds `Pool.Acquire` create/start; no lifetime timer. @@ -206,9 +209,12 @@ Open limitations until code/tests prove otherwise: ## Understand deployment and rollback seams -The Gitea workflow builds/pushes only `archied:latest` and `archie-agent:latest`. -It injects component versions into labels/metadata but publishes no versioned -tags. It runs no quality gate before Docker builds. +`.github/workflows/deploy.yml` gates on the release tags, then builds/pushes +`archied:latest` and `archie-agent:latest` plus a versioned +`linux/amd64` distribution zip attached to a GitHub Release. It injects +component versions into labels/metadata and runs the quality gate before the +Docker builds. The Gitea workflow that used to be the production build path was +dropped (`dd9bddc2`); `deployments/*.toml` are the supported profiles. Before promotion: record running image IDs/digests; config checksums (no secrets); task counts via safe GET; establish a SQLite backup procedure. @@ -218,6 +224,6 @@ does not undo forge changes, task transitions, or data-format changes. ## Treat carina facts as unverified external state -Repository guidance (2026-07-28): two instances on `carina` with distinct +Repository guidance (2026-09-18): two instances on `carina` with distinct `bot_user` values. No live process list, config, image digest, service manager, or deployment was verified. Do not invent SSH, systemd, or service commands. diff --git a/docs/data/generated/contracts.json b/docs/data/generated/contracts.json index 626b8f565..6101879dd 100644 --- a/docs/data/generated/contracts.json +++ b/docs/data/generated/contracts.json @@ -142,7 +142,8 @@ "task_id", "attempt", "stage", - "status" + "status", + "usage" ], "type": "object" }, @@ -257,7 +258,7 @@ "type": "integer" }, "height": { - "description": "Height is the pixel height for images and video.", + "description": "Height is the pixel height for images and video. nil means unknown\nor not applicable.", "type": "integer" }, "mime_type": { @@ -268,6 +269,10 @@ "description": "Path is a local filesystem path on the host that produced the\nattachment. It is the alternative to URL for outbound delivery: a\nURL is fetched by the platform, a Path is uploaded by us. A file the\nagent wrote has no URL and never gets one -- publishing it would\ntrade a delivery problem for a hosting and access-control one -- so\nsenders that support upload must read this.\n\nExactly one of URL and Path is meaningful for an outbound send.", "type": "string" }, + "raw": { + "description": "Raw carries platform-specific fields the common vocabulary does not\ncover, so adapters never need a second attachment type.", + "type": "object" + }, "type": { "description": "Type is \"image\", \"video\", \"audio\", or \"document\".", "type": "string" From 0f629a8fa955ff2ac13e9cd3024705ed6ba51e03 Mon Sep 17 00:00:00 2001 From: Sam Catlow Date: Fri, 18 Sep 2026 12:41:08 +1000 Subject: [PATCH 2/4] feat(docsgen): add a non-destructive check mode for generated drift `docs/data/generated/contracts.json` sat stale since a825db28 with nothing to notice: no task, no test, and no CI step regenerates or compares it, so the committed contract reference quietly drifted from the Go types it claims to describe. It was only found by hand-running the generator. Implements Step 6 of docs/architecture/generated-documentation.md, which already specified this command and closed the gap filed in docs/prds/contract-conformance-audit.md ("`docsgen check` is documented and does not exist"): 1. generates into a temporary directory; 2. normalizes line endings; 3. compares the schema trees with the committed output; 4. reports missing, changed, and obsolete files; 5. names the authoritative definition behind each mismatch; 6. exits non-zero without modifying the working tree. The check never writes to the working tree, so the author applies the regenerated bytes rather than the gate silently doing it. That is why this is a `check` and not a CI auto-update: the code change that shifted the contract surface should be reviewed with its regenerated artifact in one commit, and a bot commit would bless an unintended contract change with no reviewer ever seeing it. Step 9 states the same prohibition directly. Diagnostics name the Go type, so a failure says where to look: docs/data/generated/contracts.json is stale; regenerate it with: go -C tools run ./docsgen --repo-root .. schema "AgentExecutionResult" changed (authoritative definition: github.com/samcharles93/archie-core/internal/agentexec.Result) schema "MediaAttachment" changed (authoritative definition: reached from .../internal/gateway.MessageEvent) Nested definitions are attributed through the published contract that references them, iterated in sorted order so the diagnostic is deterministic when several contracts share a definition. Usage: `go -C tools run ./docsgen check --repo-root ..`, plus the existing `--out` to compare an arbitrary artifact. Reuses the write path rather than reimplementing encoding, so check and generate cannot disagree about the expected bytes. Tests cover: the committed artifact is current (the regression guard that would have caught this), changed/removed/missing/obsolete detection, and that a failing check leaves the artifact's bytes, mode, and directory contents untouched. `task docs:check` wiring into `task check` is deliberately not in this commit: Taskfile.yml is being edited concurrently in this working tree. Verified with `task check` (green). --- tools/docsgen/main.go | 267 ++++++++++++++++++++++++++++++++++++- tools/docsgen/main_test.go | 150 +++++++++++++++++++++ 2 files changed, 410 insertions(+), 7 deletions(-) diff --git a/tools/docsgen/main.go b/tools/docsgen/main.go index 4c8dd21ed..5973ea089 100644 --- a/tools/docsgen/main.go +++ b/tools/docsgen/main.go @@ -2,9 +2,12 @@ package main import ( + "bytes" "encoding/json" + "errors" "flag" "fmt" + "io/fs" "log" "os" "path/filepath" @@ -18,18 +21,252 @@ import ( "github.com/samcharles93/archie-core/internal/gateway" ) +// defaultOutputPath is the committed artifact docsgen owns, relative to the +// repository root. +const defaultOutputPath = "docs/data/generated/contracts.json" + +type mode int + +const ( + modeWrite mode = iota + modeCheck +) + func main() { - repoRoot := flag.String("repo-root", "..", "path to the Archie repository root") - out := flag.String( + args := os.Args[1:] + selected := modeWrite + // `check` is a subcommand, not a flag: it selects a comparison instead of a + // write, and the flag package would treat it as a positional argument. + if len(args) > 0 && args[0] == "check" { + selected = modeCheck + args = args[1:] + } + + flags := flag.NewFlagSet("docsgen", flag.ExitOnError) + repoRoot := flags.String("repo-root", "..", "path to the Archie repository root") + out := flags.String( "out", - "docs/data/generated/contracts.json", + defaultOutputPath, "output path relative to the repository root", ) - flag.Parse() - - if err := run(*repoRoot, *out); err != nil { + if err := flags.Parse(args); err != nil { log.Fatalf("docsgen: %v", err) } + + var err error + if selected == modeCheck { + err = check(*repoRoot, *out) + } else { + err = run(*repoRoot, *out) + } + if err != nil { + log.Fatalf("docsgen: %v", err) + } +} + +// check regenerates the artifact into a temporary directory and compares it +// with the committed output. It never writes to the working tree, so a failing +// check leaves the tree exactly as it found it -- the author, not the gate, +// applies the regenerated bytes. +func check(repoRoot, outPath string) error { + absRoot, err := filepath.Abs(repoRoot) + if err != nil { + return fmt.Errorf("resolve repository root: %w", err) + } + + committedPath := outPath + if !filepath.IsAbs(committedPath) { + committedPath = filepath.Join(absRoot, committedPath) + } + relative := displayPath(absRoot, committedPath) + + tmpDir, err := os.MkdirTemp("", "docsgen-check-") + if err != nil { + return fmt.Errorf("create temporary directory: %w", err) + } + defer func() { _ = os.RemoveAll(tmpDir) }() + + // Reuse the write path so check and generate can never disagree about + // what the committed bytes should be. + generatedPath := filepath.Join(tmpDir, filepath.Base(committedPath)) + if err := write(absRoot, generatedPath, true); err != nil { + return err + } + expected, err := os.ReadFile(generatedPath) + if err != nil { + return fmt.Errorf("read generated artifact: %w", err) + } + + committed, err := os.ReadFile(committedPath) + if errors.Is(err, fs.ErrNotExist) { + return fmt.Errorf( + "%s is missing; regenerate it with: go -C tools run ./docsgen --repo-root ..", + relative, + ) + } + if err != nil { + return fmt.Errorf("read %s: %w", relative, err) + } + + problems := compareArtifacts(relative, expected, committed) + problems = append(problems, obsoleteArtifacts(absRoot, committedPath)...) + if len(problems) == 0 { + fmt.Printf("docsgen: %s is up to date\n", relative) + return nil + } + + return fmt.Errorf( + "%s is stale; regenerate it with: go -C tools run ./docsgen --repo-root ..\n %s", + relative, + strings.Join(problems, "\n "), + ) +} + +// compareArtifacts reports every schema whose generated form differs from the +// committed form, plus a whole-file difference the schema view cannot see +// (whitespace or key ordering). +func compareArtifacts(relative string, expected, committed []byte) []string { + expectedSchemas, err := decodeSchemas(expected) + if err != nil { + return []string{fmt.Sprintf("%s: generated output is unreadable: %v", relative, err)} + } + committedSchemas, err := decodeSchemas(committed) + if err != nil { + return []string{fmt.Sprintf("%s: committed output is unreadable: %v", relative, err)} + } + + provenance := schemaProvenance(expectedSchemas) + var problems []string + for _, name := range sortedSchemaNames(expectedSchemas) { + committedSchema, ok := committedSchemas[name] + if !ok { + problems = append(problems, fmt.Sprintf( + "schema %q is missing from the committed output (%s)", + name, describeProvenance(provenance, name), + )) + continue + } + if !reflect.DeepEqual(expectedSchemas[name], committedSchema) { + problems = append(problems, fmt.Sprintf( + "schema %q changed (%s)", + name, describeProvenance(provenance, name), + )) + } + } + for _, name := range sortedSchemaNames(committedSchemas) { + if _, ok := expectedSchemas[name]; !ok { + problems = append(problems, fmt.Sprintf( + "schema %q is obsolete: no Archie-owned contract defines it", name, + )) + } + } + + if len(problems) == 0 && !bytes.Equal(expected, committed) { + problems = append(problems, "content differs from generated form (whitespace or key ordering)") + } + return problems +} + +// obsoleteArtifacts reports committed generated files in the artifact's +// directory that the generator no longer produces, so a renamed or deleted +// artifact cannot linger as dead documentation. +func obsoleteArtifacts(absRoot, committedPath string) []string { + entries, err := os.ReadDir(filepath.Dir(committedPath)) + if err != nil { + return nil + } + owned := filepath.Base(committedPath) + var problems []string + for _, entry := range entries { + if entry.IsDir() || entry.Name() == owned { + continue + } + if filepath.Ext(entry.Name()) != ".json" { + continue + } + full := filepath.Join(filepath.Dir(committedPath), entry.Name()) + problems = append(problems, fmt.Sprintf( + "%s is obsolete: no generator in tools/docsgen produces it", + displayPath(absRoot, full), + )) + } + return problems +} + +// schemaProvenance names the authoritative Go definition behind each published +// schema, so a drift report points at the code that must be reviewed rather +// than only at the generated file. +func schemaProvenance(schemas map[string]any) map[string]string { + provenance := map[string]string{} + types := currentContractTypes() + names := make([]string, 0, len(types)) + for name := range types { + names = append(names, name) + } + // Sorted: a nested definition referenced by several contracts would + // otherwise be attributed to whichever contract map iteration reached + // first, making the diagnostic nondeterministic. + sort.Strings(names) + for _, name := range names { + t := types[name] + provenance[name] = t.PkgPath() + "." + t.Name() + } + for _, name := range names { + walkSchemaRefs(schemas[name], func(target string) { + if _, known := provenance[target]; known { + return + } + provenance[target] = "reached from " + provenance[name] + }) + } + return provenance +} + +func describeProvenance(provenance map[string]string, name string) string { + if source, ok := provenance[name]; ok { + return "authoritative definition: " + source + } + return "no authoritative definition found" +} + +func walkSchemaRefs(value any, visit func(target string)) { + switch node := value.(type) { + case map[string]any: + for key, child := range node { + if key == "$ref" { + if ref, ok := child.(string); ok { + if target, found := strings.CutPrefix(ref, "#/schemas/"); found { + visit(target) + } + } + continue + } + walkSchemaRefs(child, visit) + } + case []any: + for _, child := range node { + walkSchemaRefs(child, visit) + } + } +} + +func decodeSchemas(body []byte) (map[string]any, error) { + var data struct { + Schemas map[string]any `json:"schemas"` + } + if err := json.Unmarshal(body, &data); err != nil { + return nil, err + } + return data.Schemas, nil +} + +func sortedSchemaNames(schemas map[string]any) []string { + names := make([]string, 0, len(schemas)) + for name := range schemas { + names = append(names, name) + } + sort.Strings(names) + return names } func run(repoRoot, outPath string) error { @@ -37,7 +274,21 @@ func run(repoRoot, outPath string) error { if err != nil { return fmt.Errorf("resolve repository root: %w", err) } + return write(absRoot, outPath, false) +} +// displayPath prefers a repository-relative path for diagnostics, falling back +// to the path as given when the target sits outside the repository (a temp +// directory under check, or an explicit --out). +func displayPath(absRoot, path string) string { + relative, err := filepath.Rel(absRoot, path) + if err != nil || strings.HasPrefix(relative, "..") { + return filepath.ToSlash(path) + } + return filepath.ToSlash(relative) +} + +func write(absRoot, outPath string, quiet bool) error { reflector := &jsonschema.Reflector{ ExpandedStruct: true, DoNotReference: false, @@ -83,7 +334,9 @@ func run(repoRoot, outPath string) error { return fmt.Errorf("write %s: %w", destination, err) } - fmt.Printf("docsgen: wrote %d schemas to %s\n", len(schemas), destination) + if !quiet { + fmt.Printf("docsgen: wrote %d schemas to %s\n", len(schemas), destination) + } return nil } diff --git a/tools/docsgen/main_test.go b/tools/docsgen/main_test.go index 6e2d1189f..8fb3486bb 100644 --- a/tools/docsgen/main_test.go +++ b/tools/docsgen/main_test.go @@ -180,6 +180,156 @@ func TestRunReportsFilesystemAndSourceErrors(t *testing.T) { } } +func TestCheckAcceptsTheCommittedArtifact(t *testing.T) { + repoRoot := filepath.Clean("../..") + + if err := check(repoRoot, defaultOutputPath); err != nil { + t.Fatalf("check(%q) error = %v; committed generated data is stale", defaultOutputPath, err) + } +} + +func TestCheckReportsChangedSchema(t *testing.T) { + repoRoot := filepath.Clean("../..") + output := filepath.Join(t.TempDir(), "contracts.json") + if err := run(repoRoot, output); err != nil { + t.Fatalf("run() error = %v", err) + } + mutateGeneratedData(t, output, func(data map[string]any) { + schemas := data["schemas"].(map[string]any) + schemas["AgentExecutionResult"].(map[string]any)["title"] = "tampered" + }) + + err := check(repoRoot, output) + if err == nil { + t.Fatal("check() returned nil error for a changed artifact") + } + for _, want := range []string{"changed", "AgentExecutionResult", "agentexec.Result"} { + if !strings.Contains(err.Error(), want) { + t.Errorf("check() error = %q, want substring %q", err, want) + } + } +} + +func TestCheckReportsRemovedSchema(t *testing.T) { + repoRoot := filepath.Clean("../..") + output := filepath.Join(t.TempDir(), "contracts.json") + if err := run(repoRoot, output); err != nil { + t.Fatalf("run() error = %v", err) + } + mutateGeneratedData(t, output, func(data map[string]any) { + delete(data["schemas"].(map[string]any), "MessageEvent") + }) + + err := check(repoRoot, output) + if err == nil { + t.Fatal("check() returned nil error for a removed schema") + } + if !strings.Contains(err.Error(), "MessageEvent") { + t.Errorf("check() error = %q, want the removed schema named", err) + } +} + +func TestCheckReportsMissingArtifact(t *testing.T) { + repoRoot := filepath.Clean("../..") + output := filepath.Join(t.TempDir(), "contracts.json") + + err := check(repoRoot, output) + if err == nil { + t.Fatal("check() returned nil error for a missing artifact") + } + if !strings.Contains(err.Error(), "missing") { + t.Fatalf("check() error = %q, want it to report a missing artifact", err) + } + if _, statErr := os.Stat(output); statErr == nil { + t.Fatal("check() created the missing artifact instead of reporting it") + } +} + +func TestCheckReportsObsoleteArtifact(t *testing.T) { + repoRoot := filepath.Clean("../..") + outputDir := t.TempDir() + if err := run(repoRoot, filepath.Join(outputDir, "contracts.json")); err != nil { + t.Fatalf("run() error = %v", err) + } + obsolete := filepath.Join(outputDir, "orphaned.json") + if err := os.WriteFile(obsolete, []byte("{}\n"), 0o600); err != nil { + t.Fatalf("write obsolete artifact: %v", err) + } + + err := check(repoRoot, filepath.Join(outputDir, "contracts.json")) + if err == nil { + t.Fatal("check() returned nil error for an obsolete artifact") + } + if !strings.Contains(err.Error(), "obsolete") || !strings.Contains(err.Error(), "orphaned.json") { + t.Fatalf("check() error = %q, want the obsolete file named", err) + } +} + +func TestCheckNeverModifiesTheWorkingTree(t *testing.T) { + repoRoot := filepath.Clean("../..") + outputDir := t.TempDir() + output := filepath.Join(outputDir, "contracts.json") + if err := run(repoRoot, output); err != nil { + t.Fatalf("run() error = %v", err) + } + mutateGeneratedData(t, output, func(data map[string]any) { + data["generatedBy"] = "tampered" + }) + before, err := os.ReadFile(output) + if err != nil { + t.Fatalf("read before: %v", err) + } + beforeInfo, err := os.Stat(output) + if err != nil { + t.Fatalf("stat before: %v", err) + } + + if err := check(repoRoot, output); err == nil { + t.Fatal("check() returned nil error for a changed artifact") + } + + after, err := os.ReadFile(output) + if err != nil { + t.Fatalf("read after: %v", err) + } + if !bytes.Equal(before, after) { + t.Fatal("check() rewrote the artifact under test") + } + afterInfo, err := os.Stat(output) + if err != nil { + t.Fatalf("stat after: %v", err) + } + if beforeInfo.Mode() != afterInfo.Mode() { + t.Fatalf("check() changed the artifact mode: %v -> %v", beforeInfo.Mode(), afterInfo.Mode()) + } + if entries, err := os.ReadDir(outputDir); err != nil { + t.Fatalf("read output dir: %v", err) + } else if len(entries) != 1 { + t.Fatalf("check() left %d files in the output dir, want 1", len(entries)) + } +} + +func mutateGeneratedData(t *testing.T, path string, mutate func(map[string]any)) { + t.Helper() + + raw, err := os.ReadFile(path) + if err != nil { + t.Fatalf("read %s: %v", path, err) + } + var data map[string]any + if err := json.Unmarshal(raw, &data); err != nil { + t.Fatalf("decode %s: %v", path, err) + } + mutate(data) + body, err := json.MarshalIndent(data, "", " ") + if err != nil { + t.Fatalf("encode %s: %v", path, err) + } + if err := os.WriteFile(path, append(body, '\n'), 0o600); err != nil { + t.Fatalf("write %s: %v", path, err) + } +} + func TestRewriteRef(t *testing.T) { tests := map[string]string{ "definition": "#/schemas/Contract", From 52aa774afc22e295aae9ec5159b67ba0afc369d5 Mon Sep 17 00:00:00 2001 From: Sam Catlow Date: Fri, 18 Sep 2026 12:43:54 +1000 Subject: [PATCH 3/4] feat(build): gate generated contract data in task check Closes #855 (archie-core-5gzx). The check mode added in the preceding commit had no caller, so it prevented nothing: a stale contracts.json still passed every gate. This wires it in and makes drift fail the build. - `task docs:generate` writes the committed artifact. - `task docs:check` verifies it, non-destructively. - `task check` runs `docs:check`, right after `proto:check`, so both generated-artifact surfaces are judged the same way in the same place. The check cannot be satisfied by rewriting: it generates into a temporary directory, so a failing gate leaves the artifact exactly as it found it and the author applies the regenerated bytes. That keeps the regenerated diff reviewable in the same commit as the code change that caused it, which is what Step 9 of the architecture document requires and what a CI auto-update would quietly bypass. Documents that called this gap open are updated rather than left to contradict the code: - `generated-documentation.md` Step 8 (no `docs:*` task exists -> both exist), Step 9 (drift is gated locally, not in CI), and the hook/CI policy (rule 3 is now enforced by the gate; rules 1, 2, and 4 still rely on the author, since nothing observes a commit). - `contract-conformance-audit.md` gap 1, now closed. - `config-example-drift.md` item 3, which asked for exactly this wiring. - `CLAUDE.md`, whose `task check` description listed neither `docs:check` nor `test:tools`. - The build, docs, and diagnostics skills, plus a re-baselined diagnostics snapshot: the `task_docs` delivery surface went 0 -> 6 and the `tools` module 4 -> 7 now that the tasks exist. Verified with `task check` (green), including the new `docs:check` step. --- .agents/skills/archie-build-and-env/SKILL.md | 18 +++++++--- .../archie-diagnostics-and-tooling/SKILL.md | 10 +++--- .../skills/archie-docs-and-writing/SKILL.md | 19 ++++++---- CLAUDE.md | 2 +- Taskfile.yml | 32 ++++++++++++++++- docs/architecture/generated-documentation.md | 35 ++++++++++++------- docs/prds/config-example-drift.md | 8 +++-- docs/prds/contract-conformance-audit.md | 11 +++--- 8 files changed, 95 insertions(+), 40 deletions(-) diff --git a/.agents/skills/archie-build-and-env/SKILL.md b/.agents/skills/archie-build-and-env/SKILL.md index a62610dce..33034224e 100644 --- a/.agents/skills/archie-build-and-env/SKILL.md +++ b/.agents/skills/archie-build-and-env/SKILL.md @@ -109,13 +109,13 @@ golangci-lint run ./... | `task lint` | `golangci-lint run ./...`. | | `task build` | Builds both commands into `bin/`. | | `task test` | `go test ./... -count=1` in the runtime module. | -| `task check` | `fmt` + `go fix ./...` again + `proto:lint` + `proto:check` + `vet` + `lint` + `build` + `test` + `test:tools` + `test:ui`. | +| `task check` | `fmt` + `go fix ./...` again + `proto:lint` + `proto:check` + `docs:check` + `vet` + `lint` + `build` + `test` + `test:tools` + `test:ui`. | | `task clean` | Recursively removes `bin/`; destructive. | | `task docker-build` | `docker compose build agent` only. | `task check` is the definitive gate but omits race tests, `task vuln`, and -`task docker-build`. It has no docs step: there is no documentation build to -omit. +`task docker-build`. Its docs step, `docs:check`, verifies committed generated +data and never renders anything: there is still no documentation build. Run only in authorized writable worktree. For read-only preview: `gofumpt -l .`. ## Verify the tools module separately @@ -148,8 +148,16 @@ cmp --silent docs/data/generated/contracts.json /tmp/archie-core-contracts.json ``` 2026-09-18 run wrote 11 schemas. The committed output was stale since -`4b340d2b` and was regenerated the same day; `cmp` now passes. Planned -`docsgen all` and `docsgen check` do **not** exist yet. +`4b340d2b` and was regenerated the same day; `cmp` now passes. `task docs:check` +now runs inside `task check`, so this drift fails the gate instead of waiting to +be found by hand. `docsgen all` does not exist yet. + +Prefer the task over the raw commands: + +```bash +task docs:check # non-destructive; fails loudly and names the Go type +task docs:generate # rewrites the committed artifact; review the diff +``` ## Documentation needs no build diff --git a/.agents/skills/archie-diagnostics-and-tooling/SKILL.md b/.agents/skills/archie-diagnostics-and-tooling/SKILL.md index f6df6d084..03697c9ce 100644 --- a/.agents/skills/archie-diagnostics-and-tooling/SKILL.md +++ b/.agents/skills/archie-diagnostics-and-tooling/SKILL.md @@ -242,7 +242,7 @@ These are post-fix numbers: earlier runs of the same scripts reported roughly | `internal/app/archied.Run` body span / approximate complexity | 97 / 19 | | Root internal packages / direct internal edges | 110 / 316 | | Tagged config field rows | 194 | -| Tracked paths / tracked `node_modules` paths / tracked symlinks | 1,087 / 0 / 1 | +| Tracked paths / tracked `node_modules` paths / tracked symlinks | 1,088 / 0 / 1 | | `.dockerignore` present | 1 | `source-metrics.go` and `git ls-files` reconcile here: 408 + 9 generated @@ -255,10 +255,10 @@ production lines. Editing a bundled script therefore moves `production_physical_lines` on its own, so re-baseline after any script change rather than reading the delta as a change in the product code. -The delivery snapshot found no Taskfile literals for race or docs, but four -for the `tools` module; GitHub `go gate` 4 and container publish 3; and -non-zero literals for all five composition-root anchors under -`internal/app/archied`. +The delivery snapshot found no Taskfile literals for race; docs now has 6 and +the `tools` module 7, both grown by wiring `task docs:check` into the gate; +GitHub `go gate` 4 and container publish 3; and non-zero literals for all five +composition-root anchors under `internal/app/archied`. The focused test `go test ./internal/skillscript -run '^TestRunWrapsExternalCommand$' -count=1 -v` diff --git a/.agents/skills/archie-docs-and-writing/SKILL.md b/.agents/skills/archie-docs-and-writing/SKILL.md index c0cf9354f..18cbf7360 100644 --- a/.agents/skills/archie-docs-and-writing/SKILL.md +++ b/.agents/skills/archie-docs-and-writing/SKILL.md @@ -140,21 +140,26 @@ Current behavior, verified 2026-09-18: - Tool path is `tools/docsgen`, not `tools/cmd/docsgen`. - Separate Go module replacing root module with `../`. -- Accepts `--repo-root` and `--out`; no implemented `data`, `asyncapi`, `all`, - or `check` subcommands. +- Accepts `--repo-root` and `--out`, plus a `check` subcommand that compares + against the committed artifact without writing. No implemented `data`, + `asyncapi`, or `all` subcommands. - `currentContractTypes` publishes three top-level contracts and collects referenced schemas into one JSON object. - Default output is `docs/data/generated/contracts.json`. ```bash GOTMPDIR=/tmp GOCACHE=/tmp/archie-docsgen-gocache go -C tools test -mod=readonly ./docsgen -count=1 -docs_tmp="$(mktemp /tmp/archie-contracts.XXXXXX.json)" -GOTMPDIR=/tmp GOCACHE=/tmp/archie-docsgen-gocache go -C tools run -mod=readonly ./docsgen --repo-root .. --out "$docs_tmp" -cmp "$docs_tmp" docs/data/generated/contracts.json +task docs:check # fails loudly, and names the Go type behind each mismatch ``` -Treat the PRD's `docsgen all`, `docsgen check`, and `task docs:*` commands as -`APPROVED TARGET`. +`docsgen check` and the `docs:generate`/`docs:check` tasks are implemented, and +`task check` runs `docs:check` — generated drift is gated. `docsgen data`, +`docsgen asyncapi`, and `docsgen all` remain `APPROVED TARGET`. + +The check is deliberately non-destructive: it generates into a temporary +directory, so a failing gate never rewrites the artifact it judges. Fix a +failure with `task docs:generate` and commit the result with the code change +that shifted the contract surface. ## Store documentation as repository Markdown diff --git a/CLAUDE.md b/CLAUDE.md index bb4e9596b..ce7282cd1 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -94,7 +94,7 @@ task ui # build dashboard into ui/dist (LAW asset) task fmt # gofumpt -w . && go fix ./... task vet # go vet ./... task lint # golangci-lint run ./... -task check # fmt + go fix + proto:lint + proto:check + vet + lint + build + test + test:ui (The Definitive Gate) +task check # fmt + go fix + proto:lint + proto:check + docs:check + vet + lint + build + test + test:tools + test:ui (The Definitive Gate) task dev # archied live-reload + Vite HMR together on :5173 ``` diff --git a/Taskfile.yml b/Taskfile.yml index 4356c12d4..4683f8f95 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -35,6 +35,35 @@ tasks: cmds: - bash tools/proto.sh check + docs:generate: + desc: Regenerate the committed contract reference from Archie-owned types + summary: | + Writes docs/data/generated/contracts.json. The output is committed, so + this dirties the tree deliberately: review the diff and commit it with + the code change that shifted the contract surface. Never hand-edit the + artifact. + + -mod=readonly is mandatory on every `go -C tools` command; see + task test:tools for why. + cmds: + - go -C tools run -mod=readonly ./docsgen --repo-root .. + + docs:check: + desc: Verify the committed contract reference matches Archie-owned types + summary: | + Implements Step 6 of docs/architecture/generated-documentation.md: + generates into a temporary directory, compares it with the committed + output, names the authoritative Go definition behind each mismatch, and + exits non-zero without modifying the working tree. + + Fix a failure with `task docs:generate`, then commit the result in the + same commit as the code change that shifted the contract surface. This + checks rather than regenerates on purpose: a gate that silently rewrote + the artifact would bless an unintended contract change with no reviewer + ever seeing it, which Step 9 explicitly forbids. + cmds: + - go -C tools run -mod=readonly ./docsgen check --repo-root .. + vet: desc: Run go vet cmds: @@ -226,12 +255,13 @@ tasks: - go -C tools test -mod=readonly ./... -count=1 check: - desc: "Full quality gate: fmt, go fix, proto, vet, lint, build, test, tools tests, ui tests" + desc: "Full quality gate: fmt, go fix, proto, docs, vet, lint, build, test, tools tests, ui tests" cmds: - task: fmt - go fix ./... - task: proto:lint - task: proto:check + - task: docs:check - task: vet - task: lint - task: build diff --git a/docs/architecture/generated-documentation.md b/docs/architecture/generated-documentation.md index 291ff752d..473caef53 100644 --- a/docs/architecture/generated-documentation.md +++ b/docs/architecture/generated-documentation.md @@ -327,27 +327,35 @@ When a renderer is selected it MUST: ### Step 8: developer commands -No `docs:*` task exists. `Taskfile.yml` does not reference the generator, and -`task check` does not run `docs:check`, so generated drift is currently ungated. - -The generator is run directly: +`task docs:generate` writes the committed artifact; `task docs:check` verifies +it. `task check` runs `docs:check`, so generated drift is gated rather than +advisory: a stale artifact fails the gate and names the authoritative Go +definition behind each mismatch. ```sh -go -C tools run ./docsgen all --repo-root .. -go -C tools run ./docsgen check --repo-root .. +task docs:generate +task docs:check ``` -Wiring `docs:check` into `task check` is worth doing and does not depend on the -renderer decision. +Both wrap the generator, which can still be run directly: + +```sh +go -C tools run -mod=readonly ./docsgen --repo-root .. +go -C tools run -mod=readonly ./docsgen check --repo-root .. +``` + +`docsgen all` is still target-only: `docsgen data` and `docsgen asyncapi` do not +exist, and only the single `contracts.json` artifact is generated today. ### Step 9: CI The documentation workflow was deleted on 2026-09-12. No workflow builds, checks, or publishes documentation. `deploy.yml` does not reference `docs/**`. -Reintroducing CI for documentation requires the renderer decision first. Any -such workflow MUST check generated output and MUST NOT regenerate and silently -publish uncommitted differences. +Drift is therefore gated locally, by `task check` running `docs:check` — not in +CI. Reintroducing CI for documentation requires the renderer decision first. +Any such workflow MUST check generated output and MUST NOT regenerate and +silently publish uncommitted differences. ### Step 10: cutover — done, in reverse @@ -373,11 +381,12 @@ Documents MUST describe Archie. Publishing is a separate, undecided concern. Generated documentation is committed. No pre-commit hook is installed in this repository, and no CI step runs the -generator. The policy below is the intended contract, currently unenforced: +generator. Step 3 below is enforced by `task check`; steps 1, 2, and 4 still +rely on the author, because nothing observes a commit: 1. detects authoritative definition or generator changes; 2. runs the generator; -3. runs `docsgen check`; +3. runs `docsgen check` — enforced by `task check`; 4. refuses the commit when generated changes are unstaged or validation fails. A commit MUST NOT be created while generated documentation is stale. diff --git a/docs/prds/config-example-drift.md b/docs/prds/config-example-drift.md index 4897475e7..b549338f0 100644 --- a/docs/prds/config-example-drift.md +++ b/docs/prds/config-example-drift.md @@ -78,9 +78,11 @@ empty, with an explicit allowlist for keys another binary owns. This is the mechanism that would have caught a renamed key left behind in an example, and it needs no inventory. -**3. Gate generated data in `task check`.** Add `docs:check` to the gate, closing -the admitted `generated-documentation.md:331` hole. Cheap, already implemented as -`go -C tools run ./docsgen check`. +**3. Gate generated data in `task check`.** Done. `task docs:check` runs in the +gate, closing the admitted "generated drift is currently ungated" hole. The +check mode did not exist when this was written; it was added for issue #855, +which also made the check non-destructive so the gate can never rewrite the +artifact it is judging. **4. Explicitly NOT now: a generated key inventory.** `configuration.md`'s completion criteria want "generated documentation lists every external key, diff --git a/docs/prds/contract-conformance-audit.md b/docs/prds/contract-conformance-audit.md index 98f812343..1f4afcb24 100644 --- a/docs/prds/contract-conformance-audit.md +++ b/docs/prds/contract-conformance-audit.md @@ -93,11 +93,12 @@ the survey, not about the tree. ## The two gaps this was aimed at -**1. `docsgen check` is documented and does not exist.** Still open, filed as -`archie-core-5gzx`. `generated-documentation.md:118-121` advertises -`docsgen data|asyncapi|all|check`; `tools/docsgen/main.go` has exactly two flags -and always writes, so the documented check mode cannot run and line 331's -"generated drift is currently ungated" stands. +**1. `docsgen check` is documented and does not exist.** Closed by issue #855. +(issue #855 / `archie-core-5gzx`). `tools/docsgen` now has a `check` subcommand +and `task docs:check`, wired into `task check`: it generates into a temporary +directory, compares against the committed artifact, names the authoritative Go +definition behind each mismatch, and never writes to the working tree. +`docsgen data` and `docsgen asyncapi` remain target-only. **2. The dashboard's client was not centralised while its own docstring claimed it was.** Now closed by `18e8f8f`. The docstring said "Single place that knows how From 2825b82a80038a9202571bcfda1e04db67bbbeb5 Mon Sep 17 00:00:00 2001 From: Sam Catlow Date: Fri, 18 Sep 2026 13:00:30 +1000 Subject: [PATCH 4/4] fix(docsgen): address three review findings on the check mode MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit All three Greptile findings were valid against the code; each is fixed at its cause rather than at the symptom, and each fix is covered by a test that fails when the fix is reverted. **Check could silently write.** `check` was matched only as the first argument, and leftover positional arguments were never inspected. So `docsgen --repo-root .. check` — and any misspelled subcommand — fell through to the write path and overwrote the artifact the check promises not to touch. That is the exact guarantee the mode exists to provide, so the failure was silent and in the wrong direction. Extracted argument handling into a testable `parseOptions` and made a leftover positional argument an error with usage text. Switched the flag set from `ExitOnError` to `ContinueOnError` so the error surfaces as a value rather than a process exit, which is what makes it testable. **Obsolete detection claimed files it does not own.** It treated every sibling `.json` as docsgen's and reported it obsolete, so checking an artifact in a mixed-use directory produced a false stale result, and the documented target layout — which anticipates further named artifacts in `docs/data/generated/` (catalog.json, AsyncAPI output) — would have been reported stale as soon as a second artifact legitimately existed. Ownership is now declared rather than inferred: `generatedArtifacts` maps each directory docsgen owns end-to-end to the names it produces there, and a directory docsgen does not own is never judged. A new unit test pins both halves — the orphan inside an owned directory is still caught, and a foreign `.json` in an arbitrary `--out` directory is not. **Provenance traversal stopped at one level.** Refs were walked only inside the top-level contract schemas, so a definition reachable solely through another definition resolved to "no authoritative definition found" — the diagnostic lying about the code it points at. Replaced the single pass with a breadth-first walk that follows discovered definitions transitively, seeded with every published contract first so a definition reachable from several contracts is attributed to the alphabetically first rather than to whichever the walk happened to reach. `walkSchemaRefs` now iterates map keys in sorted order for the same reason. `schemaProvenance` takes its roots as a parameter, which is what makes the transitive case testable without real types. Verified by mutation: with each fix reverted in turn, its test fails — `parseOptions(["--repo-root" ".." "check"]) returned nil error`, `obsoleteArtifacts() = [...someone-elses.json is obsolete...]`, and `provenance[Leaf] = ""`. Control run passes unmutated, and the file is byte-identical after restoration. Also confirmed end to end that the previously silent invocation now exits non-zero with usage instead of writing, and that the committed artifact is untouched by it. `task check`: exit 0, 0 failures. --- tools/docsgen/main.go | 187 ++++++++++++++++++++++++++++--------- tools/docsgen/main_test.go | 137 ++++++++++++++++++++++++--- 2 files changed, 266 insertions(+), 58 deletions(-) diff --git a/tools/docsgen/main.go b/tools/docsgen/main.go index 5973ea089..adf639820 100644 --- a/tools/docsgen/main.go +++ b/tools/docsgen/main.go @@ -7,6 +7,7 @@ import ( "errors" "flag" "fmt" + "io" "io/fs" "log" "os" @@ -32,17 +33,52 @@ const ( modeCheck ) +// generatedArtifacts declares the directories docsgen owns end-to-end, and the +// file names it produces in each. Obsolete detection is scoped to these +// directories: an arbitrary --out location is not ours to judge, and the target +// layout anticipates further named artifacts here (catalog.json, AsyncAPI +// output), which must not be reported as stale merely for existing. +var generatedArtifacts = map[string][]string{ + "docs/data/generated": {"contracts.json"}, +} + +// options is a parsed command line. +type options struct { + mode mode + repoRoot string + out string +} + func main() { - args := os.Args[1:] + parsed, err := parseOptions(os.Args[1:]) + if err != nil { + log.Fatalf("docsgen: %v", err) + } + + if parsed.mode == modeCheck { + err = check(parsed.repoRoot, parsed.out) + } else { + err = run(parsed.repoRoot, parsed.out) + } + if err != nil { + log.Fatalf("docsgen: %v", err) + } +} + +// parseOptions resolves the optional `check` subcommand and the flags that +// follow it. A leftover positional argument is an error rather than something +// to ignore: `docsgen --repo-root .. check` would otherwise quietly fall through +// to the write path and overwrite the artifact that check promises not to +// touch, and a misspelled subcommand would do the same silently. +func parseOptions(args []string) (options, error) { selected := modeWrite - // `check` is a subcommand, not a flag: it selects a comparison instead of a - // write, and the flag package would treat it as a positional argument. if len(args) > 0 && args[0] == "check" { selected = modeCheck args = args[1:] } - flags := flag.NewFlagSet("docsgen", flag.ExitOnError) + flags := flag.NewFlagSet("docsgen", flag.ContinueOnError) + flags.SetOutput(io.Discard) repoRoot := flags.String("repo-root", "..", "path to the Archie repository root") out := flags.String( "out", @@ -50,18 +86,17 @@ func main() { "output path relative to the repository root", ) if err := flags.Parse(args); err != nil { - log.Fatalf("docsgen: %v", err) + return options{}, err + } + if extra := flags.Args(); len(extra) > 0 { + return options{}, fmt.Errorf( + "unexpected argument %q; usage: docsgen [check] [--repo-root DIR] [--out PATH] "+ + "(`check` must come first)", + extra[0], + ) } - var err error - if selected == modeCheck { - err = check(*repoRoot, *out) - } else { - err = run(*repoRoot, *out) - } - if err != nil { - log.Fatalf("docsgen: %v", err) - } + return options{mode: selected, repoRoot: *repoRoot, out: *out}, nil } // check regenerates the artifact into a temporary directory and compares it @@ -109,7 +144,7 @@ func check(repoRoot, outPath string) error { } problems := compareArtifacts(relative, expected, committed) - problems = append(problems, obsoleteArtifacts(absRoot, committedPath)...) + problems = append(problems, obsoleteArtifacts(absRoot, filepath.Dir(committedPath))...) if len(problems) == 0 { fmt.Printf("docsgen: %s is up to date\n", relative) return nil @@ -135,7 +170,7 @@ func compareArtifacts(relative string, expected, committed []byte) []string { return []string{fmt.Sprintf("%s: committed output is unreadable: %v", relative, err)} } - provenance := schemaProvenance(expectedSchemas) + provenance := schemaProvenance(expectedSchemas, contractRoots()) var problems []string for _, name := range sortedSchemaNames(expectedSchemas) { committedSchema, ok := committedSchemas[name] @@ -168,60 +203,105 @@ func compareArtifacts(relative string, expected, committed []byte) []string { } // obsoleteArtifacts reports committed generated files in the artifact's -// directory that the generator no longer produces, so a renamed or deleted -// artifact cannot linger as dead documentation. -func obsoleteArtifacts(absRoot, committedPath string) []string { - entries, err := os.ReadDir(filepath.Dir(committedPath)) +// obsoleteArtifacts reports files present in a directory docsgen owns that the +// generator no longer produces, so a renamed or deleted artifact cannot linger +// as dead documentation. A directory docsgen does not own is never judged: its +// JSON belongs to someone else. +func obsoleteArtifacts(absRoot, dir string) []string { + owned, ours := ownedArtifacts(absRoot, dir) + if !ours { + return nil + } + entries, err := os.ReadDir(dir) if err != nil { return nil } - owned := filepath.Base(committedPath) var problems []string for _, entry := range entries { - if entry.IsDir() || entry.Name() == owned { + if entry.IsDir() || owned[entry.Name()] { continue } if filepath.Ext(entry.Name()) != ".json" { continue } - full := filepath.Join(filepath.Dir(committedPath), entry.Name()) problems = append(problems, fmt.Sprintf( "%s is obsolete: no generator in tools/docsgen produces it", - displayPath(absRoot, full), + displayPath(absRoot, filepath.Join(dir, entry.Name())), )) } return problems } +// ownedArtifacts reports which file names docsgen owns in dir, and whether dir +// is one docsgen owns at all. +func ownedArtifacts(absRoot, dir string) (map[string]bool, bool) { + relative, err := filepath.Rel(absRoot, dir) + if err != nil { + return nil, false + } + names, ours := generatedArtifacts[filepath.ToSlash(relative)] + if !ours { + return nil, false + } + owned := make(map[string]bool, len(names)) + for _, name := range names { + owned[name] = true + } + return owned, true +} + // schemaProvenance names the authoritative Go definition behind each published // schema, so a drift report points at the code that must be reviewed rather -// than only at the generated file. -func schemaProvenance(schemas map[string]any) map[string]string { - provenance := map[string]string{} - types := currentContractTypes() - names := make([]string, 0, len(types)) - for name := range types { - names = append(names, name) +// than only at the generated file. roots maps each published contract to its +// Go definition; every other schema is attributed to the contract that pulled +// it in, transitively, because a definition reachable only through another +// definition still belongs to the contract that caused it to be published. +func schemaProvenance(schemas map[string]any, roots map[string]string) map[string]string { + // attribution maps each schema to the published contract it came in under. + attribution := make(map[string]string, len(roots)) + queue := make([]string, 0, len(roots)) + for _, name := range sortedStringKeys(roots) { + attribution[name] = name + queue = append(queue, name) } - // Sorted: a nested definition referenced by several contracts would - // otherwise be attributed to whichever contract map iteration reached - // first, making the diagnostic nondeterministic. - sort.Strings(names) - for _, name := range names { - t := types[name] - provenance[name] = t.PkgPath() + "." + t.Name() - } - for _, name := range names { - walkSchemaRefs(schemas[name], func(target string) { - if _, known := provenance[target]; known { + + // Breadth-first, seeded with every contract before any nested definition is + // followed, so a definition reachable from several contracts is attributed + // to the alphabetically first of them rather than to whichever the walk + // happened to reach. + for len(queue) > 0 { + current := queue[0] + queue = queue[1:] + root := attribution[current] + walkSchemaRefs(schemas[current], func(target string) { + if _, known := attribution[target]; known { return } - provenance[target] = "reached from " + provenance[name] + attribution[target] = root + queue = append(queue, target) }) } + + provenance := make(map[string]string, len(attribution)) + for name, root := range attribution { + if source, isRoot := roots[name]; isRoot { + provenance[name] = source + continue + } + provenance[name] = "reached from " + roots[root] + } return provenance } +func sortedStringKeys(values map[string]string) []string { + keys := make([]string, 0, len(values)) + for key := range values { + keys = append(keys, key) + } + sort.Strings(keys) + return keys +} + func describeProvenance(provenance map[string]string, name string) string { if source, ok := provenance[name]; ok { return "authoritative definition: " + source @@ -229,10 +309,29 @@ func describeProvenance(provenance map[string]string, name string) string { return "no authoritative definition found" } +// contractRoots labels each published contract with the Go definition it comes +// from, which is what makes a drift report name code rather than a file. +func contractRoots() map[string]string { + types := currentContractTypes() + roots := make(map[string]string, len(types)) + for name, t := range types { + roots[name] = t.PkgPath() + "." + t.Name() + } + return roots +} + func walkSchemaRefs(value any, visit func(target string)) { switch node := value.(type) { case map[string]any: - for key, child := range node { + // Sorted so a schema reachable by more than one path is visited in the + // same order every run; provenance must not depend on map iteration. + keys := make([]string, 0, len(node)) + for key := range node { + keys = append(keys, key) + } + sort.Strings(keys) + for _, key := range keys { + child := node[key] if key == "$ref" { if ref, ok := child.(string); ok { if target, found := strings.CutPrefix(ref, "#/schemas/"); found { diff --git a/tools/docsgen/main_test.go b/tools/docsgen/main_test.go index 8fb3486bb..e23c6a13c 100644 --- a/tools/docsgen/main_test.go +++ b/tools/docsgen/main_test.go @@ -245,23 +245,132 @@ func TestCheckReportsMissingArtifact(t *testing.T) { } } -func TestCheckReportsObsoleteArtifact(t *testing.T) { - repoRoot := filepath.Clean("../..") - outputDir := t.TempDir() - if err := run(repoRoot, filepath.Join(outputDir, "contracts.json")); err != nil { - t.Fatalf("run() error = %v", err) - } - obsolete := filepath.Join(outputDir, "orphaned.json") - if err := os.WriteFile(obsolete, []byte("{}\n"), 0o600); err != nil { - t.Fatalf("write obsolete artifact: %v", err) +func TestParseOptions(t *testing.T) { + tests := []struct { + name string + args []string + wantMode mode + wantOut string + wantErr string + }{ + {name: "write by default", wantMode: modeWrite, wantOut: defaultOutputPath}, + {name: "check first", args: []string{"check"}, wantMode: modeCheck, wantOut: defaultOutputPath}, + { + name: "check with flags", + args: []string{"check", "--repo-root", ".."}, + wantMode: modeCheck, + wantOut: defaultOutputPath, + }, + { + name: "check honours out", + args: []string{"check", "--out", "/tmp/artifacts.json"}, + wantMode: modeCheck, + wantOut: "/tmp/artifacts.json", + }, + { + name: "check after flags would otherwise silently write", + args: []string{"--repo-root", "..", "check"}, + wantErr: "unexpected argument", + }, + {name: "misspelled subcommand", args: []string{"chekc"}, wantErr: "unexpected argument"}, + { + name: "stray trailing argument", + args: []string{"--out", "/tmp/artifacts.json", "stray"}, + wantErr: "unexpected argument", + }, } - err := check(repoRoot, filepath.Join(outputDir, "contracts.json")) - if err == nil { - t.Fatal("check() returned nil error for an obsolete artifact") + for _, test := range tests { + t.Run(test.name, func(t *testing.T) { + got, err := parseOptions(test.args) + if test.wantErr != "" { + if err == nil { + t.Fatalf("parseOptions(%q) returned nil error, want %q", test.args, test.wantErr) + } + if !strings.Contains(err.Error(), test.wantErr) { + t.Fatalf("parseOptions(%q) error = %q, want substring %q", test.args, err, test.wantErr) + } + return + } + if err != nil { + t.Fatalf("parseOptions(%q) error = %v", test.args, err) + } + if got.mode != test.wantMode { + t.Errorf("parseOptions(%q).mode = %v, want %v", test.args, got.mode, test.wantMode) + } + if got.out != test.wantOut { + t.Errorf("parseOptions(%q).out = %q, want %q", test.args, got.out, test.wantOut) + } + }) } - if !strings.Contains(err.Error(), "obsolete") || !strings.Contains(err.Error(), "orphaned.json") { - t.Fatalf("check() error = %q, want the obsolete file named", err) +} + +func TestSchemaProvenanceFollowsNestedDefinitions(t *testing.T) { + schemas := map[string]any{ + "Root": map[string]any{ + "properties": map[string]any{ + "middle": map[string]any{"$ref": "#/schemas/Middle"}, + }, + }, + "Middle": map[string]any{ + "properties": map[string]any{ + "leaf": map[string]any{"$ref": "#/schemas/Leaf"}, + }, + }, + "Leaf": map[string]any{"type": "string"}, + } + + provenance := schemaProvenance(schemas, map[string]string{"Root": "example.Root"}) + + if got := provenance["Root"]; got != "example.Root" { + t.Errorf("provenance[Root] = %q, want the root definition", got) + } + if got := provenance["Middle"]; got != "reached from example.Root" { + t.Errorf("provenance[Middle] = %q, want it reached from the root", got) + } + // Leaf is reachable only through Middle. A one-level walk misses it, and the + // report then claims no authoritative definition exists. + if got := provenance["Leaf"]; got != "reached from example.Root" { + t.Errorf("provenance[Leaf] = %q, want it reached from the root transitively", got) + } +} + +func TestObsoleteArtifactsReportsUnownedFileInAnOwnedDirectory(t *testing.T) { + root := t.TempDir() + dir := filepath.Join(root, "docs", "data", "generated") + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatalf("create generated dir: %v", err) + } + for name, body := range map[string]string{ + "contracts.json": "{}\n", // owned + "orphaned.json": "{}\n", // obsolete + "notes.txt": "not json\n", + } { + if err := os.WriteFile(filepath.Join(dir, name), []byte(body), 0o600); err != nil { + t.Fatalf("write %s: %v", name, err) + } + } + + problems := obsoleteArtifacts(root, dir) + if len(problems) != 1 { + t.Fatalf("obsoleteArtifacts() = %q, want exactly the orphan", problems) + } + if !strings.Contains(problems[0], "orphaned.json") || !strings.Contains(problems[0], "obsolete") { + t.Fatalf("obsoleteArtifacts() = %q, want the orphan named", problems) + } +} + +func TestObsoleteArtifactsNeverJudgesAnUnownedDirectory(t *testing.T) { + root := t.TempDir() + // An arbitrary --out location: the JSON beside the artifact belongs to + // whoever put it there, and docsgen must not claim it as its own. + dir := t.TempDir() + if err := os.WriteFile(filepath.Join(dir, "someone-elses.json"), []byte("{}\n"), 0o600); err != nil { + t.Fatalf("write foreign json: %v", err) + } + + if problems := obsoleteArtifacts(root, dir); len(problems) != 0 { + t.Fatalf("obsoleteArtifacts() = %q, want none for an unowned directory", problems) } }