From 98c40322857f9eb96fac93de1684587e9cdddb49 Mon Sep 17 00:00:00 2001 From: Sam Catlow Date: Fri, 18 Sep 2026 13:42:33 +1000 Subject: [PATCH] docs(prds): design the code graph as an MCP surface Supersedes symbol-reference-index.md, which scoped only the data layer and left the interface unstated. Deleted rather than archived: it was written today, never implemented, and keeping both would leave two documents claiming authority over one capability. Three things the narrower doc did not settle. The interface is MCP. archied consumes MCP and serves none, so exposing the graph that way reaches Claude Code, pi, codex and Archie's own agents through one surface with no coupling to the daemon. Diagrams are deliberately not a tool: a model handed an accurate subgraph writes its own Mermaid, and the valuable half was never the rendering. Every answer declares what it could not see. RTA over-approximates interface dispatch and reflection is invisible, so the *extract yaegi tables make symbols look reachable regardless. An agent trusting a confident wrong graph is worse off than one that greps. A code graph does not power a documentation chatbot. It answers structural questions about Go symbols; answering "how do I configure a second identity" means retrieving prose from markdown. I had previously implied one substrate served both. It does not, and building it that way would serve neither well. --- docs/prds/code-graph-mcp.md | 110 ++++++++++++++++++++++++++ docs/prds/symbol-reference-index.md | 117 ---------------------------- 2 files changed, 110 insertions(+), 117 deletions(-) create mode 100644 docs/prds/code-graph-mcp.md delete mode 100644 docs/prds/symbol-reference-index.md diff --git a/docs/prds/code-graph-mcp.md b/docs/prds/code-graph-mcp.md new file mode 100644 index 000000000..9627ce15f --- /dev/null +++ b/docs/prds/code-graph-mcp.md @@ -0,0 +1,110 @@ +# Code graph over MCP (`tools/codegraph`) + +Status: proposed. Supersedes `symbol-reference-index.md`, which scoped only the +data layer. + +## The problem + +Establishing what a change touches currently costs an agent eight to fifteen +greps, and the answers are wrong in specific ways. `grep -rn "Register("` +cannot separate `plugin.Registry.Register` from `captureintake.Receiver.Register` +from `forgerpc.Server.Register`, all of which exist here. It also matches the +sibling checkouts under `.worktrees/`, which are near-complete copies of the +module, so every count is silently inflated. + +That cost lands hardest in investigation, triage and the TDD loop, where the +first question is always "what does this touch". + +## Decision: exact analysis, not semantic retrieval + +Tools like Greptile parse with tree-sitter, summarise each node, embed the +summaries, and retrieve semantically. That design exists because they must +serve any language. This repository is one Go module, and Go's type checker is +ground truth: + +| Question | Source of truth | +| --- | --- | +| Who references this symbol? | `go/packages` + `go/types`, resolved by identity | +| What calls what? | `callgraph` RTA over SSA | +| What imports what? | `go list -deps` | + +No vector database, no embedding cost, no edges invented by a summary that +drifted from the code. For a single-language repository this is both cheaper +and strictly more accurate. Copy the interface those tools expose, not their +architecture. + +**A code graph does not power a documentation chatbot.** It answers structural +questions about Go symbols. Answering "how do I configure a second identity" +means retrieving prose from markdown, which needs chunking, embedding and +similarity search over a different corpus entirely. That remains a separate +capability with its own design. Conflating the two would build one system that +does neither well. + +## Interface: an MCP server + +`archied` already consumes MCP as a client (`internal/app/agentworker` +starts providers) and serves none. Exposing the graph over MCP reaches Claude +Code, pi, codex and Archie's own agents through one surface, with no coupling +to the daemon: it is configured like any other entry in `[[mcp_servers]]`. + +Four tools, each returning JSON with file:line citations: + +- `references(symbol)` — every reference site, classified `prod` / `test` / + `generated` / `dynamic` +- `callers(symbol, depth)` — inbound call edges +- `callees(symbol, depth)` — outbound call edges +- `blast_radius(file | symbol)` — the transitive neighbourhood a change reaches, + which is the single call meant to replace the grep sequence + +Diagrams are deliberately not a tool. A model given an accurate subgraph writes +its own Mermaid; a diagram endpoint would only fix the rendering of something +the model already has. + +## Decisions + +**Every answer declares what it could not see.** RTA over-approximates +interface dispatch, and reflection is invisible: every `reflect.ValueOf` entry +in `wfextract`, `gateextract`, `pluginextract`, `secretextract` and +`logextract` makes a symbol look reachable whether or not anything calls it. +Each response carries a `dynamic_edges` field naming those boundaries. An agent +trusting a confident wrong graph is worse off than one that greps, so the graph +states its own limits rather than implying completeness. + +**Warm in memory, keyed on HEAD, never persisted to disk.** `deadcode` does +whole-program RTA in under seven seconds; this does less work. The server +rebuilds when HEAD changes and refuses its cache outright when the tree is +dirty. Nothing is written to disk, because a persisted index becomes a rendered +view that outlives its evidence. That is the failure that made `STATUS.md` +report 90 of 91 findings re-verified while most statuses were 77 commits stale, +and the same one that made `reachaudit` serve a previous tree's analysis from +`/tmp` until its cache was keyed on HEAD. + +**`tools/codegraph`, not `internal/`.** It is developer tooling, not daemon +runtime, so it belongs beside `docsgen`, `contractaudit` and `reachaudit` in +the `tools` module, which `task test:tools` already covers. Start as a single +Go file; split only when it earns it. + +**No new dependency.** `golang.org/x/tools` is already in `go.sum` at v0.49.0. + +## How it is verified + +Red first. Table-driven tests against small fixture modules built in +`t.TempDir()`, asserting the behaviour that distinguishes this from grep: + +1. Two methods spelled `Register` on different types resolve to two separate + records, and a reference to one does not increment the other. +2. A symbol referenced only from `_test.go` reports zero `prod` references. +3. A symbol referenced only from a `reflect.ValueOf` table reports zero `prod` + references and one `dynamic`, and names the table in `dynamic_edges`. +4. A generated file's references land in `generated`, not `prod`. + +Mutation-verify each: reintroduce the conflated behaviour and watch the +assertion fail. + +Acceptance is a live cross-check rather than a unit test. Run it against this +tree and confirm it reports zero production references for +`worktree.Manager.Resume`, `webui.NewDangerousService` and +`secret.DefaultRegistry`. All three were confirmed dead by hand, and `deadcode` +either stayed silent on them or, in the case of a package var, cannot represent +them at all. Agreeing with `deadcode` is not the bar. Catching what `deadcode` +structurally cannot is. diff --git a/docs/prds/symbol-reference-index.md b/docs/prds/symbol-reference-index.md deleted file mode 100644 index 81fbadf71..000000000 --- a/docs/prds/symbol-reference-index.md +++ /dev/null @@ -1,117 +0,0 @@ -# Type-resolved symbol reference index (`tools/symbolrefs`) - -Status: proposed. Single-file Go tool in the `tools` module. - -## The gap - -Three questions get asked constantly during audits, and each currently gets -answered by a different unreliable method: - -| Question | Today | Problem | -| --- | --- | --- | -| Is this reachable from a shipped binary? | `task deadcode` | Sound but incomplete, functions only | -| Who references this symbol? | `grep` / `ast-grep` | Matches spelling, not identity | -| Is this package in any binary? | `tools/reachaudit` | Package granularity | - -The middle row is the hole. `grep -rn "Register("` cannot tell -`plugin.Registry.Register` from `captureintake.Receiver.Register` from -`forgerpc.Server.Register`, all of which exist here. It also matches the three -sibling checkouts under `.worktrees/`, which are near-complete copies of the -module, so every count is inflated by a factor nobody notices. - -This matters because reachability alone does not settle a finding. RTA -over-approximates: it assumes every address-taken function may be called, and -every `reflect.ValueOf` entry in `wfextract`, `gateextract`, `pluginextract`, -`secretextract` and `logextract` is address-taken. `deadcode` is therefore -silent on `worktree.Manager.Resume` despite it having zero callers anywhere in -the tree. `deadcode` also reports functions only, so a dead package var like -`secret.DefaultRegistry` is structurally invisible to it. - -A reference index answers what reachability cannot: **a symbol with zero -non-test reference sites is dead regardless of what RTA believes about its -address.** The two reports are complementary, and neither subsumes the other. - -## What it does - -`packages.Load` over the root module with `NeedTypes | NeedTypesInfo | -NeedSyntax | NeedDeps`, then for every identifier in every file resolve -`TypesInfo.Uses[ident]` to a `types.Object` and attribute the reference to its -declaring object. Identity comes from the type checker, so spelling collisions -and sibling worktrees both disappear: `go list` resolves by import path and -never walks `.worktrees/`. - -Each reference site is classified by the file that holds it: - -- `prod` — an ordinary non-test file in the module -- `test` — `_test.go` -- `generated` — matches the standard generated-file header, plus - `internal/contracts/**` and `ui/dist/` -- `dynamic` — a `reflect.ValueOf` table or other runtime-loading site, which - means static analysis cannot see the real caller - -`dynamic` is a separate class rather than folded into `prod` because the whole -point is to stop those sites laundering a dead symbol into a live-looking one. - -Output is JSONL on stdout, one record per package-level declaration: - -```json -{"symbol":"internal/worktree.Manager.Resume","kind":"method", - "decl":{"file":"internal/worktree/worktree.go","line":261}, - "refs":{"prod":0,"test":4,"generated":0,"dynamic":0}, - "sites":[{"file":"internal/worktree/worktree_test.go","line":88,"class":"test"}]} -``` - -So the query that took me six hand-run greps this session becomes one line: - -```sh -task symbolrefs | jq -c 'select(.refs.prod == 0 and .refs.dynamic == 0)' -``` - -## Scope - -In: package-level funcs, methods, types, vars, consts in the root module. -Out: local variables, struct fields, reachability (that is `task deadcode`), -and any write path. The tool is read-only and advisory. It never proposes -deletion, matching `reachaudit`'s existing stance. - -## Decisions - -**Single file, `tools/symbolrefs/symbolrefs.go`.** It is dev tooling, not -product code, so it belongs beside `docsgen` and `contractaudit` in the `tools` -module rather than under `internal/`. That module is already covered by -`task test:tools`, which `task check` runs. - -**No persisted index.** `deadcode` does whole-program RTA in under 7 seconds; -this does strictly less work (no SSA, no call graph). Anything cached becomes a -rendered view that outlives its evidence, which is the exact failure that made -`STATUS.md` claim 90 of 91 findings re-verified while most statuses were 77 -commits stale, and the same reason `reachaudit`'s `/tmp` cache had to be keyed -on HEAD. Recompute; do not reconcile. - -**Not in `task check`.** It reports a standing backlog, so gating on it fails -every run until the backlog is zero. Same rationale as `task deadcode` and -`task vuln`. - -**No new dependency.** `golang.org/x/tools` is already in `go.sum` at v0.49.0. - -## How it is verified - -Red first, per the repo protocol. The tests are table-driven against small -fixture modules built in `t.TempDir()`, asserting the classification that -distinguishes this tool from grep: - -1. Two distinct methods spelled `Register` on different types resolve to two - separate records, and a reference to one does not increment the other. -2. A symbol referenced only from `_test.go` reports `prod: 0`. -3. A symbol referenced only from a `reflect.ValueOf` table reports `prod: 0` - and `dynamic: 1`, not `prod: 1`. -4. A generated file's references land in `generated`, not `prod`. - -Mutation-verify each: reintroduce the conflated behaviour and watch the -assertion fail, per `HANDOFF.md` rule 5. - -Acceptance is a live cross-check, not a unit test: run it against the tree and -confirm it reports `prod: 0` for `worktree.Manager.Resume`, -`webui.NewDangerousService` and `secret.DefaultRegistry`, all three of which -were confirmed dead by hand this session and all three of which `deadcode` -either missed or could not represent.