From 8d0716900dabc003cd346e0b58e41cc54e854460 Mon Sep 17 00:00:00 2001 From: Sam Catlow Date: Fri, 18 Sep 2026 12:40:35 +1000 Subject: [PATCH] chore(tools): make unreachable-code analysis runnable from the Taskfile Add `task deadcode` (whole-program RTA from ./cmd/...) and `task reachaudit` (the existing classifier on top of it). Neither joins `task check`: both report a standing backlog, so gating on them fails every run until that backlog is zero, same as `task vuln`. `task deadcode` shells to `go run ...@latest` rather than a local binary on purpose. A deadcode built against an older Go than this module exits non-zero and prints nothing, which reads as "no dead code" instead of "analysis failed" -- the locally installed copy was built against 1.26 against a 1.27 module and had been silently reporting an empty tree. reachaudit had two traps that made it answer confidently from nothing: its --repo default pointed at a path that does not exist on either machine, and it reused /tmp/reachaudit-deadcode.txt with no staleness check, so it would serve a previous tree's analysis indefinitely. Key the cache on HEAD and refuse it outright when the tree is dirty. Also lands the design for tools/symbolrefs, the type-resolved reference index that covers what reachability cannot: RTA treats every address-taken function as live, so every reflect.ValueOf entry in the *extract tables launders a dead symbol into a reachable-looking one, and deadcode reports functions only, leaving dead package vars unrepresentable. --- Taskfile.yml | 37 +++++++++ docs/prds/symbol-reference-index.md | 117 ++++++++++++++++++++++++++++ tools/reachaudit/reachaudit.py | 8 +- 3 files changed, 159 insertions(+), 3 deletions(-) create mode 100644 docs/prds/symbol-reference-index.md diff --git a/Taskfile.yml b/Taskfile.yml index 4683f8f95..4f103d5c8 100644 --- a/Taskfile.yml +++ b/Taskfile.yml @@ -93,6 +93,43 @@ tasks: cmds: - govulncheck ./... + deadcode: + desc: "Report functions no shipped binary can reach (whole-program RTA from ./cmd/...). Not part of task check: it reports a standing backlog, so gating on it would fail every run until that backlog is zero." + summary: | + Answers "is this reachable in production", which no linter does: + golangci-lint's unused check is package-scoped, so anything exported + and called from a sibling package looks live even when no binary + reaches it. + + Run via `go run ...@latest` deliberately. A locally installed deadcode + built against an older Go than this module exits non-zero and prints + NOTHING, which reads as "no dead code" instead of "analysis failed". + Building it against the repo toolchain makes that failure impossible. + + -test=false is the point of the report: a function reached only by its + own tests is not reachable in production. + + Read the output in one direction only. RTA over-approximates what is + reachable, so "unreachable func" is sound and worth acting on, while + silence proves nothing: interface dispatch and any address-taken + function (every reflect.ValueOf entry in the *extract tables) is + assumed reachable whether or not it is ever called. + cmds: + - go run golang.org/x/tools/cmd/deadcode@latest -test=false ./cmd/... + + reachaudit: + desc: "Classify unreachable code as tracked / untracked / dynamic (tools/reachaudit). Wraps task deadcode with provenance and tracker status." + summary: | + deadcode says a function is unreachable. reachaudit says whether that + is a known gap, a forgotten one, or an artifact of runtime loading + (yaegi, reflection) that must never be deleted. + + --refresh re-runs the analysis. Without it the tool reuses + /tmp/reachaudit-deadcode.txt with no staleness check, which will + happily report last week's tree against today's code. + cmds: + - python3 tools/reachaudit/reachaudit.py --repo . {{.CLI_ARGS}} + ui:deps: desc: Install the dashboard's node dependencies (ui/node_modules is gitignored) internal: true diff --git a/docs/prds/symbol-reference-index.md b/docs/prds/symbol-reference-index.md new file mode 100644 index 000000000..81fbadf71 --- /dev/null +++ b/docs/prds/symbol-reference-index.md @@ -0,0 +1,117 @@ +# Type-resolved symbol reference index (`tools/symbolrefs`) + +Status: proposed. Single-file Go tool in the `tools` module. + +## The gap + +Three questions get asked constantly during audits, and each currently gets +answered by a different unreliable method: + +| Question | Today | Problem | +| --- | --- | --- | +| Is this reachable from a shipped binary? | `task deadcode` | Sound but incomplete, functions only | +| Who references this symbol? | `grep` / `ast-grep` | Matches spelling, not identity | +| Is this package in any binary? | `tools/reachaudit` | Package granularity | + +The middle row is the hole. `grep -rn "Register("` cannot tell +`plugin.Registry.Register` from `captureintake.Receiver.Register` from +`forgerpc.Server.Register`, all of which exist here. It also matches the three +sibling checkouts under `.worktrees/`, which are near-complete copies of the +module, so every count is inflated by a factor nobody notices. + +This matters because reachability alone does not settle a finding. RTA +over-approximates: it assumes every address-taken function may be called, and +every `reflect.ValueOf` entry in `wfextract`, `gateextract`, `pluginextract`, +`secretextract` and `logextract` is address-taken. `deadcode` is therefore +silent on `worktree.Manager.Resume` despite it having zero callers anywhere in +the tree. `deadcode` also reports functions only, so a dead package var like +`secret.DefaultRegistry` is structurally invisible to it. + +A reference index answers what reachability cannot: **a symbol with zero +non-test reference sites is dead regardless of what RTA believes about its +address.** The two reports are complementary, and neither subsumes the other. + +## What it does + +`packages.Load` over the root module with `NeedTypes | NeedTypesInfo | +NeedSyntax | NeedDeps`, then for every identifier in every file resolve +`TypesInfo.Uses[ident]` to a `types.Object` and attribute the reference to its +declaring object. Identity comes from the type checker, so spelling collisions +and sibling worktrees both disappear: `go list` resolves by import path and +never walks `.worktrees/`. + +Each reference site is classified by the file that holds it: + +- `prod` — an ordinary non-test file in the module +- `test` — `_test.go` +- `generated` — matches the standard generated-file header, plus + `internal/contracts/**` and `ui/dist/` +- `dynamic` — a `reflect.ValueOf` table or other runtime-loading site, which + means static analysis cannot see the real caller + +`dynamic` is a separate class rather than folded into `prod` because the whole +point is to stop those sites laundering a dead symbol into a live-looking one. + +Output is JSONL on stdout, one record per package-level declaration: + +```json +{"symbol":"internal/worktree.Manager.Resume","kind":"method", + "decl":{"file":"internal/worktree/worktree.go","line":261}, + "refs":{"prod":0,"test":4,"generated":0,"dynamic":0}, + "sites":[{"file":"internal/worktree/worktree_test.go","line":88,"class":"test"}]} +``` + +So the query that took me six hand-run greps this session becomes one line: + +```sh +task symbolrefs | jq -c 'select(.refs.prod == 0 and .refs.dynamic == 0)' +``` + +## Scope + +In: package-level funcs, methods, types, vars, consts in the root module. +Out: local variables, struct fields, reachability (that is `task deadcode`), +and any write path. The tool is read-only and advisory. It never proposes +deletion, matching `reachaudit`'s existing stance. + +## Decisions + +**Single file, `tools/symbolrefs/symbolrefs.go`.** It is dev tooling, not +product code, so it belongs beside `docsgen` and `contractaudit` in the `tools` +module rather than under `internal/`. That module is already covered by +`task test:tools`, which `task check` runs. + +**No persisted index.** `deadcode` does whole-program RTA in under 7 seconds; +this does strictly less work (no SSA, no call graph). Anything cached becomes a +rendered view that outlives its evidence, which is the exact failure that made +`STATUS.md` claim 90 of 91 findings re-verified while most statuses were 77 +commits stale, and the same reason `reachaudit`'s `/tmp` cache had to be keyed +on HEAD. Recompute; do not reconcile. + +**Not in `task check`.** It reports a standing backlog, so gating on it fails +every run until the backlog is zero. Same rationale as `task deadcode` and +`task vuln`. + +**No new dependency.** `golang.org/x/tools` is already in `go.sum` at v0.49.0. + +## How it is verified + +Red first, per the repo protocol. The tests are table-driven against small +fixture modules built in `t.TempDir()`, asserting the classification that +distinguishes this tool from grep: + +1. Two distinct methods spelled `Register` on different types resolve to two + separate records, and a reference to one does not increment the other. +2. A symbol referenced only from `_test.go` reports `prod: 0`. +3. A symbol referenced only from a `reflect.ValueOf` table reports `prod: 0` + and `dynamic: 1`, not `prod: 1`. +4. A generated file's references land in `generated`, not `prod`. + +Mutation-verify each: reintroduce the conflated behaviour and watch the +assertion fail, per `HANDOFF.md` rule 5. + +Acceptance is a live cross-check, not a unit test: run it against the tree and +confirm it reports `prod: 0` for `worktree.Manager.Resume`, +`webui.NewDangerousService` and `secret.DefaultRegistry`, all three of which +were confirmed dead by hand this session and all three of which `deadcode` +either missed or could not represent. diff --git a/tools/reachaudit/reachaudit.py b/tools/reachaudit/reachaudit.py index b356d4d09..222df0c08 100755 --- a/tools/reachaudit/reachaudit.py +++ b/tools/reachaudit/reachaudit.py @@ -67,8 +67,10 @@ def cmd_closure(repo: Path) -> set[str]: def deadcode_findings(repo: Path) -> list[str]: """Run deadcode, or reuse a cached run. Falls back to an empty list with a loud warning rather than silently reporting zero findings.""" - cache = Path("/tmp/reachaudit-deadcode.txt") - if cache.exists(): + head = run(["git", "rev-parse", "HEAD"], repo) or "unknown" + dirty = "-dirty" if run(["git", "status", "--porcelain"], repo) else "" + cache = Path(f"/tmp/reachaudit-deadcode-{head[:12]}{dirty}.txt") + if cache.exists() and not dirty: return cache.read_text().splitlines() print("running deadcode (this compiles the module, may take a minute)...", file=sys.stderr) out = subprocess.run( @@ -138,7 +140,7 @@ def consumers(repo: Path, pkg: str) -> tuple[int, int]: def main() -> int: ap = argparse.ArgumentParser() - ap.add_argument("--repo", default=str(Path.home() / "projects/archie-core")) + ap.add_argument("--repo", default=str(Path(__file__).resolve().parents[2])) ap.add_argument("--json") ap.add_argument("--md") ap.add_argument("--refresh", action="store_true", help="re-run deadcode instead of using the cache")