fix(tools): stop contractaudit crediting dead paths as live consumers

Renaming the dashboard's request helper from req( to request( broke the
consumer extractor: it matched the call by name, so 16 live routes were
reported unconsumed. The extractor now keys on the call argument instead,
which survives a rename.

The first attempt at that fix was worse than the bug, and is recorded here
because the direction of the error is the whole point. Crediting any quoted
/api/... literal meant a comment, a retired-route array, or a display label
counted as proof a route was reachable -- so deleting every real request
still reported the surface as healthy. A false "unconsumed" is a loud,
fixable failure; a false "consumed" is a silent hole in the gate, which is
worse than having no gate at all.

A path now counts as a consumer only when it is the first argument of a call
in code. That excludes bare constants, dead arrays, object properties, and
commented-out calls, while still accepting any function name. Both halves are
pinned by tests: TestScanFileCreditsAnyCallSignature and
TestScanFileIgnoresNonCallLiterals, plus a surface-level
TestNegativeCommentedOutCallIsStillReported that the vacuous version fails.
Re-injecting either half of the bug fails the suite.

Requiring a call lost no real consumer: internal/webui still reads 49/49.
The remaining limits (path must be the first argument; template
interpolations unseen; routes credited per path rather than per verb) are
declared in docs/contract-declarations.json and the PRD rather than left
implicit.

Also corrects a measurement I had quoted inconsistently: proto/state/v1
declares 45 RPCs, not 43 or 46.
This commit is contained in:
sam committed 2026-09-15 13:49:47 +10:00
1 parent 7b3bfe5b2f
commit 22cb0c6de5
4 files changed
+260 -81

No files matched your search

+1 -1
View File
@@ -24,7 +24,7 @@
"blindSpots": [
{
"surface": "internal/webui",
"reason": "Consumer extraction is textual over ui/src. A path assembled by string concatenation rather than written as a literal is invisible; scanFile fails loudly when a call shape it cannot parse appears, but it cannot see a path that is never a literal. Routes whose consumer is outside the dashboard (the asset mount, the health probes, the capture-intake webhook) are excluded from this surface by design: they have different consumers and are not part of the dashboard API contract."
"reason": "A path counts as a consumer only when it is the first argument of a call in code. The function need not be named -- naming it broke on rename, when matching `req|fetch` reported 16 live routes as unconsumed after the dashboard's helper became `request(` -- but a bare constant, a dead array, an object property, or a commented-out call must not count, because crediting any quoted literal makes the surface pass on a file that issues no requests at all. Limits: a path passed as any argument other than the first is not seen (accepting a comma would start crediting array elements); a request inside a template-literal interpolation is not seen; a path built from a bare /api fragment is reported as unparsable; and a route is credited to every HTTP verb, so a route whose specific verb has no caller reads as consumed. Routes whose consumer is outside the dashboard (the asset mount, the health probes, the capture-intake webhook) are excluded from this surface by design: they have different consumers and are not part of the dashboard API contract."
},
{
"surface": "proto/state/v1/state.proto",
+24 -5
View File
@@ -120,11 +120,30 @@ presented as dead":
- **Runtime-loaded code** (Yaegi, reflection) — a route or RPC reached only
through dynamic dispatch reads as unconsumed. Labelled with the package list.
- **JSX call-site extraction is textual.** It matches path literals in
`ui/src/**`, so a path built by string concatenation is invisible. The
extractor must **fail loudly on an unexpected call shape** rather than
under-report — under-reporting is the failure mode that makes a gate worse than
no gate.
- **Dashboard call-site extraction is textual, and deliberately narrow.** A path
counts as a consumer only when it is the *first argument of a call* in code:
the function need not be named (so renaming `req(` to `request(` is fine), but
a bare constant, a dead array, an object property, or a commented-out call
must not count. Both halves are load-bearing, and they pull in opposite
directions.
Naming the call broke on rename: matching `req|fetch` reported 16 live routes
as unconsumed the moment the dashboard's helper was renamed. Dropping the
call requirement was **worse**, and is recorded here because it is the more
tempting mistake: crediting any quoted `/api/...` literal meant a comment, a
retired-route array, or a display label proved a route was reachable, so
deleting every real request still read as healthy. The two directions are not
symmetric — a false "unconsumed" is a loud, fixable failure, while a false
"consumed" is a silent hole, which is worse than having no gate.
Remaining limits: a path passed as any argument other than the first is not
seen (accepting a comma would start crediting array elements and restore the
vacuity above); a request written inside a template-literal interpolation is
not seen; and a path built from a bare `/api` fragment is reported as
unparsable rather than silently contributing nothing.
- **Routes are credited per path, not per verb.** A literal does not say which
HTTP verb carries it, so a route is credited to every verb and a route whose
specific verb has no caller reads as consumed.
- **An `EventSource` subscription is a consumer.** `/events` and
`/api/logs/stream` are consumed via `EventSource`, not `fetch`; an extractor
that only looks for `fetch` reports them unconsumed and is simply wrong.
+129 -7
View File
@@ -195,20 +195,142 @@ func TestNegativeStaleDeclarationIsReported(t *testing.T) {
}
// TestNegativeUnparsableCallShapeFailsLoudly proves the extractor refuses to
// under-report. A call shape it cannot parse must be an error, never a
// silently smaller consumer set -- under-reporting is how a gate stops
// checking without anyone noticing.
// under-report. A path built from a bare /api fragment must be an error, never
// a silently smaller consumer set.
func TestNegativeUnparsableCallShapeFailsLoudly(t *testing.T) {
body := `
export const api = {
exotic: (p) => fetch(buildPath("/api/things"), { method: "GET" }),
};
const base = "/api";
const url = base + "/things";
`
_, err := scanFile("ui/src/base/exotic.jsx", body)
if err == nil {
t.Fatal("unparsable call shape was accepted; the extractor would under-report")
t.Fatal("a path built from a bare /api fragment was accepted; the extractor would under-report")
}
if !strings.Contains(err.Error(), "would under-report") {
t.Errorf("error = %v, want the under-report explanation", err)
}
}
// TestNegativeCommentedOutCallIsStillReported is the anti-vacuity test at the
// surface level. Commenting out a real call leaves the path text in the file,
// so an extractor that credits any quoted path reports the route as healthy
// while nothing calls it -- a silent hole rather than a loud failure. The route
// must become an undeclared finding.
func TestNegativeCommentedOutCallIsStillReported(t *testing.T) {
root := fixture(t)
apiPath := filepath.Join(root, "ui/src/base/api.jsx")
body, err := os.ReadFile(apiPath)
if err != nil {
t.Fatal(err)
}
live := `summary: () => request("/api/summary"),`
if !strings.Contains(string(body), live) {
t.Fatal("fixture does not contain the expected live call; the client changed shape")
}
commented := strings.Replace(string(body), live, `// retired: () => request("/api/summary"),`, 1)
if err := os.WriteFile(apiPath, []byte(commented), 0o644); err != nil {
t.Fatal(err)
}
finding, ok := find(auditOf(t, root), "GET /api/summary")
if !ok {
t.Fatal("a route whose only remaining mention is a comment was not reported")
}
if finding.Class != Undeclared {
t.Errorf("class = %q, want %q: a comment cannot prove a route is reachable", finding.Class, Undeclared)
}
}
// TestScanFileCreditsAnyCallSignature pins the lesson from a real regression:
// the first version of this extractor matched the call by name (`req|fetch`),
// and rewriting the dashboard's helper from `req(` to `request(` silently
// reported 16 live routes as unconsumed. Extraction must key on the path
// literal as a call argument, not on the name of the function carrying it.
func TestScanFileCreditsAnyCallSignature(t *testing.T) {
cases := []struct {
name string
body string
want []string
}{
{
name: "the original helper name",
body: `const a = req("/api/summary");`,
want: []string{"/api/summary"},
},
{
name: "a renamed helper",
body: `const a = request("/api/summary");`,
want: []string{"/api/summary"},
},
{
name: "a name the extractor has never heard of",
body: `const a = someOtherTransport("/api/summary");`,
want: []string{"/api/summary"},
},
{
name: "an interpolated path in backticks",
body: "const a = request(`/api/tasks/${id}`);",
want: []string{"/api/tasks/${id}"},
},
{
name: "an EventSource subscription",
body: `const s = new EventSource("/events");`,
want: []string{"/events"},
},
{
name: "a path with a query string",
body: `const a = request("/api/logs" + qs(params));`,
want: []string{"/api/logs"},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got, err := scanFile("ui/src/base/client.jsx", tc.body)
if err != nil {
t.Fatalf("scanFile: %v", err)
}
if len(got) != len(tc.want) {
t.Fatalf("scanFile(%q) = %v, want %v", tc.body, got, tc.want)
}
for i := range tc.want {
if got[i] != tc.want[i] {
t.Errorf("scanFile(%q)[%d] = %q, want %q", tc.body, i, got[i], tc.want[i])
}
}
})
}
}
// TestScanFileIgnoresNonCallLiterals is the anti-vacuity half, and the more
// important one. Crediting any quoted path -- rather than a path passed to a
// call -- makes the whole surface pass on a file that issues no requests at
// all: a comment, a retired-route array, or a display label would each count as
// proof of consumption. That failure is silent, which makes it worse than a
// false finding.
func TestScanFileIgnoresNonCallLiterals(t *testing.T) {
cases := []struct {
name string
body string
}{
{"a constant that is never passed to anything", `const p = "/api/summary";`},
{"a comment quoting a path", `// the route "/api/summary" is dead`},
{"an array of retired paths", `const RETIRED = ["/api/summary", "/api/tasks"];`},
{"an object property", `const cfg = { path: "/api/summary" };`},
{"a template constant", "const p = `/api/summary`;"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
got, err := scanFile("ui/src/base/client.jsx", tc.body)
if err != nil {
t.Fatalf("scanFile: %v", err)
}
if len(got) != 0 {
t.Errorf(
"scanFile(%q) = %v, want no consumers: a literal that is not a call argument cannot prove a route is reachable",
tc.body, got,
)
}
})
}
}
+106 -68
View File
@@ -36,39 +36,43 @@ var routePattern = regexp.MustCompile(`(?:HandleFunc|Handle)\(\s*"([A-Z]+)\s+(/[
// read as an unconsumed route when it is in fact a mounted one.
var routeLiteralPattern = regexp.MustCompile(`(?m)^\s*(?:const\s+)?([A-Za-z0-9_]+)\s*=\s*"([A-Z]+)\s+(/[^"]*)"`)
// Consumer call shapes. Go's regexp has no backreferences, so each quoting
// style gets its own alternative rather than a backreference pair.
// Consumer call shapes.
//
// EventSource counts as a consumer: /events and /api/logs/stream are
// subscribed to, not fetched, and an extractor that only looks for fetch
// reports them unconsumed and is simply wrong.
// Extraction is deliberately name-agnostic: it matches the path *literal*
// rather than the call that carries it. An earlier version matched `req|fetch`
// by function name, and rewriting the client's helper from `req(` to
// `request(` silently reported 16 live routes as unconsumed. The contract this
// surface audits is "the dashboard calls this path", and a literal is how the
// dashboard expresses that regardless of what the surrounding call is called.
var (
fetchPathPatterns = []*regexp.Regexp{
regexp.MustCompile(`(?:req|fetch)\(\s*"([^"]+)"`),
regexp.MustCompile("(?:req|fetch)\\(\\s*`([^`]+)`"),
}
eventPathPatterns = []*regexp.Regexp{
regexp.MustCompile(`new\s+EventSource\(\s*"([^"]+)"`),
regexp.MustCompile("new\\s+EventSource\\(\\s*`([^`]+)`"),
}
// A bare path literal, so a call routed through a helper this extractor
// does not know about is still credited.
pathLiteralPattern = regexp.MustCompile(`"(/[a-zA-Z0-9_./{}:-]*)"`)
// pathLiteralPattern matches a path literal that is the first argument of a
// call, in either quoting style. Both are needed: a fixed path is written
// with double quotes and an interpolated one with backticks.
//
// Two properties are load-bearing, and they pull in opposite directions.
//
// It does not name the function, because naming it broke on rename: matching
// `req|fetch` reported 16 live routes as unconsumed the moment the
// dashboard's helper became `request(`.
//
// It requires a call, because dropping *that* requirement made the check
// vacuous: crediting any quoted `/api/...` literal meant a comment, a dead
// array, or a display label counted as proof that the route was consumed, so
// deleting every real request still read as healthy. The two directions are
// not symmetric -- a false "unconsumed" is a loud, fixable failure, while a
// false "consumed" is a silent hole in the gate, which is worse than having
// no gate at all.
//
// Known limit: the path must be the *first* argument. Accepting a comma
// before the literal would start crediting array elements, restoring the
// vacuity this pattern exists to prevent.
pathLiteralPattern = regexp.MustCompile("\\b[A-Za-z_$][A-Za-z0-9_$]*\\s*\\(\\s*[\"`](/(?:api/|events)[^\"`\\s]*)")
// Every fetch/req/EventSource call's opening, captured with its first
// argument, so the extractor can require that argument to be a literal
// rather than hoping a shape it does not recognise never appears.
callOpenPattern = regexp.MustCompile(`\b(?:req|fetch|new\s+EventSource)\s*\(\s*([^\s)\n]{0,80})`)
// A declaration, not a call: `async function req(path) {`.
declPattern = regexp.MustCompile(`\bfunction\s+(?:req|fetch)\s*\(`)
// The wrapper delegating on a variable: `fetch(path, {`. This carries no
// path by construction, so it is not an under-report.
delegatePattern = regexp.MustCompile(`\b(?:req|fetch)\s*\(\s*[a-z][A-Za-z0-9_]*\s*[,)]`)
// The first argument opening with a quote, which is what makes it a
// literal path this extractor must have extracted.
literalArgOnly = regexp.MustCompile("^[\"`]")
// An EventSource subscription built from a variable is skipped for the
// same reason as the wrapper delegation above.
// fragmentPattern matches a literal that begins an API path but does not
// hold the whole route, e.g. `"/api" + "/tasks"`. Such a route cannot be
// credited, so the file is reported rather than silently contributing
// nothing -- under-reporting is how a gate stops checking unnoticed.
fragmentPattern = regexp.MustCompile("[\"`](/api)[\"`]")
tmpPattern = regexp.MustCompile(`\$\{[^}]*\}`)
routeParamPat = regexp.MustCompile(`\{[^}]+\}`)
@@ -177,52 +181,86 @@ func (s *webUISurface) consumerPaths() (map[string]string, error) {
return consumed, nil
}
// scanFile returns every API path literal in one dashboard source file.
// scanFile returns every request path in one dashboard source file.
//
// It fails loudly on a call it cannot parse. Under-reporting is how a gate
// silently stops checking, so a `fetch`/`req`/`EventSource` whose first
// argument is neither a literal nor one of the two shapes that legitimately
// carries no path -- a declaration, or the wrapper delegating on a variable --
// is an error naming the file rather than a quietly smaller consumer set.
// Two things must be true of a mention before it counts as a consumer: it must
// be the first argument of a call, and it must be code rather than prose. The
// first excludes bare constants, dead arrays, and object properties; the second
// excludes a commented-out call, which otherwise leaves a call-shaped path in
// the file and reads as live.
//
// The only failure it reports is a path it cannot see whole. That is a narrow
// guard rather than a heuristic: it fires when a file concatenates a bare /api
// fragment into a route, because no literal then holds the path and the
// extractor would otherwise contribute nothing for that route without saying so.
func scanFile(rel, body string) ([]string, error) {
var literals []string
for _, pattern := range append(append([]*regexp.Regexp{}, fetchPathPatterns...), eventPathPatterns...) {
for _, match := range pattern.FindAllStringSubmatch(body, -1) {
literals = append(literals, match[1])
}
}
for _, match := range pathLiteralPattern.FindAllStringSubmatch(body, -1) {
literals = append(literals, match[1])
}
// Every call whose first argument is not a literal, minus the declarations
// and variable delegations that legitimately produce no path.
//
// The guard is deliberately one-directional. There is no matching "found no
// calls" condition: a file may legitimately carry a path literal with no
// call of its own (a router table, a link), and the aggregate check in
// consumerPaths already fails when the whole surface yields nothing.
declarations := len(declPattern.FindAllString(body, -1))
delegations := len(delegatePattern.FindAllString(body, -1))
for _, match := range callOpenPattern.FindAllStringSubmatch(body, -1) {
first := match[1]
if first == "" || literalArgOnly.MatchString(first) {
continue
}
// A non-literal first argument. Accounted for by a declaration or a
// delegation, or an under-report.
if declarations+delegations > 0 {
declarations--
continue
}
code := stripJSComments(body)
if fragmentPattern.MatchString(code) {
return nil, fmt.Errorf(
"%s: fetch/req/EventSource call with non-literal first argument %q; the extractor cannot see the path it builds and would under-report",
rel, first,
"%s: builds an API path from a bare /api fragment; the extractor cannot see the whole route and would under-report",
rel,
)
}
var literals []string
for _, match := range pathLiteralPattern.FindAllStringSubmatch(code, -1) {
literals = append(literals, match[1])
}
return literals, nil
}
// stripJSComments removes // and /* */ comments while leaving string, template,
// and regex literals intact, so the extractor measures code rather than prose
// about code. Without it, commenting out a call still reads as a live consumer
// -- the same silent hole as crediting a bare literal.
//
// Known limit: template-literal interpolations are treated as string content, so
// a request written inside `${...}` is not seen. No such call exists in ui/src.
func stripJSComments(src string) string {
var b strings.Builder
b.Grow(len(src))
var quote byte
for i := 0; i < len(src); {
c := src[i]
var next byte
if i+1 < len(src) {
next = src[i+1]
}
if quote != 0 {
b.WriteByte(c)
if c == '\\' && i+1 < len(src) {
b.WriteByte(next)
i += 2
continue
}
if c == quote {
quote = 0
}
i++
continue
}
switch {
case c == '"' || c == '\'' || c == '`':
quote = c
b.WriteByte(c)
i++
case c == '/' && next == '/':
for i < len(src) && src[i] != '\n' {
i++
}
case c == '/' && next == '*':
i += 2
for i < len(src) && !(src[i] == '*' && i+1 < len(src) && src[i+1] == '/') {
i++
}
i += 2
default:
b.WriteByte(c)
i++
}
}
return b.String()
}
// normalizeRoute strips query strings and replaces both `${...}` template
// interpolations and `{param}` wildcards with a single placeholder, so a
// parameter's *name* never causes a false mismatch. `{id}` in the server and