fix(tools): stop contractaudit crediting dead paths as live consumers
Renaming the dashboard's request helper from req( to request( broke the consumer extractor: it matched the call by name, so 16 live routes were reported unconsumed. The extractor now keys on the call argument instead, which survives a rename. The first attempt at that fix was worse than the bug, and is recorded here because the direction of the error is the whole point. Crediting any quoted /api/... literal meant a comment, a retired-route array, or a display label counted as proof a route was reachable -- so deleting every real request still reported the surface as healthy. A false "unconsumed" is a loud, fixable failure; a false "consumed" is a silent hole in the gate, which is worse than having no gate at all. A path now counts as a consumer only when it is the first argument of a call in code. That excludes bare constants, dead arrays, object properties, and commented-out calls, while still accepting any function name. Both halves are pinned by tests: TestScanFileCreditsAnyCallSignature and TestScanFileIgnoresNonCallLiterals, plus a surface-level TestNegativeCommentedOutCallIsStillReported that the vacuous version fails. Re-injecting either half of the bug fails the suite. Requiring a call lost no real consumer: internal/webui still reads 49/49. The remaining limits (path must be the first argument; template interpolations unseen; routes credited per path rather than per verb) are declared in docs/contract-declarations.json and the PRD rather than left implicit. Also corrects a measurement I had quoted inconsistently: proto/state/v1 declares 45 RPCs, not 43 or 46.
This commit is contained in:
4 files changed
+260
-81
No files matched your search
@@ -24,7 +24,7 @@
|
||||
"blindSpots": [
|
||||
{
|
||||
"surface": "internal/webui",
|
||||
"reason": "Consumer extraction is textual over ui/src. A path assembled by string concatenation rather than written as a literal is invisible; scanFile fails loudly when a call shape it cannot parse appears, but it cannot see a path that is never a literal. Routes whose consumer is outside the dashboard (the asset mount, the health probes, the capture-intake webhook) are excluded from this surface by design: they have different consumers and are not part of the dashboard API contract."
|
||||
"reason": "A path counts as a consumer only when it is the first argument of a call in code. The function need not be named -- naming it broke on rename, when matching `req|fetch` reported 16 live routes as unconsumed after the dashboard's helper became `request(` -- but a bare constant, a dead array, an object property, or a commented-out call must not count, because crediting any quoted literal makes the surface pass on a file that issues no requests at all. Limits: a path passed as any argument other than the first is not seen (accepting a comma would start crediting array elements); a request inside a template-literal interpolation is not seen; a path built from a bare /api fragment is reported as unparsable; and a route is credited to every HTTP verb, so a route whose specific verb has no caller reads as consumed. Routes whose consumer is outside the dashboard (the asset mount, the health probes, the capture-intake webhook) are excluded from this surface by design: they have different consumers and are not part of the dashboard API contract."
|
||||
},
|
||||
{
|
||||
"surface": "proto/state/v1/state.proto",
|
||||
|
||||
@@ -120,11 +120,30 @@ presented as dead":
|
||||
|
||||
- **Runtime-loaded code** (Yaegi, reflection) — a route or RPC reached only
|
||||
through dynamic dispatch reads as unconsumed. Labelled with the package list.
|
||||
- **JSX call-site extraction is textual.** It matches path literals in
|
||||
`ui/src/**`, so a path built by string concatenation is invisible. The
|
||||
extractor must **fail loudly on an unexpected call shape** rather than
|
||||
under-report — under-reporting is the failure mode that makes a gate worse than
|
||||
no gate.
|
||||
- **Dashboard call-site extraction is textual, and deliberately narrow.** A path
|
||||
counts as a consumer only when it is the *first argument of a call* in code:
|
||||
the function need not be named (so renaming `req(` to `request(` is fine), but
|
||||
a bare constant, a dead array, an object property, or a commented-out call
|
||||
must not count. Both halves are load-bearing, and they pull in opposite
|
||||
directions.
|
||||
|
||||
Naming the call broke on rename: matching `req|fetch` reported 16 live routes
|
||||
as unconsumed the moment the dashboard's helper was renamed. Dropping the
|
||||
call requirement was **worse**, and is recorded here because it is the more
|
||||
tempting mistake: crediting any quoted `/api/...` literal meant a comment, a
|
||||
retired-route array, or a display label proved a route was reachable, so
|
||||
deleting every real request still read as healthy. The two directions are not
|
||||
symmetric — a false "unconsumed" is a loud, fixable failure, while a false
|
||||
"consumed" is a silent hole, which is worse than having no gate.
|
||||
|
||||
Remaining limits: a path passed as any argument other than the first is not
|
||||
seen (accepting a comma would start crediting array elements and restore the
|
||||
vacuity above); a request written inside a template-literal interpolation is
|
||||
not seen; and a path built from a bare `/api` fragment is reported as
|
||||
unparsable rather than silently contributing nothing.
|
||||
- **Routes are credited per path, not per verb.** A literal does not say which
|
||||
HTTP verb carries it, so a route is credited to every verb and a route whose
|
||||
specific verb has no caller reads as consumed.
|
||||
- **An `EventSource` subscription is a consumer.** `/events` and
|
||||
`/api/logs/stream` are consumed via `EventSource`, not `fetch`; an extractor
|
||||
that only looks for `fetch` reports them unconsumed and is simply wrong.
|
||||
|
||||
@@ -195,20 +195,142 @@ func TestNegativeStaleDeclarationIsReported(t *testing.T) {
|
||||
}
|
||||
|
||||
// TestNegativeUnparsableCallShapeFailsLoudly proves the extractor refuses to
|
||||
// under-report. A call shape it cannot parse must be an error, never a
|
||||
// silently smaller consumer set -- under-reporting is how a gate stops
|
||||
// checking without anyone noticing.
|
||||
// under-report. A path built from a bare /api fragment must be an error, never
|
||||
// a silently smaller consumer set.
|
||||
func TestNegativeUnparsableCallShapeFailsLoudly(t *testing.T) {
|
||||
body := `
|
||||
export const api = {
|
||||
exotic: (p) => fetch(buildPath("/api/things"), { method: "GET" }),
|
||||
};
|
||||
const base = "/api";
|
||||
const url = base + "/things";
|
||||
`
|
||||
_, err := scanFile("ui/src/base/exotic.jsx", body)
|
||||
if err == nil {
|
||||
t.Fatal("unparsable call shape was accepted; the extractor would under-report")
|
||||
t.Fatal("a path built from a bare /api fragment was accepted; the extractor would under-report")
|
||||
}
|
||||
if !strings.Contains(err.Error(), "would under-report") {
|
||||
t.Errorf("error = %v, want the under-report explanation", err)
|
||||
}
|
||||
}
|
||||
|
||||
// TestNegativeCommentedOutCallIsStillReported is the anti-vacuity test at the
|
||||
// surface level. Commenting out a real call leaves the path text in the file,
|
||||
// so an extractor that credits any quoted path reports the route as healthy
|
||||
// while nothing calls it -- a silent hole rather than a loud failure. The route
|
||||
// must become an undeclared finding.
|
||||
func TestNegativeCommentedOutCallIsStillReported(t *testing.T) {
|
||||
root := fixture(t)
|
||||
|
||||
apiPath := filepath.Join(root, "ui/src/base/api.jsx")
|
||||
body, err := os.ReadFile(apiPath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
live := `summary: () => request("/api/summary"),`
|
||||
if !strings.Contains(string(body), live) {
|
||||
t.Fatal("fixture does not contain the expected live call; the client changed shape")
|
||||
}
|
||||
commented := strings.Replace(string(body), live, `// retired: () => request("/api/summary"),`, 1)
|
||||
if err := os.WriteFile(apiPath, []byte(commented), 0o644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
finding, ok := find(auditOf(t, root), "GET /api/summary")
|
||||
if !ok {
|
||||
t.Fatal("a route whose only remaining mention is a comment was not reported")
|
||||
}
|
||||
if finding.Class != Undeclared {
|
||||
t.Errorf("class = %q, want %q: a comment cannot prove a route is reachable", finding.Class, Undeclared)
|
||||
}
|
||||
}
|
||||
|
||||
// TestScanFileCreditsAnyCallSignature pins the lesson from a real regression:
|
||||
// the first version of this extractor matched the call by name (`req|fetch`),
|
||||
// and rewriting the dashboard's helper from `req(` to `request(` silently
|
||||
// reported 16 live routes as unconsumed. Extraction must key on the path
|
||||
// literal as a call argument, not on the name of the function carrying it.
|
||||
func TestScanFileCreditsAnyCallSignature(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
body string
|
||||
want []string
|
||||
}{
|
||||
{
|
||||
name: "the original helper name",
|
||||
body: `const a = req("/api/summary");`,
|
||||
want: []string{"/api/summary"},
|
||||
},
|
||||
{
|
||||
name: "a renamed helper",
|
||||
body: `const a = request("/api/summary");`,
|
||||
want: []string{"/api/summary"},
|
||||
},
|
||||
{
|
||||
name: "a name the extractor has never heard of",
|
||||
body: `const a = someOtherTransport("/api/summary");`,
|
||||
want: []string{"/api/summary"},
|
||||
},
|
||||
{
|
||||
name: "an interpolated path in backticks",
|
||||
body: "const a = request(`/api/tasks/${id}`);",
|
||||
want: []string{"/api/tasks/${id}"},
|
||||
},
|
||||
{
|
||||
name: "an EventSource subscription",
|
||||
body: `const s = new EventSource("/events");`,
|
||||
want: []string{"/events"},
|
||||
},
|
||||
{
|
||||
name: "a path with a query string",
|
||||
body: `const a = request("/api/logs" + qs(params));`,
|
||||
want: []string{"/api/logs"},
|
||||
},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got, err := scanFile("ui/src/base/client.jsx", tc.body)
|
||||
if err != nil {
|
||||
t.Fatalf("scanFile: %v", err)
|
||||
}
|
||||
if len(got) != len(tc.want) {
|
||||
t.Fatalf("scanFile(%q) = %v, want %v", tc.body, got, tc.want)
|
||||
}
|
||||
for i := range tc.want {
|
||||
if got[i] != tc.want[i] {
|
||||
t.Errorf("scanFile(%q)[%d] = %q, want %q", tc.body, i, got[i], tc.want[i])
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// TestScanFileIgnoresNonCallLiterals is the anti-vacuity half, and the more
|
||||
// important one. Crediting any quoted path -- rather than a path passed to a
|
||||
// call -- makes the whole surface pass on a file that issues no requests at
|
||||
// all: a comment, a retired-route array, or a display label would each count as
|
||||
// proof of consumption. That failure is silent, which makes it worse than a
|
||||
// false finding.
|
||||
func TestScanFileIgnoresNonCallLiterals(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
body string
|
||||
}{
|
||||
{"a constant that is never passed to anything", `const p = "/api/summary";`},
|
||||
{"a comment quoting a path", `// the route "/api/summary" is dead`},
|
||||
{"an array of retired paths", `const RETIRED = ["/api/summary", "/api/tasks"];`},
|
||||
{"an object property", `const cfg = { path: "/api/summary" };`},
|
||||
{"a template constant", "const p = `/api/summary`;"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
got, err := scanFile("ui/src/base/client.jsx", tc.body)
|
||||
if err != nil {
|
||||
t.Fatalf("scanFile: %v", err)
|
||||
}
|
||||
if len(got) != 0 {
|
||||
t.Errorf(
|
||||
"scanFile(%q) = %v, want no consumers: a literal that is not a call argument cannot prove a route is reachable",
|
||||
tc.body, got,
|
||||
)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
+106
-68
@@ -36,39 +36,43 @@ var routePattern = regexp.MustCompile(`(?:HandleFunc|Handle)\(\s*"([A-Z]+)\s+(/[
|
||||
// read as an unconsumed route when it is in fact a mounted one.
|
||||
var routeLiteralPattern = regexp.MustCompile(`(?m)^\s*(?:const\s+)?([A-Za-z0-9_]+)\s*=\s*"([A-Z]+)\s+(/[^"]*)"`)
|
||||
|
||||
// Consumer call shapes. Go's regexp has no backreferences, so each quoting
|
||||
// style gets its own alternative rather than a backreference pair.
|
||||
// Consumer call shapes.
|
||||
//
|
||||
// EventSource counts as a consumer: /events and /api/logs/stream are
|
||||
// subscribed to, not fetched, and an extractor that only looks for fetch
|
||||
// reports them unconsumed and is simply wrong.
|
||||
// Extraction is deliberately name-agnostic: it matches the path *literal*
|
||||
// rather than the call that carries it. An earlier version matched `req|fetch`
|
||||
// by function name, and rewriting the client's helper from `req(` to
|
||||
// `request(` silently reported 16 live routes as unconsumed. The contract this
|
||||
// surface audits is "the dashboard calls this path", and a literal is how the
|
||||
// dashboard expresses that regardless of what the surrounding call is called.
|
||||
var (
|
||||
fetchPathPatterns = []*regexp.Regexp{
|
||||
regexp.MustCompile(`(?:req|fetch)\(\s*"([^"]+)"`),
|
||||
regexp.MustCompile("(?:req|fetch)\\(\\s*`([^`]+)`"),
|
||||
}
|
||||
eventPathPatterns = []*regexp.Regexp{
|
||||
regexp.MustCompile(`new\s+EventSource\(\s*"([^"]+)"`),
|
||||
regexp.MustCompile("new\\s+EventSource\\(\\s*`([^`]+)`"),
|
||||
}
|
||||
// A bare path literal, so a call routed through a helper this extractor
|
||||
// does not know about is still credited.
|
||||
pathLiteralPattern = regexp.MustCompile(`"(/[a-zA-Z0-9_./{}:-]*)"`)
|
||||
// pathLiteralPattern matches a path literal that is the first argument of a
|
||||
// call, in either quoting style. Both are needed: a fixed path is written
|
||||
// with double quotes and an interpolated one with backticks.
|
||||
//
|
||||
// Two properties are load-bearing, and they pull in opposite directions.
|
||||
//
|
||||
// It does not name the function, because naming it broke on rename: matching
|
||||
// `req|fetch` reported 16 live routes as unconsumed the moment the
|
||||
// dashboard's helper became `request(`.
|
||||
//
|
||||
// It requires a call, because dropping *that* requirement made the check
|
||||
// vacuous: crediting any quoted `/api/...` literal meant a comment, a dead
|
||||
// array, or a display label counted as proof that the route was consumed, so
|
||||
// deleting every real request still read as healthy. The two directions are
|
||||
// not symmetric -- a false "unconsumed" is a loud, fixable failure, while a
|
||||
// false "consumed" is a silent hole in the gate, which is worse than having
|
||||
// no gate at all.
|
||||
//
|
||||
// Known limit: the path must be the *first* argument. Accepting a comma
|
||||
// before the literal would start crediting array elements, restoring the
|
||||
// vacuity this pattern exists to prevent.
|
||||
pathLiteralPattern = regexp.MustCompile("\\b[A-Za-z_$][A-Za-z0-9_$]*\\s*\\(\\s*[\"`](/(?:api/|events)[^\"`\\s]*)")
|
||||
|
||||
// Every fetch/req/EventSource call's opening, captured with its first
|
||||
// argument, so the extractor can require that argument to be a literal
|
||||
// rather than hoping a shape it does not recognise never appears.
|
||||
callOpenPattern = regexp.MustCompile(`\b(?:req|fetch|new\s+EventSource)\s*\(\s*([^\s)\n]{0,80})`)
|
||||
// A declaration, not a call: `async function req(path) {`.
|
||||
declPattern = regexp.MustCompile(`\bfunction\s+(?:req|fetch)\s*\(`)
|
||||
// The wrapper delegating on a variable: `fetch(path, {`. This carries no
|
||||
// path by construction, so it is not an under-report.
|
||||
delegatePattern = regexp.MustCompile(`\b(?:req|fetch)\s*\(\s*[a-z][A-Za-z0-9_]*\s*[,)]`)
|
||||
// The first argument opening with a quote, which is what makes it a
|
||||
// literal path this extractor must have extracted.
|
||||
literalArgOnly = regexp.MustCompile("^[\"`]")
|
||||
// An EventSource subscription built from a variable is skipped for the
|
||||
// same reason as the wrapper delegation above.
|
||||
// fragmentPattern matches a literal that begins an API path but does not
|
||||
// hold the whole route, e.g. `"/api" + "/tasks"`. Such a route cannot be
|
||||
// credited, so the file is reported rather than silently contributing
|
||||
// nothing -- under-reporting is how a gate stops checking unnoticed.
|
||||
fragmentPattern = regexp.MustCompile("[\"`](/api)[\"`]")
|
||||
|
||||
tmpPattern = regexp.MustCompile(`\$\{[^}]*\}`)
|
||||
routeParamPat = regexp.MustCompile(`\{[^}]+\}`)
|
||||
@@ -177,52 +181,86 @@ func (s *webUISurface) consumerPaths() (map[string]string, error) {
|
||||
return consumed, nil
|
||||
}
|
||||
|
||||
// scanFile returns every API path literal in one dashboard source file.
|
||||
// scanFile returns every request path in one dashboard source file.
|
||||
//
|
||||
// It fails loudly on a call it cannot parse. Under-reporting is how a gate
|
||||
// silently stops checking, so a `fetch`/`req`/`EventSource` whose first
|
||||
// argument is neither a literal nor one of the two shapes that legitimately
|
||||
// carries no path -- a declaration, or the wrapper delegating on a variable --
|
||||
// is an error naming the file rather than a quietly smaller consumer set.
|
||||
// Two things must be true of a mention before it counts as a consumer: it must
|
||||
// be the first argument of a call, and it must be code rather than prose. The
|
||||
// first excludes bare constants, dead arrays, and object properties; the second
|
||||
// excludes a commented-out call, which otherwise leaves a call-shaped path in
|
||||
// the file and reads as live.
|
||||
//
|
||||
// The only failure it reports is a path it cannot see whole. That is a narrow
|
||||
// guard rather than a heuristic: it fires when a file concatenates a bare /api
|
||||
// fragment into a route, because no literal then holds the path and the
|
||||
// extractor would otherwise contribute nothing for that route without saying so.
|
||||
func scanFile(rel, body string) ([]string, error) {
|
||||
var literals []string
|
||||
for _, pattern := range append(append([]*regexp.Regexp{}, fetchPathPatterns...), eventPathPatterns...) {
|
||||
for _, match := range pattern.FindAllStringSubmatch(body, -1) {
|
||||
literals = append(literals, match[1])
|
||||
}
|
||||
}
|
||||
for _, match := range pathLiteralPattern.FindAllStringSubmatch(body, -1) {
|
||||
literals = append(literals, match[1])
|
||||
}
|
||||
|
||||
// Every call whose first argument is not a literal, minus the declarations
|
||||
// and variable delegations that legitimately produce no path.
|
||||
//
|
||||
// The guard is deliberately one-directional. There is no matching "found no
|
||||
// calls" condition: a file may legitimately carry a path literal with no
|
||||
// call of its own (a router table, a link), and the aggregate check in
|
||||
// consumerPaths already fails when the whole surface yields nothing.
|
||||
declarations := len(declPattern.FindAllString(body, -1))
|
||||
delegations := len(delegatePattern.FindAllString(body, -1))
|
||||
for _, match := range callOpenPattern.FindAllStringSubmatch(body, -1) {
|
||||
first := match[1]
|
||||
if first == "" || literalArgOnly.MatchString(first) {
|
||||
continue
|
||||
}
|
||||
// A non-literal first argument. Accounted for by a declaration or a
|
||||
// delegation, or an under-report.
|
||||
if declarations+delegations > 0 {
|
||||
declarations--
|
||||
continue
|
||||
}
|
||||
code := stripJSComments(body)
|
||||
if fragmentPattern.MatchString(code) {
|
||||
return nil, fmt.Errorf(
|
||||
"%s: fetch/req/EventSource call with non-literal first argument %q; the extractor cannot see the path it builds and would under-report",
|
||||
rel, first,
|
||||
"%s: builds an API path from a bare /api fragment; the extractor cannot see the whole route and would under-report",
|
||||
rel,
|
||||
)
|
||||
}
|
||||
var literals []string
|
||||
for _, match := range pathLiteralPattern.FindAllStringSubmatch(code, -1) {
|
||||
literals = append(literals, match[1])
|
||||
}
|
||||
return literals, nil
|
||||
}
|
||||
|
||||
// stripJSComments removes // and /* */ comments while leaving string, template,
|
||||
// and regex literals intact, so the extractor measures code rather than prose
|
||||
// about code. Without it, commenting out a call still reads as a live consumer
|
||||
// -- the same silent hole as crediting a bare literal.
|
||||
//
|
||||
// Known limit: template-literal interpolations are treated as string content, so
|
||||
// a request written inside `${...}` is not seen. No such call exists in ui/src.
|
||||
func stripJSComments(src string) string {
|
||||
var b strings.Builder
|
||||
b.Grow(len(src))
|
||||
var quote byte
|
||||
for i := 0; i < len(src); {
|
||||
c := src[i]
|
||||
var next byte
|
||||
if i+1 < len(src) {
|
||||
next = src[i+1]
|
||||
}
|
||||
if quote != 0 {
|
||||
b.WriteByte(c)
|
||||
if c == '\\' && i+1 < len(src) {
|
||||
b.WriteByte(next)
|
||||
i += 2
|
||||
continue
|
||||
}
|
||||
if c == quote {
|
||||
quote = 0
|
||||
}
|
||||
i++
|
||||
continue
|
||||
}
|
||||
switch {
|
||||
case c == '"' || c == '\'' || c == '`':
|
||||
quote = c
|
||||
b.WriteByte(c)
|
||||
i++
|
||||
case c == '/' && next == '/':
|
||||
for i < len(src) && src[i] != '\n' {
|
||||
i++
|
||||
}
|
||||
case c == '/' && next == '*':
|
||||
i += 2
|
||||
for i < len(src) && !(src[i] == '*' && i+1 < len(src) && src[i+1] == '/') {
|
||||
i++
|
||||
}
|
||||
i += 2
|
||||
default:
|
||||
b.WriteByte(c)
|
||||
i++
|
||||
}
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
|
||||
// normalizeRoute strips query strings and replaces both `${...}` template
|
||||
// interpolations and `{param}` wildcards with a single placeholder, so a
|
||||
// parameter's *name* never causes a false mismatch. `{id}` in the server and
|
||||
|
||||
Reference in new issue
Block a user