TestInvestigateDocumentSuspendsAndResumes fails intermittently with a request digest mismatch #2083

Closed
opened 2026-11-21 03:51:46 +00:00 by sam · 1 comment
Owner

TestInvestigateDocumentSuspendsAndResumes in modules/runs fails about 5 runs in 30 on unchanged main, with an agentloop request digest mismatch when the resumed run replays. The run journal is supposed to replay deterministically, so either the test builds a request with unstable ordering (a map, a timestamp), or replay itself is nondeterministic, which would be a real bug in resume.

Reproduce: go test -short -count=30 -run TestInvestigateDocumentSuspendsAndResumes ./modules/runs/

Done when: the cause is found and fixed where the unstable value is produced, and 200 consecutive runs pass.

`TestInvestigateDocumentSuspendsAndResumes` in `modules/runs` fails about 5 runs in 30 on unchanged main, with an agentloop request digest mismatch when the resumed run replays. The run journal is supposed to replay deterministically, so either the test builds a request with unstable ordering (a map, a timestamp), or replay itself is nondeterministic, which would be a real bug in resume. Reproduce: `go test -short -count=30 -run TestInvestigateDocumentSuspendsAndResumes ./modules/runs/` Done when: the cause is found and fixed where the unstable value is produced, and 200 consecutive runs pass.
sam added the mediumbug labels 2026-11-21 03:51:46 +00:00
Author
Owner

Fixed in 1989948ab. This was a real resume bug, not a test problem. encoding/json/v2 doesn't sort map keys unless asked with json.Deterministic(true). The record-cause prompt was built from a map (modules/runs/memory.go), so its key order changed from run to run. agentloop digests the whole request (agentloop/journal.go digestRequest), so the replayed request sometimes didn't match the journal and the resumed run parked with "journal diverged".

Fix: encode with json.Deterministic(true) where those bytes are produced: the record-cause and follow-through prompts, connector action input (which feeds the judge request and its idempotency key), and reference values rendered into prompts.

Verified: 200 consecutive passes in the worktree, and 100 more on main after the merge. task check passed. Optional upstream hardening: ai-sdk's digestRequest/encodeRequest could encode deterministically themselves, so the digest wouldn't depend on callers producing canonical bytes.

Fixed in 1989948ab. This was a real resume bug, not a test problem. `encoding/json/v2` doesn't sort map keys unless asked with `json.Deterministic(true)`. The record-cause prompt was built from a map (`modules/runs/memory.go`), so its key order changed from run to run. agentloop digests the whole request (`agentloop/journal.go` `digestRequest`), so the replayed request sometimes didn't match the journal and the resumed run parked with "journal diverged". Fix: encode with `json.Deterministic(true)` where those bytes are produced: the record-cause and follow-through prompts, connector action input (which feeds the judge request and its idempotency key), and reference values rendered into prompts. Verified: 200 consecutive passes in the worktree, and 100 more on main after the merge. `task check` passed. Optional upstream hardening: ai-sdk's `digestRequest`/`encodeRequest` could encode deterministically themselves, so the digest wouldn't depend on callers producing canonical bytes.
sam closed this issue 2026-11-21 04:02:18 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sam/archie#2083