Chat turns have token budgets and per-person rate limits #2074
Notifications
Due Date
No due date set.
Blocks
Reference: sam/archie#2074
Reference in new issue
Block a user
Why: a recursive or hostile prompt can make one chat turn loop model calls and tool calls without end — "keep searching until you find it" — burning model cost and hammering downstream connectors (denial of wallet). Even permitted actions can exceed what the task, the person, or the org should spend. Today a turn is bounded only by its step cap and timeout: nothing bounds its tokens, and nothing rate-limits turns or tool calls per person or per org.
Today:
Source: the revised RFC exported at transcripts/2026-11-20-chat-authorization-rfc-3fa43269.md in sam/archie-product; citations below are RFC section numbers.
Required behaviour:
Out of scope: #2067 tool discovery, #2068 taint, #2069 memory scopes, #2070 connector credential modes, #2071 admin tools refused in chat, #2072 approval binding, #2073 image/CSP rendering, #2075 audit/revocation/kill-switch, #2078 enforcement on every entry point, #2079 chat-triggered workflow execution contracts, #2080 disclosure authorization per audience/destination. The existing per-run/per-org-day action budget in judgement's Cost guard is not changed here. Chat has no subagents yet, so subagent budget inheritance is not in scope.
Design notes: modules/chat owns this — turn.go for the token budget and loop detection (the stream is consumed in drain, so the consecutive-call counter lives beside it), api.go sendTurn for the per-person turn limit, tools.go call for tool-call limits. Count turns and tool calls through modules/chat/internal/store (the store stays the source of truth; no state lives only in memory). Deny by default: an over-limit request is refused, never queued. Pitfall: maxConcurrentTurns = 4 is global, not per person — a per-person limit must not be bypassable by opening many conversations. A resumed turn still obeys the budgets, and child calls charge the parent turn's budget.
Tests (resource-exhaustion boundary — table cases, fakes, no DB, per AGENTS.md; extend the fakes in modules/chat/access_test.go: fakeStore, fakeJudge, fakeToolbox):
Done when: task check passes, and on a running binary: a turn repeating the same tool call with identical arguments is cut off at the 3rd consecutive call with the refusal visible in the transcript; a turn that spends its token budget stops with the budget message; sending turns past the per-person limit answers 429 with the reason; once the org limit is reached, turns from everyone are refused; cancelling a turn stops it visibly.
Open questions for the maintainer:
Part of #2066.