Chat turns have token budgets and per-person rate limits #2074

Open
opened 2026-11-20 08:41:10 +00:00 by sam · 0 comments
Owner

Why: a recursive or hostile prompt can make one chat turn loop model calls and tool calls without end — "keep searching until you find it" — burning model cost and hammering downstream connectors (denial of wallet). Even permitted actions can exceed what the task, the person, or the org should spend. Today a turn is bounded only by its step cap and timeout: nothing bounds its tokens, and nothing rate-limits turns or tool calls per person or per org.

Today:

  • modules/chat/turn.go holds the turn's limits as constants: maxSteps = 16, turnTimeout = 10 minutes, maxConcurrentTurns = 4, historyTurns = 20. generate calls core.StreamText with MaxSteps: maxSteps only; core.GenerateOptions also has a MaxTokens field that is not set.
  • modules/chat/api.go sendTurn accepts any message from any signed-in person — no per-person or per-org limit on turns.
  • Tool calls go through modules/chat/tools.go call; there is no per-person or per-org counter on tool calls and no loop detection: the same tool with the same arguments can repeat until the step cap. connectors.Digest(action, input) is the canonical-arguments digest judgement already uses.
  • The only existing budgets are judgement's per-run and per-org-day changing-action budgets (the Cost guard in modules/judgement/cost.go, configured as ActionsPerRun/ActionsPerDay in governance settings under PUT /api/settings) — those count judged actions, not turns or tool calls, and are a separate mechanism.
  • No rate limiter is importable by the new modules: internal/ratelimit belongs to an old service, and depguard lets modules/** import only internal/platform/**.

Source: the revised RFC exported at transcripts/2026-11-20-chat-authorization-rfc-3fa43269.md in sam/archie-product; citations below are RFC section numbers.

Required behaviour:

  1. Bound every turn's spend: pass MaxTokens in the turn's core.GenerateOptions as a named constant beside maxSteps, and when the budget is spent the turn stops with a visible budget message in the transcript (RFC 8 availability and cost; RFC 5.1 cumulative limits).
  2. Bound turns and tool calls per person and per org within a window: sendTurn answers 429 with a clear reason when the person exceeds the limit; an org-wide limit refuses the same way even when the person is under theirs; a tool call refused by a limit returns an error to the model and is recorded in the turn's events like any failed call (RFC 8).
  3. Limit cumulative effects across turns and concurrent requests, not per action alone: a per-refund limit does not enforce a per-case or per-day limit — cumulative business effects are bounded per task, user and tenant, and child calls charge the parent's budget (RFC 8; RFC 5.1).
  4. Loop detection within one turn: the same tool called with identical canonical arguments (connectors.Digest of action and input) more than twice consecutively is refused — the call returns an error to the model instead of running (RFC 8; RFC 5.1).
  5. Cancellation and circuit breakers: an operator can cancel a turn (cancelTurn exists), and a breaker can be tripped for a tool, tool version, connector binding, agent, tenant or destination; a tripped breaker fails closed — new work is refused, never queued "for later" (RFC 8).
  6. Fail closed when a dependency needed for the decision fails: if the counting store, policy, vault, audit or downstream check cannot be read, the turn or call is refused (RFC 8; RFC 1 fail closed when facts are missing; RFC 6 unavailable policy = no protected operation).
  7. The existing step cap and timeout stay as they are (maxSteps, turnTimeout) — this issue adds limits beyond the 16-step cap, it does not lower it.

Out of scope: #2067 tool discovery, #2068 taint, #2069 memory scopes, #2070 connector credential modes, #2071 admin tools refused in chat, #2072 approval binding, #2073 image/CSP rendering, #2075 audit/revocation/kill-switch, #2078 enforcement on every entry point, #2079 chat-triggered workflow execution contracts, #2080 disclosure authorization per audience/destination. The existing per-run/per-org-day action budget in judgement's Cost guard is not changed here. Chat has no subagents yet, so subagent budget inheritance is not in scope.

Design notes: modules/chat owns this — turn.go for the token budget and loop detection (the stream is consumed in drain, so the consecutive-call counter lives beside it), api.go sendTurn for the per-person turn limit, tools.go call for tool-call limits. Count turns and tool calls through modules/chat/internal/store (the store stays the source of truth; no state lives only in memory). Deny by default: an over-limit request is refused, never queued. Pitfall: maxConcurrentTurns = 4 is global, not per person — a per-person limit must not be bypassable by opening many conversations. A resumed turn still obeys the budgets, and child calls charge the parent turn's budget.

Tests (resource-exhaustion boundary — table cases, fakes, no DB, per AGENTS.md; extend the fakes in modules/chat/access_test.go: fakeStore, fakeJudge, fakeToolbox):

  • Loop detection, one table with a fake tool recording its inputs: identical calls 1 and 2 run, the 3rd consecutive identical call is refused, a different call resets the streak, and identical calls after an intervening different call run again (guards the ">2 consecutive" regression). Must fail before the fix.
  • Turn budget: a fake provider/options check that generate passes MaxTokens (guards the budget being wired into the call, not just declared).
  • Rate limits, one table over a fake counting store: under the limit sendTurn accepts; at the person's limit it answers 429; at the org limit it refuses even when the person is under theirs; when the counting store errors, the turn is refused (fail closed).

Done when: task check passes, and on a running binary: a turn repeating the same tool call with identical arguments is cut off at the 3rd consecutive call with the refusal visible in the transcript; a turn that spends its token budget stops with the budget message; sending turns past the per-person limit answers 429 with the reason; once the org limit is reached, turns from everyone are refused; cancelling a turn stops it visibly.

Open questions for the maintainer:

  1. Where the counters live: window counts in Postgres through the chat store (durable, shared across processes — the single binary has no Redis) or an in-memory bucket per process (lost on restart)? The RFC requires bounds, not a specific store.
  2. Exact numbers: the per-turn token budget and the per-person/per-org windows and limits (the old spec suggested 100k tokens; the code already caps steps at 16 and this issue keeps it).
  3. Constants in modules/chat, or governance settings edited through the control plane? AGENTS.md requires a setting to work end to end (API and dashboard) if it is one.
  4. Should provider spend and connector quotas be hard-capped in this issue, or measured and alerted first (detection hooks are #2075)?

Part of #2066.

**Why**: a recursive or hostile prompt can make one chat turn loop model calls and tool calls without end — "keep searching until you find it" — burning model cost and hammering downstream connectors (denial of wallet). Even permitted actions can exceed what the task, the person, or the org should spend. Today a turn is bounded only by its step cap and timeout: nothing bounds its tokens, and nothing rate-limits turns or tool calls per person or per org. **Today**: - modules/chat/turn.go holds the turn's limits as constants: maxSteps = 16, turnTimeout = 10 minutes, maxConcurrentTurns = 4, historyTurns = 20. generate calls core.StreamText with MaxSteps: maxSteps only; core.GenerateOptions also has a MaxTokens field that is not set. - modules/chat/api.go sendTurn accepts any message from any signed-in person — no per-person or per-org limit on turns. - Tool calls go through modules/chat/tools.go call; there is no per-person or per-org counter on tool calls and no loop detection: the same tool with the same arguments can repeat until the step cap. connectors.Digest(action, input) is the canonical-arguments digest judgement already uses. - The only existing budgets are judgement's per-run and per-org-day changing-action budgets (the Cost guard in modules/judgement/cost.go, configured as ActionsPerRun/ActionsPerDay in governance settings under PUT /api/settings) — those count judged actions, not turns or tool calls, and are a separate mechanism. - No rate limiter is importable by the new modules: internal/ratelimit belongs to an old service, and depguard lets modules/** import only internal/platform/**. **Source**: the revised RFC exported at transcripts/2026-11-20-chat-authorization-rfc-3fa43269.md in sam/archie-product; citations below are RFC section numbers. **Required behaviour**: 1. Bound every turn's spend: pass MaxTokens in the turn's core.GenerateOptions as a named constant beside maxSteps, and when the budget is spent the turn stops with a visible budget message in the transcript (RFC 8 availability and cost; RFC 5.1 cumulative limits). 2. Bound turns and tool calls per person and per org within a window: sendTurn answers 429 with a clear reason when the person exceeds the limit; an org-wide limit refuses the same way even when the person is under theirs; a tool call refused by a limit returns an error to the model and is recorded in the turn's events like any failed call (RFC 8). 3. Limit cumulative effects across turns and concurrent requests, not per action alone: a per-refund limit does not enforce a per-case or per-day limit — cumulative business effects are bounded per task, user and tenant, and child calls charge the parent's budget (RFC 8; RFC 5.1). 4. Loop detection within one turn: the same tool called with identical canonical arguments (connectors.Digest of action and input) more than twice consecutively is refused — the call returns an error to the model instead of running (RFC 8; RFC 5.1). 5. Cancellation and circuit breakers: an operator can cancel a turn (cancelTurn exists), and a breaker can be tripped for a tool, tool version, connector binding, agent, tenant or destination; a tripped breaker fails closed — new work is refused, never queued "for later" (RFC 8). 6. Fail closed when a dependency needed for the decision fails: if the counting store, policy, vault, audit or downstream check cannot be read, the turn or call is refused (RFC 8; RFC 1 fail closed when facts are missing; RFC 6 unavailable policy = no protected operation). 7. The existing step cap and timeout stay as they are (maxSteps, turnTimeout) — this issue adds limits beyond the 16-step cap, it does not lower it. **Out of scope**: #2067 tool discovery, #2068 taint, #2069 memory scopes, #2070 connector credential modes, #2071 admin tools refused in chat, #2072 approval binding, #2073 image/CSP rendering, #2075 audit/revocation/kill-switch, #2078 enforcement on every entry point, #2079 chat-triggered workflow execution contracts, #2080 disclosure authorization per audience/destination. The existing per-run/per-org-day action budget in judgement's Cost guard is not changed here. Chat has no subagents yet, so subagent budget inheritance is not in scope. **Design notes**: modules/chat owns this — turn.go for the token budget and loop detection (the stream is consumed in drain, so the consecutive-call counter lives beside it), api.go sendTurn for the per-person turn limit, tools.go call for tool-call limits. Count turns and tool calls through modules/chat/internal/store (the store stays the source of truth; no state lives only in memory). Deny by default: an over-limit request is refused, never queued. Pitfall: maxConcurrentTurns = 4 is global, not per person — a per-person limit must not be bypassable by opening many conversations. A resumed turn still obeys the budgets, and child calls charge the parent turn's budget. **Tests** (resource-exhaustion boundary — table cases, fakes, no DB, per AGENTS.md; extend the fakes in modules/chat/access_test.go: fakeStore, fakeJudge, fakeToolbox): - Loop detection, one table with a fake tool recording its inputs: identical calls 1 and 2 run, the 3rd consecutive identical call is refused, a different call resets the streak, and identical calls after an intervening different call run again (guards the ">2 consecutive" regression). Must fail before the fix. - Turn budget: a fake provider/options check that generate passes MaxTokens (guards the budget being wired into the call, not just declared). - Rate limits, one table over a fake counting store: under the limit sendTurn accepts; at the person's limit it answers 429; at the org limit it refuses even when the person is under theirs; when the counting store errors, the turn is refused (fail closed). **Done when**: task check passes, and on a running binary: a turn repeating the same tool call with identical arguments is cut off at the 3rd consecutive call with the refusal visible in the transcript; a turn that spends its token budget stops with the budget message; sending turns past the per-person limit answers 429 with the reason; once the org limit is reached, turns from everyone are refused; cancelling a turn stops it visibly. **Open questions for the maintainer**: 1. Where the counters live: window counts in Postgres through the chat store (durable, shared across processes — the single binary has no Redis) or an in-memory bucket per process (lost on restart)? The RFC requires bounds, not a specific store. 2. Exact numbers: the per-turn token budget and the per-person/per-org windows and limits (the old spec suggested 100k tokens; the code already caps steps at 16 and this issue keeps it). 3. Constants in modules/chat, or governance settings edited through the control plane? AGENTS.md requires a setting to work end to end (API and dashboard) if it is one. 4. Should provider spend and connector quotas be hard-capped in this issue, or measured and alerted first (detection hooks are #2075)? Part of #2066.
sam added a new dependency 2026-11-20 08:53:27 +00:00
sam added the featurehigh labels 2026-11-20 10:04:31 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Reference: sam/archie#2074