Chat offers the model only the tools its person may use #2067

Closed
opened 2026-11-20 08:41:09 +00:00 by sam · 1 comment
Owner

Why: Today the model is shown every connector tool in its prompt, and access is checked only when a tool is called. A person who may not use a tool can still see its name, description and parameter schema ("list all your tools"), and a prompt-injected model can emit a call to a withheld tool that only fails at execution time. Discovery must be filtered before prompt construction, and any invocation outside the permitted set must fail closed.

Today: modules/chat/tools.go tools() builds the tool set from every d.Toolbox.Declarations() entry plus builtins(), and toolSummaries() feeds that same set into the system prompt via prompts.ChatSystem in modules/chat/turn.go generate(). The turn acts as the conversation owner's current principal (actingAs in tools.go, put into ctx by generate()), but discovery is unfiltered: access is checked only at call time, in call() (m.allowed with ActionRun for actions, ActionRead for lookups, resource kind KindConnector) and in guarded() wrapping each builtin. There is no separate versioned tool manifest in chat; the running connectors' declaration set plus the builtins is the de facto manifest, and the session leg is the owner binding.

Source: the revised RFC exported at transcripts/2026-11-20-chat-authorization-rfc-3fa43269.md in sam/archie-product; citations below are RFC section numbers.

Required behaviour:

  1. Before each model call, tools() omits every connector declaration the acting principal may not use, making the same per-tool decision call() already makes: m.allowed with ActionRun for actions and ActionRead for lookups on the connector resource (RFC 3 stage 3, RFC 5.1).
  2. Builtins are filtered the same way using the action and resource kind each is guarded with: run_list/run_get read KindRun, approval_list reads KindApproval, memory_list reads KindSettings, memory_save/memory_forget update KindSettings, dashboard_navigate stays (RFC 3 stage 3).
  3. Roles grant eligibility for business actions, not blanket tool implementations or credentials: the discovery and invocation decision is per connector action, resolved against the acting principal's grants on stable, tenant-qualified resources, never on model-supplied names or IDs (RFC 2, RFC 1).
  4. The tool set handed to the model and the tool summaries in the system prompt come from the same filtered set, so a withheld tool's name, description and schema appear nowhere the model can probe (RFC 5.1).
  5. Unknown, stale or omitted tool IDs fail at invocation: a call naming a connector or action outside the pinned, running registry is refused without running and without echoing what exists (RFC 3 stage 4, RFC 5.1, RFC 6).
  6. Fail closed: when the policy chain is unavailable (governance.ErrChainUnavailable) or any fact needed for the decision is missing, tools are omitted at discovery and refused at invocation (RFC 1, RFC 6).
  7. Effective permission for a turn is the intersection of the acting person's current access and the declared tool universe, never the union; actingAs re-reads the owner's memberships every turn, so access the owner has since lost applies immediately, including on a resumed turn (RFC 1, RFC 3 stage 2).
  8. Every invocation of a permitted tool is still judged and recorded as a decision under the turn (call() assesses via Judge.Assess with the turn ID as run ID), so cumulative effect limits across turns and concurrent requests charge against a decision trail (RFC 5.1); the budget arithmetic itself is #2074.

Out of scope: #2068 taint, #2069 memory scopes, #2070 connector credential modes, #2071 admin tools refused in chat, #2072 approval binding, #2073 image/CSP rendering, #2074 budgets and rate limits, #2075 audit, revocation and kill-switch, #2078 enforcement on every entry point, #2079 chat-triggered workflow execution contracts, #2080 disclosure authorization per audience and destination. Also out of scope: versioned-manifest admission and pinning of reviewed tool versions beyond what the running declaration set already provides (open question below).

Design notes: owning module is modules/chat: tools() and builtins() in modules/chat/tools.go, prompt assembly in modules/chat/turn.go generate(). Reuse m.allowed unchanged as the single per-tool decision, called once per declaration and per builtin and skipping denied ones; the decision point is modules/governance/engine.go Authorize, with ErrChainUnavailable treated as deny. Behaviour-only, no schema migration. Pitfalls: filter after actingAs so a resumed turn still acts as its owner; never let the model influence which tools are listed; keep the call-time check in call() and guarded() as the second enforcement point so a hallucinated or injected call to an omitted tool is refused, never run.

Tests: security-boundary seam, so table tests with fakes and no DB, alongside modules/chat/access_test.go:

  • an admin's tools() includes a connector action tool; a viewer's omits it, and the omitted name appears nowhere in toolSummaries() output
  • a viewer still gets the same connector's lookup tools while its action tools are omitted
  • a viewer's builtin set omits memory_save but includes run_list
  • with the policy chain unavailable, no connector tools are offered
  • calling an omitted or unknown tool directly through call() is refused and nothing executes
  • a principal whose role changed between turns gets the narrower set on the next turn

Done when: task check passes with the new tests; on a running binary (task dev), signed in as a viewer, asking chat "what tools do you have" lists only tools the viewer may use and an attempted call to a withheld tool is refused at call time; signed in as an admin, the same conversation lists the action tools.

Open questions for the maintainer:

  • The access resource is per connector (resource id is the connector name), so all of a connector's tools share one decision. RFC 2 defines per business-action/tool-version permissions (discover, request_action, invoke, approve, manage_definition). Should discovery move to per-tool resources (for example resource id connector/action) so policy can admit some tools of a connector and not others?
  • Chat has no versioned tool manifest today; the running connectors' declarations are the manifest. What represents a reviewed, pinned version that a declaration change must be re-admitted against before it is offered (RFC 5.1)?

Part of #2066.

**Why**: Today the model is shown every connector tool in its prompt, and access is checked only when a tool is called. A person who may not use a tool can still see its name, description and parameter schema ("list all your tools"), and a prompt-injected model can emit a call to a withheld tool that only fails at execution time. Discovery must be filtered before prompt construction, and any invocation outside the permitted set must fail closed. **Today**: modules/chat/tools.go tools() builds the tool set from every d.Toolbox.Declarations() entry plus builtins(), and toolSummaries() feeds that same set into the system prompt via prompts.ChatSystem in modules/chat/turn.go generate(). The turn acts as the conversation owner's current principal (actingAs in tools.go, put into ctx by generate()), but discovery is unfiltered: access is checked only at call time, in call() (m.allowed with ActionRun for actions, ActionRead for lookups, resource kind KindConnector) and in guarded() wrapping each builtin. There is no separate versioned tool manifest in chat; the running connectors' declaration set plus the builtins is the de facto manifest, and the session leg is the owner binding. **Source**: the revised RFC exported at transcripts/2026-11-20-chat-authorization-rfc-3fa43269.md in sam/archie-product; citations below are RFC section numbers. **Required behaviour**: 1. Before each model call, tools() omits every connector declaration the acting principal may not use, making the same per-tool decision call() already makes: m.allowed with ActionRun for actions and ActionRead for lookups on the connector resource (RFC 3 stage 3, RFC 5.1). 2. Builtins are filtered the same way using the action and resource kind each is guarded with: run_list/run_get read KindRun, approval_list reads KindApproval, memory_list reads KindSettings, memory_save/memory_forget update KindSettings, dashboard_navigate stays (RFC 3 stage 3). 3. Roles grant eligibility for business actions, not blanket tool implementations or credentials: the discovery and invocation decision is per connector action, resolved against the acting principal's grants on stable, tenant-qualified resources, never on model-supplied names or IDs (RFC 2, RFC 1). 4. The tool set handed to the model and the tool summaries in the system prompt come from the same filtered set, so a withheld tool's name, description and schema appear nowhere the model can probe (RFC 5.1). 5. Unknown, stale or omitted tool IDs fail at invocation: a call naming a connector or action outside the pinned, running registry is refused without running and without echoing what exists (RFC 3 stage 4, RFC 5.1, RFC 6). 6. Fail closed: when the policy chain is unavailable (governance.ErrChainUnavailable) or any fact needed for the decision is missing, tools are omitted at discovery and refused at invocation (RFC 1, RFC 6). 7. Effective permission for a turn is the intersection of the acting person's current access and the declared tool universe, never the union; actingAs re-reads the owner's memberships every turn, so access the owner has since lost applies immediately, including on a resumed turn (RFC 1, RFC 3 stage 2). 8. Every invocation of a permitted tool is still judged and recorded as a decision under the turn (call() assesses via Judge.Assess with the turn ID as run ID), so cumulative effect limits across turns and concurrent requests charge against a decision trail (RFC 5.1); the budget arithmetic itself is #2074. **Out of scope**: #2068 taint, #2069 memory scopes, #2070 connector credential modes, #2071 admin tools refused in chat, #2072 approval binding, #2073 image/CSP rendering, #2074 budgets and rate limits, #2075 audit, revocation and kill-switch, #2078 enforcement on every entry point, #2079 chat-triggered workflow execution contracts, #2080 disclosure authorization per audience and destination. Also out of scope: versioned-manifest admission and pinning of reviewed tool versions beyond what the running declaration set already provides (open question below). **Design notes**: owning module is modules/chat: tools() and builtins() in modules/chat/tools.go, prompt assembly in modules/chat/turn.go generate(). Reuse m.allowed unchanged as the single per-tool decision, called once per declaration and per builtin and skipping denied ones; the decision point is modules/governance/engine.go Authorize, with ErrChainUnavailable treated as deny. Behaviour-only, no schema migration. Pitfalls: filter after actingAs so a resumed turn still acts as its owner; never let the model influence which tools are listed; keep the call-time check in call() and guarded() as the second enforcement point so a hallucinated or injected call to an omitted tool is refused, never run. **Tests**: security-boundary seam, so table tests with fakes and no DB, alongside modules/chat/access_test.go: - an admin's tools() includes a connector action tool; a viewer's omits it, and the omitted name appears nowhere in toolSummaries() output - a viewer still gets the same connector's lookup tools while its action tools are omitted - a viewer's builtin set omits memory_save but includes run_list - with the policy chain unavailable, no connector tools are offered - calling an omitted or unknown tool directly through call() is refused and nothing executes - a principal whose role changed between turns gets the narrower set on the next turn **Done when**: task check passes with the new tests; on a running binary (task dev), signed in as a viewer, asking chat "what tools do you have" lists only tools the viewer may use and an attempted call to a withheld tool is refused at call time; signed in as an admin, the same conversation lists the action tools. **Open questions for the maintainer**: - The access resource is per connector (resource id is the connector name), so all of a connector's tools share one decision. RFC 2 defines per business-action/tool-version permissions (discover, request_action, invoke, approve, manage_definition). Should discovery move to per-tool resources (for example resource id connector/action) so policy can admit some tools of a connector and not others? - Chat has no versioned tool manifest today; the running connectors' declarations are the manifest. What represents a reviewed, pinned version that a declaration change must be re-admitted against before it is offered (RFC 5.1)? Part of #2066.
sam added a new dependency 2026-11-20 08:52:31 +00:00
sam added the featurehigh labels 2026-11-20 10:04:31 +00:00
Author
Owner

Implemented in 96fb0afcd.

  • Discovery now makes the same per-tool decision as invocation: a connector lookup needs read, an action needs run, and every guarded builtin carries the action and resource kind it checks. The model's tool schemas and the system prompt summaries are built once from that filtered set.
  • A tool name claimed by two declarations, or by a declaration and a builtin, is withheld from all of them rather than letting one shadow the other.
  • The stream client fixes the wire tools for the turn, so a guarded provider re-reads the owner's access and the running declarations before every model request. If an offered tool is no longer permitted or declared, the turn stops before that request; it never sends a stale schema or extends the stale summary.
  • A call to a declaration the registry no longer holds is refused before anything runs and names nothing. The call-time check remains behind discovery as the second enforcement point.
  • Memory notes are omitted from the prompt when the person may not read settings, so the prompt path does not bypass the filtered memory tools.

Tests in modules/chat/access_test.go cover admin versus viewer discovery, lookup versus action on one connector, builtin filtering, an unavailable policy chain, a stale declaration that must not run, a role change between turns, a withdrawal between model requests, and name collisions. task check passes on the commit.

The remaining verification is the live one from the issue: a disposable binary with an agent model configured, signed in as a viewer, checking the captured provider request and an attempted withheld call. It was not run here; the tests assert the same provider-facing set and the no-execution case.

Implemented in 96fb0afcd. - Discovery now makes the same per-tool decision as invocation: a connector lookup needs read, an action needs run, and every guarded builtin carries the action and resource kind it checks. The model's tool schemas and the system prompt summaries are built once from that filtered set. - A tool name claimed by two declarations, or by a declaration and a builtin, is withheld from all of them rather than letting one shadow the other. - The stream client fixes the wire tools for the turn, so a guarded provider re-reads the owner's access and the running declarations before every model request. If an offered tool is no longer permitted or declared, the turn stops before that request; it never sends a stale schema or extends the stale summary. - A call to a declaration the registry no longer holds is refused before anything runs and names nothing. The call-time check remains behind discovery as the second enforcement point. - Memory notes are omitted from the prompt when the person may not read settings, so the prompt path does not bypass the filtered memory tools. Tests in modules/chat/access_test.go cover admin versus viewer discovery, lookup versus action on one connector, builtin filtering, an unavailable policy chain, a stale declaration that must not run, a role change between turns, a withdrawal between model requests, and name collisions. task check passes on the commit. The remaining verification is the live one from the issue: a disposable binary with an agent model configured, signed in as a viewer, checking the captured provider request and an attempted withheld call. It was not run here; the tests assert the same provider-facing set and the no-execution case.
sam closed this issue 2026-11-20 14:47:08 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Reference: sam/archie#2067