Replace model roles with an instance model catalogue, per-org allow-lists, BYO providers and aliases #1205

Closed
opened 2026-10-27 13:39:26 +00:00 by sam · 0 comments
Owner

Today models are chosen by role (cfg.Models[role]: builder/planner/triage/review plus embedding/transcription), hard-coded across workflow stages, chat and MCP sampling with a silent builder fallback. There is no model picker in chat or on the canvas.

Design (persona council, maintainer decisions):

  • Instance catalogue at org-sys: providers (class, base_url, credential binding) and model entries, plus chat/embedding/transcription defaults. This is the whole configuration for a single-org install; org UI stays hidden until a second org exists.
  • Per-org allow-list: the instance admin picks which catalogue entries each org may use.
  • Org BYO providers behind an instance flag: an org may add providers under its own credentials when enabled. Secret refs and bindings must belong to the writing org (owner check on write).
  • Alias table: steps and chat reference an alias (e.g. fast, default); each org maps aliases to entries it may use, falling back to the instance mapping. Empty means the default alias for the purpose.
  • One resolver resolve(org, alias-or-empty, purpose) used by agent steps, implement, pr-review, MCP sampling, chat and embedding. Unknown alias fails at apply time and at dispatch, never substituted. Roles and every Models[role] read are deleted, no shim.
  • Usage records: every model call appends {org, source (task/chat/curator), task, workflow, step, alias, resolved provider/model, input/output tokens, time} to a state-store usage table. No pricing or billing UI yet; chargeback is a later aggregation over these records. Chat records org-sys until the gateway carries an org.
  • Embedding stays instance-only until memory is partitioned by org.

Slices: (1) archie-core-dajez credential leak fix; (2) instance catalogue and aliases replace roles, single resolver, step/chat pickers, apply/dispatch validation; (3) usage records; (4) per-task org resolution; (5) org Settings: allow-list, alias remap, BYO providers behind flag; (6) org-scoped chat in the gateway.

Done when: no Models[role] lookup remains; a self-hoster configures models in one instance screen; steps and chat pick aliases; a second org sees only allowed or own models; every model call leaves a usage record; task check passes.

Notes

Slice 2 landed: aliases replace roles (model-aliases resource, config.ResolveModel), step and chat pickers, apply-time check for workflow definitions. Not done in slice 2: a separate instance catalogue of entries (aliases point straight at provider/model refs); it arrives with the per-org allow-list in slice 5.
Slice 3 landed: model_usage table, RecordUsage RPC (run credential limited to its own task, task rows billed to the task's org), recorded from task agent runs, worker MCP sampling, gateway chat turns, titles, sampling and curators. Embedding and transcription calls are not recorded yet.
Slice 4 landed: TaskModelAliases RPC (org from the task row), daemon sends the task org's effective aliases to the agent, egress and credential resolver use task.Org. An org's own model-aliases resource is read but nothing writes it yet; that is slice 5.
Slice 5 landed: org-model-policy (deny by default), org model-aliases and provider-settings validated against it, ModelsFor resolution, TaskModels RPC carries org providers, resolver grants org-own provider keys, /org/models tab and Org access section. Gap: an org cannot create its own credential bindings; the instance admin adds a binding with that org.

Close reason

Slices 1-6 landed: leak fix, aliases, usage records, task org resolution, org policy/aliases/providers, org-scoped chat models, usage and sessions.


Imported from beads archie-core-9kzqi · GitHub #1510 · type feature · priority P1 · created 2026-10-10 · closed 2026-10-10

Today models are chosen by role (cfg.Models[role]: builder/planner/triage/review plus embedding/transcription), hard-coded across workflow stages, chat and MCP sampling with a silent builder fallback. There is no model picker in chat or on the canvas. Design (persona council, maintainer decisions): - Instance catalogue at org-sys: providers (class, base_url, credential binding) and model entries, plus chat/embedding/transcription defaults. This is the whole configuration for a single-org install; org UI stays hidden until a second org exists. - Per-org allow-list: the instance admin picks which catalogue entries each org may use. - Org BYO providers behind an instance flag: an org may add providers under its own credentials when enabled. Secret refs and bindings must belong to the writing org (owner check on write). - Alias table: steps and chat reference an alias (e.g. fast, default); each org maps aliases to entries it may use, falling back to the instance mapping. Empty means the default alias for the purpose. - One resolver resolve(org, alias-or-empty, purpose) used by agent steps, implement, pr-review, MCP sampling, chat and embedding. Unknown alias fails at apply time and at dispatch, never substituted. Roles and every Models[role] read are deleted, no shim. - Usage records: every model call appends {org, source (task/chat/curator), task, workflow, step, alias, resolved provider/model, input/output tokens, time} to a state-store usage table. No pricing or billing UI yet; chargeback is a later aggregation over these records. Chat records org-sys until the gateway carries an org. - Embedding stays instance-only until memory is partitioned by org. Slices: (1) archie-core-dajez credential leak fix; (2) instance catalogue and aliases replace roles, single resolver, step/chat pickers, apply/dispatch validation; (3) usage records; (4) per-task org resolution; (5) org Settings: allow-list, alias remap, BYO providers behind flag; (6) org-scoped chat in the gateway. Done when: no Models[role] lookup remains; a self-hoster configures models in one instance screen; steps and chat pick aliases; a second org sees only allowed or own models; every model call leaves a usage record; task check passes. ### Notes Slice 2 landed: aliases replace roles (model-aliases resource, config.ResolveModel), step and chat pickers, apply-time check for workflow definitions. Not done in slice 2: a separate instance catalogue of entries (aliases point straight at provider/model refs); it arrives with the per-org allow-list in slice 5. Slice 3 landed: model_usage table, RecordUsage RPC (run credential limited to its own task, task rows billed to the task's org), recorded from task agent runs, worker MCP sampling, gateway chat turns, titles, sampling and curators. Embedding and transcription calls are not recorded yet. Slice 4 landed: TaskModelAliases RPC (org from the task row), daemon sends the task org's effective aliases to the agent, egress and credential resolver use task.Org. An org's own model-aliases resource is read but nothing writes it yet; that is slice 5. Slice 5 landed: org-model-policy (deny by default), org model-aliases and provider-settings validated against it, ModelsFor resolution, TaskModels RPC carries org providers, resolver grants org-own provider keys, /org/models tab and Org access section. Gap: an org cannot create its own credential bindings; the instance admin adds a binding with that org. ### Close reason Slices 1-6 landed: leak fix, aliases, usage records, task org resolution, org policy/aliases/providers, org-scoped chat models, usage and sessions. --- _Imported from beads `archie-core-9kzqi` · GitHub #1510 · type feature · priority P1 · created 2026-10-10 · closed 2026-10-10_
sam added the highfeature labels 2026-10-27 13:39:26 +00:00
sam self-assigned this 2026-10-27 13:39:26 +00:00
sam closed this issue 2026-10-27 13:39:26 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Reference: sam/archie#1205