Dynamically-defined curators: curator definitions as data #1595

Open
opened 2026-10-27 13:41:01 +00:00 by sam · 0 comments
Owner

Sam's requirement: "Curators should be dynamically created and defined, in addition to being able to be defined in code/config/webui" and "No configuration should ever be hard-coded." Today a curator exists only if a Go package implements curator.CuratorEngine and bootstrap.go calls curatorRegistry.Register on it. Nothing about a curator is definable as data, and no curator can even declare a tool set (see dependency below). This epic makes a curator definition a persisted, API-editable entity that a generic engine interprets.

Scope

  • A curator definition is data: name, interval, cooldown, on_input, tools, skills, memory_engine, conversations, model, instructions (free-form), enabled. The declared shape is exactly the existing curator.Manifest plus free-form instructions and identity -- no second shape is invented.
  • Three definition sources: code (a registered Go engine), config (seed data), WebUI/API (the store). The store is authoritative; config is seed data merged at startup and the store wins a name collision.
  • A generic definition-driven engine (one Go engine) executes any definition: it builds the declared tool set through Registrar.Tools, composes the system prompt from the definition's free-form instructions, and runs the pass through the shared model runtime with the declared tools attached.
  • Dynamic mutation: Registry.Unregister plus Runtime.Add/Runtime.Remove, so a create/update/delete from the API applies without a daemon restart; a removed curator's in-flight pass is cancelled.
  • REST surface mirroring /api/bindings: GET|POST /api/curators, GET|PATCH|DELETE /api/curators/{id}, same authorizeTaskMutation guard.
  • No live-path defaults: interval, cooldown, tools, model, and memory engine are definition/config values. A definition missing an interval is refused with a clear error; nothing falls back to a code constant at pass time.
  • The motivating first consumer: a project/orchestrator curator declared as data that uses forge issue tools to review, refine, and propose issues on a repo.

Non-goals

  • No fixed catalogue of curator "types" -- definitions are not a union of known kinds.
  • No chat tool to create or edit curators (see security paths).
  • No new dashboard SPA editor page; the REST surface is the WebUI definition source and the dashboard can adopt it separately.
  • No rework of the curator sampler family or the two shipped reference curators beyond removing their live-path interval constants.
  • No change to failure isolation or the "curator output never feeds its own trigger" invariant.

Acceptance criteria

  1. A curator is fully definable as data through both config and REST, with the store authoritative over config on a name collision.
  2. The generic engine runs the declared tool set with free-form instructions; a tool outside the declared set is unreachable; an unknown declared name fails that curator's pass only.
  3. No live-path constant supplies interval, cooldown, tools, model, or memory engine.
  4. Create/update/delete takes effect without a daemon restart, and deleting a definition cancels its in-flight pass and leaves no goroutine.
  5. state.proto gains matching RPCs for every persisted method added (no Go interface method without an RPC), and the State Store contract doc is updated in the same change.
  6. A healthy-but-idle curator is visible in logs; no curator is silently invisible.
  7. The project/orchestrator curator runs from data with its declared forge issue tools.

Security and failure paths

  • A definition is an execution plan that grants an LLM a tool set. Mutation authority is identical to /api/bindings (authorizeTaskMutation); no weaker path, and no chat tool that can mint curators (a curator that can create curators is privilege escalation).
  • Tool resolution fails closed: an unknown declared name fails the pass rather than silently narrowing the set; the registry already refuses a tools-declaring curator when no ToolBuilder is bound.
  • Deleting a definition must cancel an in-flight pass (context) before the row is gone; a bad definition degrades that curator only, never the daemon or a chat turn.
  • Error sentinels crossing the State Store boundary are matched on exact canonical message, which is part of the wire contract.

Dependencies

  • This epic depends on the tool-binding gap being closed: Registrar.Tools (curator.ToolBuilder) has no implementation anywhere and curatorLLMRunner drops tools, so no curator can declare tools today. Tracked as the first child issue here.
  • The motivating consumer's forge issue tools (list/create/comment issues for a repo) do not exist in the chat tool set at all; that belongs to the existing tools registry work (#161 / #140), not to this epic.
  • proto/state/v1/state.proto is one gRPC service fronting every ratified store contract; the persisted-definition change must extend it rather than adding a second authority.
  • The State Store is a standalone process that solely owns archie.db; neither the daemon nor the Gateway may open it directly.

Merged from the other tracker

[EPIC] Dynamically-defined curators: definitions as data (code/config/webui). Store/API is authoritative; config is seed data.

Sam: "Curators should be dynamically created and defined, in addition to being able to be defined in code/config/webui." and "No configuration should ever be hard-coded."

Design: docs/prds/dynamic-curators.md. GitHub epic: https://github.com/samcharles93/archie-core/issues/789

Scope: definition = curator.Manifest + name/enabled/instructions, persisted and REST-editable; generic engine (one Go engine) runs any definition via Registrar.Tools + instructions; three sources (code/config/store) with the store authoritative over config; Registry.Unregister + Runtime.Add/Remove for live mutation; REST CRUD mirroring /api/bindings; no live-path constant for interval/cooldown/tools/model/memory engine.

Non-goals: fixed catalogue of curator types; chat tool that mints curators; dashboard SPA editor; forge issue tools (existing tools work #161/#140).

Load-bearing gap: no ToolBuilder implementation exists, so no curator can declare tools (registry.go:123 fails registration) and curatorLLMRunner drops tools/MaxSteps. That is phase 1.

Notes

2026-10-25: ToolBuilder exists and is wired (internal/infrastructure/toolbuilder), curatorLLMRunner forwards Tools/MaxSteps, Runtime.ReplaceDefinitions applies definitions. Remaining: trnd.2 (per-curator Add/Remove) and trnd.3 (persistence + CRUD).


Imported from beads archie-core-trnd · GitHub #807 · body merged from the fuller GitHub #789 · type epic · priority P2 · created 2026-09-12

Sam's requirement: **"Curators should be dynamically created and defined, in addition to being able to be defined in code/config/webui"** and **"No configuration should ever be hard-coded."** Today a curator exists only if a Go package implements `curator.CuratorEngine` and `bootstrap.go` calls `curatorRegistry.Register` on it. Nothing about a curator is definable as data, and no curator can even declare a tool set (see dependency below). This epic makes a curator definition a persisted, API-editable entity that a generic engine interprets. ### Scope - A **curator definition is data**: name, interval, cooldown, `on_input`, `tools`, `skills`, `memory_engine`, `conversations`, `model`, `instructions` (free-form), `enabled`. The declared shape is exactly the existing `curator.Manifest` plus free-form instructions and identity -- no second shape is invented. - **Three definition sources**: code (a registered Go engine), config (seed data), WebUI/API (the store). **The store is authoritative**; config is seed data merged at startup and the store wins a name collision. - A **generic definition-driven engine** (one Go engine) executes any definition: it builds the declared tool set through `Registrar.Tools`, composes the system prompt from the definition's free-form instructions, and runs the pass through the shared model runtime with the declared tools attached. - **Dynamic mutation**: `Registry.Unregister` plus `Runtime.Add`/`Runtime.Remove`, so a create/update/delete from the API applies without a daemon restart; a removed curator's in-flight pass is cancelled. - **REST surface** mirroring `/api/bindings`: `GET|POST /api/curators`, `GET|PATCH|DELETE /api/curators/{id}`, same `authorizeTaskMutation` guard. - **No live-path defaults**: interval, cooldown, tools, model, and memory engine are definition/config values. A definition missing an interval is refused with a clear error; nothing falls back to a code constant at pass time. - The motivating first consumer: a **project/orchestrator curator** declared as data that uses forge issue tools to review, refine, and propose issues on a repo. ### Non-goals - No fixed catalogue of curator "types" -- definitions are not a union of known kinds. - No chat tool to create or edit curators (see security paths). - No new dashboard SPA editor page; the REST surface is the WebUI definition source and the dashboard can adopt it separately. - No rework of the curator sampler family or the two shipped reference curators beyond removing their live-path interval constants. - No change to failure isolation or the "curator output never feeds its own trigger" invariant. ### Acceptance criteria 1. A curator is fully definable as data through both config and REST, with the store authoritative over config on a name collision. 2. The generic engine runs the declared tool set with free-form instructions; a tool outside the declared set is unreachable; an unknown declared name fails that curator's pass only. 3. No live-path constant supplies interval, cooldown, tools, model, or memory engine. 4. Create/update/delete takes effect without a daemon restart, and deleting a definition cancels its in-flight pass and leaves no goroutine. 5. `state.proto` gains matching RPCs for every persisted method added (no Go interface method without an RPC), and the State Store contract doc is updated in the same change. 6. A healthy-but-idle curator is visible in logs; no curator is silently invisible. 7. The project/orchestrator curator runs from data with its declared forge issue tools. ### Security and failure paths - A definition is an execution plan that grants an LLM a tool set. Mutation authority is identical to `/api/bindings` (`authorizeTaskMutation`); no weaker path, and no chat tool that can mint curators (a curator that can create curators is privilege escalation). - Tool resolution fails closed: an unknown declared name fails the pass rather than silently narrowing the set; the registry already refuses a tools-declaring curator when no `ToolBuilder` is bound. - Deleting a definition must cancel an in-flight pass (context) before the row is gone; a bad definition degrades that curator only, never the daemon or a chat turn. - Error sentinels crossing the State Store boundary are matched on exact canonical message, which is part of the wire contract. ### Dependencies - This epic **depends on the tool-binding gap** being closed: `Registrar.Tools` (`curator.ToolBuilder`) has no implementation anywhere and `curatorLLMRunner` drops tools, so no curator can declare tools today. Tracked as the first child issue here. - The motivating consumer's forge issue tools (list/create/comment issues for a repo) do not exist in the chat tool set at all; that belongs to the existing tools registry work ([#161](https://github.com/samcharles93/archie-core/issues/161) / [#140](https://github.com/samcharles93/archie-core/issues/140)), not to this epic. - `proto/state/v1/state.proto` is one gRPC service fronting every ratified store contract; the persisted-definition change must extend it rather than adding a second authority. - The State Store is a standalone process that solely owns `archie.db`; neither the daemon nor the Gateway may open it directly. ### Merged from the other tracker [EPIC] Dynamically-defined curators: definitions as data (code/config/webui). Store/API is authoritative; config is seed data. Sam: "Curators should be dynamically created and defined, in addition to being able to be defined in code/config/webui." and "No configuration should ever be hard-coded." Design: docs/prds/dynamic-curators.md. GitHub epic: https://github.com/samcharles93/archie-core/issues/789 Scope: definition = curator.Manifest + name/enabled/instructions, persisted and REST-editable; generic engine (one Go engine) runs any definition via Registrar.Tools + instructions; three sources (code/config/store) with the store authoritative over config; Registry.Unregister + Runtime.Add/Remove for live mutation; REST CRUD mirroring /api/bindings; no live-path constant for interval/cooldown/tools/model/memory engine. Non-goals: fixed catalogue of curator types; chat tool that mints curators; dashboard SPA editor; forge issue tools (existing tools work #161/#140). Load-bearing gap: no ToolBuilder implementation exists, so no curator can declare tools (registry.go:123 fails registration) and curatorLLMRunner drops tools/MaxSteps. That is phase 1. ### Notes 2026-10-25: ToolBuilder exists and is wired (internal/infrastructure/toolbuilder), curatorLLMRunner forwards Tools/MaxSteps, Runtime.ReplaceDefinitions applies definitions. Remaining: trnd.2 (per-curator Add/Remove) and trnd.3 (persistence + CRUD). --- _Imported from beads `archie-core-trnd` · GitHub #807 · body merged from the fuller GitHub #789 · type epic · priority P2 · created 2026-09-12_
sam added the mediumepic labels 2026-10-27 13:41:01 +00:00
sam self-assigned this 2026-10-27 13:41:01 +00:00
sam added a new dependency 2026-11-14 14:45:07 +00:00
sam added the deferred label 2026-11-15 17:26:15 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Reference: sam/archie#1595