workflow: decide step answers typed questions with a decision model #1539

Open
opened 2026-10-27 13:40:48 +00:00 by sam · 0 comments
Owner

Many workflow decisions are classification, not generation: triage choosing implement, tdd or feasibility; the feasibility fit verdict; routing an event to a workflow; switch conditions over free text. Today each runs a full agent step in a container. Native decision models served on an OpenAI-compatible server's /v1/systemone endpoint (TypeSafe System One API) answer typed questions (choice, score, yes/no) about a state in about a second, with a probability per option and a confidence.

Wanted: a decide step type. Settings: the model alias (a provider whose model reports the decisions output modality), the state (templated from task, inputs or earlier step outputs), and named questions with type, instructions and criteria. Outputs: per question the answer, probabilities and confidence, usable by when and switch. A min_confidence setting routes a low-confidence answer to a fallback: an agent step or human.approve. Runs in the daemon, no container. The model is configured through the control plane like other providers.

Done when triage's classify step can be replaced by a decide step that picks the workflow, a low-confidence answer falls back to the agent step, and the decision with its probabilities is visible on the task.

Acceptance criteria

  1. Triage's classify step can be replaced by a decide step that picks the workflow.
  2. A low-confidence answer, or an OpenAI refusal, falls back to the agent step.
  3. The decision with its probabilities is visible on the task.
  4. The step works with both an OpenAI and a llama.cpp System One provider configured through the control plane.

Notes

Needs ai-sdk-e1q (decide package: OpenAI Decisions API and llama.cpp System One) released and bumped in go.mod first. The step works with either provider; predicate maps to System One noul, and an OpenAI refusal takes the low-confidence fallback.


Imported from beads archie-core-qprg5 · GitHub #1661 · type feature · priority P2 · created 2026-10-12

Many workflow decisions are classification, not generation: triage choosing implement, tdd or feasibility; the feasibility fit verdict; routing an event to a workflow; switch conditions over free text. Today each runs a full agent step in a container. Native decision models served on an OpenAI-compatible server's /v1/systemone endpoint (TypeSafe System One API) answer typed questions (choice, score, yes/no) about a state in about a second, with a probability per option and a confidence. Wanted: a decide step type. Settings: the model alias (a provider whose model reports the decisions output modality), the state (templated from task, inputs or earlier step outputs), and named questions with type, instructions and criteria. Outputs: per question the answer, probabilities and confidence, usable by when and switch. A min_confidence setting routes a low-confidence answer to a fallback: an agent step or human.approve. Runs in the daemon, no container. The model is configured through the control plane like other providers. Done when triage's classify step can be replaced by a decide step that picks the workflow, a low-confidence answer falls back to the agent step, and the decision with its probabilities is visible on the task. ### Acceptance criteria 1. Triage's classify step can be replaced by a decide step that picks the workflow. 2. A low-confidence answer, or an OpenAI refusal, falls back to the agent step. 3. The decision with its probabilities is visible on the task. 4. The step works with both an OpenAI and a llama.cpp System One provider configured through the control plane. ### Notes Needs ai-sdk-e1q (decide package: OpenAI Decisions API and llama.cpp System One) released and bumped in go.mod first. The step works with either provider; predicate maps to System One noul, and an OpenAI refusal takes the low-confidence fallback. --- _Imported from beads `archie-core-qprg5` · GitHub #1661 · type feature · priority P2 · created 2026-10-12_
sam added the mediumfeature labels 2026-10-27 13:40:48 +00:00
sam self-assigned this 2026-10-27 13:40:48 +00:00
sam added the deferred label 2026-11-15 17:26:16 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sam/archie#1539