Commit Graph
1809 Commits
Author SHA1 Message Date
sam 9e4e15dafb feat(harness): declare and enforce a workflow's harness needs at dispatch
docs/prds/external-agent-harness.md ("Contract"): workflow definition
validation must reject a stage whose needs the selected harness cannot
meet -- capture tools without an adapter, or a gate retry without a
resume verb. The step vocabulary had no static way to declare either.

Added task.WorkflowInterface.Needs{Captures, GateRetries}, parsed and
validated (GateRetries >= 0) the same way Inputs/Repository already are.
Only a Kit profile is checked against it: an image profile's built-in
agent loop always supports both, so Needs constrains only the harness path.

Two enforcement points, chosen for where each fact is actually known
without a network fetch at save time:
- Captures vs adapter is knowable from config alone (profile.Adapter, the
  agentexec.harnessAdapters table), so daemon.pinTaskProfile checks it
  right where it already resolves the profile (validateProfileMeetsNeeds),
  parking the task the same way an unconfigured profile already does.
- GateRetries vs the Kit's actual agent-sessions resume verb is only known
  once the Kit composition is fetched, which kitrun.Launcher.Launch
  already does -- so kit.ValidateNeeds runs there (new kitrun.Request.
  GateRetries field, threaded from the workflow's parsed Needs in
  daemon.runKitTask) rather than inventing an image-fetching workflow-save
  validator.

Deliberately NOT built: "a Kit profile named by no workflow is a
validation error" (external-agent-harness.md "Selection"). Read literally
("a Kit profile [that is] named by no workflow"), this is a
cross-resource check -- an agent-profiles entry unreferenced by anything
in workflow-definitions -- and no control-plane Definition validator can
see another resource today (Definition.Validate is a pure []byte->error
function by design, matching every existing resource kind). Building
that crossing is a real, separate piece of infrastructure with its own
ordering questions (does defining a Kit profile before its workflow now
fail?), not a two-line addition, so it is left for a follow-on rather
than bolted on here.

Also fixes a test bug caught by rerunning with a short -timeout per
instruction: TestRunKitTaskThreadsGateRetriesToTheLauncher's fake NATS
subscriber never responded, so runKitTask's call into runViaAgent blocked
forever waiting for a reply bounded by a zero (unconfigured) wall-clock
budget. Fixed by responding, matching the sibling test's pattern.
2026-09-27 05:25:39 +10:00
sam 4d6f49f7f3 docs(harness): pin per-stage profile override as workflow.call, not a mid-container switch
docs/prds/external-agent-harness.md's "Selection" says a stage may name a
different profile than its workflow's. Investigated before writing new
runtime code, per instruction not to build container-switching machinery:
a Kit task runs in one container provisioned for one profile (Container
layout), so a stage needing a different profile cannot be a mid-container
runner swap without inventing per-stage re-provisioning the PRD does not
ask for and Container layout argues against ("cost scales with the run,
not with how many agents it fans out to").

The requirement is already met by workflow.call plus per-task profile
resolution, both pre-existing: postgres.Store.StartCall stores the
callee's OWN Workflow name (not the caller's), and daemon.pinTaskProfile
resolves a task's profile purely from its own WorkflowDefinitionYAML.
Chaining these means a stage wanting a different profile is its own
callee workflow, started via workflow.call, exactly the way
event-automation.md already describes ("the callee runs as its own run
with its own agent profile").

No new production code, only a PRD clarification (dev-docs regenerated)
plus TestWorkflowCallGivesACalleeItsOwnProfile (daemon_test.go), which
closes the gap between two already-tested facts that nothing previously
stated together: TestWorkflowCallerConformance (staterpc) proves a
callee's Workflow field is independent of its caller's; this proves that
independence is what makes Kit-vs-image runner selection independent too,
composing a caller pinned to an image profile with a real StartCall-
created callee pinned to a Kit profile.
2026-09-27 05:25:39 +10:00
sam adf44d827f feat(controlplane): agent profiles as their own live control-plane resource
docs/prds/external-agent-harness.md ("Selection"): a Kit profile is an org
resource that applies without a restart. Profiles were bundled inside
container-runtime-policies (ApplyMode: restart-required), which is correct
for Image/MaxConcurrency/Network but not for Profiles, which nothing
startup-builds: daemon.configFor reads config.Holder fresh per dispatch
(pinTaskProfile -> Containers.Profile).

Split Profiles into its own resource, AgentProfileKind ("agent-profiles"),
ApplyMode: live, seeded from [containers.profiles] and validated with the
same rules the file document uses (config.ValidateAgentProfiles, factored
out of ContainerConfig.ValidateProfiles so the two cannot disagree).
config.ContainerConfig.Profiles is now json:"-" so container-runtime-policies
never carries it (one authoritative resource per field, not two), and
runtime_config.go layers the new resource in by wholesale map replacement
(matching ModelRoleAssignmentsKind's precedent) rather than merging into the
file's map. Added Profiles to reload.go's Containers reloadable-subfield set
to match this.

Deliberately NOT built here: true per-org multi-tenant storage for this
resource (docs/prds/orgs-and-access.md places agent profiles under org
ownership, granted to workspaces). No existing control-plane resource kind
in this registry does real per-org storage yet either -- schedules,
workflow settings, etc. are all single instance-wide documents -- so
agent-profiles follows that same established, tested pattern rather than
inventing org-scoped storage unilaterally for this one resource. Org/
workspace grants over agent-profiles are a follow-on, not a redo: nothing
about this resource's shape needs to change to add that scoping later.
2026-09-27 05:25:38 +10:00
sam 04dfa29dcb fix(webhook): reject rate-limited deliveries with 429 instead of chat prose or a false 202
Router.Route returned the chat-facing rate-limit prose with a nil error, so
the webhook handler had no way to tell a genuine dispatch outcome from a
rejection: DeliverTo "origin" echoed the prose as a 200 reply, and any
other route got a 202 for an event that was actually dropped before
reaching the LLM, so the source never retries it.

ChatReply now carries RateLimited, set by a new Router.RouteResult (Route
is now a thin wrapper over it) so chat channels that render Text to a
human are unaffected. The webhook handler checks it and answers 429,
matching the precedent internal/webhookguard.RateLimiter's other callers
already set.
2026-09-27 05:20:30 +10:00
sam 8d6c3e3c99 fix(archied): republish the config snapshot when live settings apply
applyWorkflowExecutionSettings swapped the running config with cfgHolder.Set,
so the dashboard's published projection kept the previous budgets until the
next reload. Route it through publishConfig like every other config change.

Fixes #1148
2026-09-27 05:09:24 +10:00
sam 69ba1f19bc Merge crew/pi-dashboard-sweep: container document shape, systemd READY, capture route constant, contract tests 2026-09-27 05:04:57 +10:00
sam 9b0f7e82e5 Merge crew/codex-1184: provider secret resolution test, watch outage reporting, projection parity guard 2026-09-27 05:04:43 +10:00
sam 7470ac310d test(controlplane): guard mirrored settings projections 2026-09-26 21:02:16 +10:00
sam fcbdd757e9 fix(webui): mount the capture intake route from the receiver's constant
The bypass mux mounted captureintake's route from a local literal that
duplicated captureintake.Path, with a comment acknowledging the drift risk:
the kind of duplication that silently stops matching after a rename
(archie-core-1175). Reference the constant instead, so the receiver owns the
one spelling of the route and a rename there cannot unmount here.

The existing routing tests are the drift coverage: TestCaptureIntakeMountedOnBypassMux
drives Handler() with a POST to the real path, so a rename on either side that
stopped matching would fail it; no new test is needed. This gives internal/webui
its first infrastructure import -- the deleted literal existed so webui could
mount an http.Handler without depending on the receiver's concrete package, but
mounting by the owner's constant is exactly the boundary the intake cutover
drew: webui mounts and reads, captureintake owns the write path.
2026-09-26 20:44:40 +10:00
sam a696735de1 fix(deploy): announce systemd READY from archie-gateway and archie-state-store
2fcc32d8 gave archied sd_notify: one READY=1 when the run loop starts serving.
RunGateway and RunStateStore serve contracts and health surfaces of their own --
the State Store even takes -ready-addr for a readiness HTTP listener -- but told
systemd nothing, so run as Type=notify either would hang until TimeoutStartSec
and be killed in a restart loop while perfectly healthy (archie-core-1174).

Both now announce at the moment each declares itself serving, the same notion
archied's runLoop announces beside healthSurface.markServing -- no second
definition of started. The Gateway announces from its serving tail
(serveGatewayListener, extracted from RunGateway), after the listener is bound;
the State Store announces after its optional surfaces have started, because a
readiness surface that fails to bind fails the boot and READY must not precede
it, and before serveStateStore, which begins accepting the moment it is called.
No unit file changes Type=notify for them; the helper already lives in package
archied beside both entry points, so no move was needed.

Tests follow the notify pattern the daemon's announcement already has: the
Gateway test drives its serving tail over a bound loopback listener with a
stub contract and judges READY against a real Snapshot RPC on the same socket;
the State Store test drives the real RunStateStore boot over a pgtest database
and waits for the datagram. Both failed red before the announcement existed --
no datagram within the window -- and pass after.
2026-09-26 20:39:56 +10:00
sam 5b1f46b7c3 fix(archied): report workflow settings watch outages 2026-09-26 20:38:48 +10:00
sam 45421770a2 test(store): a resource request ID is idempotent per kind
The per-kind request-ID dedup was fixed in ece0f28c for the SQLite store and
carried into the PostgreSQL rewrite (cebbcb56) from day one: ResourceByRequest
filters WHERE kind = $1 AND request_id = $2 and resource_history carries
UNIQUE(kind, request_id). What neither rewrite kept is the test that pins the
cross-kind half of the contract -- the deleted internal/store suite had one, and
the Postgres suite only covers same-kind replay.

Pin it, and prove the failure mode it guards against: with the kind filter
dropped from the query, a request ID reused across two kinds silently returns
the first kind's history row under the second kind's write -- wrong kind, wrong
value, no error (archie-core-1179's latent hazard). Restore the filter and both
kinds write their own revisions, with per-kind replay still idempotent.
2026-09-26 20:29:49 +10:00
sam c74b89b13f test(controlplane): refuse a resolved provider snapshot at the seed
archie-core-1000's failure mode was silent persistence: provider-settings seeded
from an already-resolved snapshot stored the process-local ARCHIE_PROVIDER_<hash>
env name with no api_key_ref, and once the resource outranked the file no later
boot had a reference left to resolve. The producer was fixed in 2485e671 and the
stored shape is refused in 3443e6b3, but nothing pinned the seed path's own
refusal: ImportConfig over a resolved snapshot must skip the kind, store nothing,
and report why -- the State Store still starts and the file document stays in
force -- rather than wedging on it or storing it.

Pin that contract, and prove the refusal comes from the guard: with
rejectBootDerivedProviderEnv removed the test fails with 'provider-settings was
stored from a resolved snapshot'.
2026-09-26 20:25:05 +10:00
sam 524c9d8cb5 test(scheduling): pin the schedule interval's duration contract
scheduling.Duration landed in 84450ac9 (archie-core-1169) with no test over the
JSON contract the schedules document and the jobs table both depend on: the
interval marshals the human string time.ParseDuration reads, unmarshals both
that string and the nanosecond count every schedule stored before the shape
existed carries, and refuses a non-duration string with ErrInvalidSpec. Pin it,
because a regression here fails silently in the worst direction -- the document
writes a count and an operator is back to typing 1800000000000 for one hour.
2026-09-26 20:18:41 +10:00
sam b381ff52b3 fix(control-plane): give the container document its own shape
container-runtime-policies seeded and validated config.ContainerConfig directly,
the last resource in the registry whose document was the internal config struct
(archie-core-1171). encoding/json fell back to that struct's Go field names --
Image, MaxConcurrency, LegacyEnabled, Profiles -- the only Go-cased keys the Web
UI still rendered, and case-insensitive matching let a lowercase misspelling
("maxconcurrency") decode as the field while "max_concurrency" was read back
half-applied.

Define the document here instead: containerRuntimePolicies with explicit
snake_case tags and agentProfile at the profile nesting level, the same shape
092194cd gave the channel document. The two durations stay config.Duration,
which already writes the string the file accepts; this leak was the keys only.
The mapping to config.ContainerConfig is one function the validator and the
runtime layering share.

Reading a legacy document is deliberate, so it stays strict about everything
else: DecodeRenamingLegacyKeys renames the seven Go-cased keys an existing store
holds (and the four at the profile level), drops LegacyEnabled -- the removed
containers.enabled switch, which had no runtime consumer and no current
counterpart -- and keeps DisallowUnknownFields, because swallowing an unknown
key decodes as unset and an unset image is a broken worker. A misspelling is
refused.

normalizeContainerPolicies, through the Definition.Normalize hook the channel
document already uses, rewrites a legacy document into the current shape on
write, so a stored document converges the next time it is saved. The layering
keeps its profile semantics -- a document that omits profiles (one stored before
they existed) inherits the file's, one that carries them replaces the file's --
and now checks both spellings of the key.

docs/development/control-plane.md's 'adding a database-owned field' still
instructed the untagged practice this class of issue removes; it now names the
wire-shape rule, the legacy-key tolerance and the Normalize hook instead.
2026-09-26 20:18:38 +10:00
sam bad7463a50 test(config): cover stored provider secret resolution 2026-09-26 20:15:32 +10:00
sam 0254fdaf8d fix(archied): extract accessDeps to bring stateStoreDeps under the complexity limit
The equivalent fix was made in the crew/qc8p worktree but never committed there before the merge, so the merge silently carried the pre-fix version through despite a clean, no-conflict merge. Re-applied directly on main; task check clean.
2026-09-26 15:30:19 +10:00
sam affbcde47e Merge crew/qc8p: archie-core-qc8p 2026-09-26 15:27:12 +10:00
sam 2c97eee95d test: loosen order- and count-pinned completeness checks, reference defaultContainerImage
Readiness probe test asserts each subsystem registers exactly once instead of
an ordered five-item list. Config descriptor test drops the exact-count check
and its stale settings.js rationale. Container defaults test references the
production constant.

Refs #1229 #1231 #1232
2026-09-26 15:25:02 +10:00
sam e6364a6e15 chore(notice): attribute the cedar-go dependency (archie-core-qc8p) 2026-09-26 15:23:32 +10:00
sam 2cb5ae3418 feat(access): archied access reset --org/--instance on the State Store host (archie-core-qc8p) 2026-09-26 15:23:32 +10:00
sam 020c6fb9d9 feat(access): the chain at the dashboard, API and dispatch; denial records and health surfacing (archie-core-qc8p) 2026-09-26 15:23:32 +10:00
sam 77a41ecfd9 feat(access): the chain at the dashboard, API and dispatch, with denial records and health reporting (archie-core-qc8p) 2026-09-26 15:23:32 +10:00
sam c41e2d76ad feat(access): policy, denial and principal persistence on the State Store contract (archie-core-qc8p) 2026-09-26 15:23:32 +10:00
sam c6d880a8f8 feat(access): cedar-go engine for the policy chain, shipped roles and cross-org forbid (archie-core-qc8p) 2026-09-26 15:22:01 +10:00
sam cd368851d1 feat(access): policy schema, actions, request assembly and Authorizer contract (archie-core-qc8p) 2026-09-26 15:22:01 +10:00
sam 8d7dae7d54 docs(claude): dev-docs pages are mandatory pre-reading, not colour, and carry into subagent prompts 2026-09-26 14:50:58 +10:00
sam bb6e8a648c docs(development): record today's two merge-surfaced bugs and the go-test-cache trap 2026-09-26 14:45:41 +10:00
sam be5174a660 docs(claude): sharpen the test-quality bar against config/DB fixtures and shipped test helpers 2026-09-26 14:40:10 +10:00
sam bcfe0f71c4 fix(workflow): sort the call-cycle walk's DFS roots for a deterministic report
refuseCallCycles iterated calls (a map) in range order to pick its DFS root, so the same a<->b cycle reported as 'a -> b -> a' on one run and 'b -> a -> b' on the next -- flaky in CI (TestValidateCollectionWorkflowCalls/indirect_cycle failed ~40% of local runs) and a genuinely unstable error message for an operator reading it twice. Sort the roots.
2026-09-26 14:05:39 +10:00
sam 5612b2c853 fix(agentworker): give the workflow-call test its own NATS server instead of the ambient default
TestExecuteTaskRequestStartsWorkflowCallCallee built its transport with no URL, so it silently fell through to nats.DefaultURL (127.0.0.1:4222). It passed locally against a developer's own NATS server and failed in CI with no servers available for connection. Start an ephemeral server per the sibling test's pattern and dial it explicitly.
2026-09-26 13:56:16 +10:00
sam 4464cc41cc Merge crew/t2db-41: archie-core-t2db.41 2026-09-26 13:39:51 +10:00
sam beb54f7f6c fix(staterpc): drive the workflow-call conformance test's callee through running before completed 2026-09-26 13:39:25 +10:00
sam 93906f8fae style(workflow): adopt gate formatting simplification in the workflow.call tests 2026-09-26 13:36:41 +10:00
sam 70e436377c chore(docs): regenerate dev-docs index for workflow-calls 2026-09-26 13:36:41 +10:00
sam 5e8ab5cde9 docs(prds): de-date the org-enablement sentence in workflow-calls 2026-09-26 13:36:41 +10:00
sam f8c394dd80 test(agentworker): workflow.call callee start through the real State Store gRPC path (archie-core-t2db.41) 2026-09-26 13:36:41 +10:00
sam 08deeb57a7 docs(prds): record the workflow-caller pair and grant widening in the State Store contract (archie-core-t2db.41) 2026-09-26 13:36:41 +10:00
sam a3c0c76615 feat(store): workflow.call contract — callee enqueue, parent-scoped status read, task-scoped grant widening (archie-core-t2db.41) 2026-09-26 13:36:41 +10:00
sam 208a1a9c38 feat(workflow): workflow.call step with save-time input and cycle checks (archie-core-t2db.41) 2026-09-26 13:36:41 +10:00
sam 5c30e971f1 docs(prds): decisive design for workflow.call runtime and save-time checks (archie-core-t2db.41) 2026-09-26 13:36:40 +10:00
sam b7001ab898 Merge crew/qdbo-5: archie-core-qdbo.5 2026-09-26 13:35:04 +10:00
sam 49e61224a8 feat(controlplane): re-layer model, repository, scheduling and provider settings live on a watch (archie-core-zfb0.1)
Flip provider-settings, model-role-assignments, repository-policies and
scheduling-policy to ApplyMode live, and give the daemon a watch per kind
(keepWatch's reconnection and backoff rules): on a new stored version the
watch re-runs boot.runtimeConfig over the published snapshot and republishes
through config.Holder, keeping last-known-good when the layered document
fails to validate and reporting through apply status.

The stored model-role assignments now own cfg.Models entirely (the map
merge would resurrect a role the store deleted on every re-layering), and a
provider or role update rebuilds the gateway chat runtime's provider set
(swapped runtime, read per turn) and the chat model manager's offered list.
2026-09-26 13:28:01 +10:00
sam 71d98fe9b4 feat(statestore): enforce the execution transition table and map ErrIllegalTransition over the wire
Every caller-supplied status write (Transition, ParkTask, Requeue,
RetryTask, BeginRemediation) now checks the shared taskstate transition
table inside the write's transaction, after the from guard decides
staleness. An off-table pair returns the new storecontract.ErrIllegalTransition,
carried over gRPC through mapError/unmapError on (FailedPrecondition,
"illegal transition"). Claim, BeginRemediation and RecoverStale pin
their pairs in SQL; a test pins those pairs to the table so a query
edit cannot move a write off it. Requeue/RetryTask share one guarded
requeue body. (archie-core-qdbo.5)
2026-09-26 13:13:20 +10:00
sam e7e3ea9d49 fix(postgres): renumber org migrations past 0012, already taken by installed-packages 2026-09-26 11:44:21 +10:00
sam ba23fab742 feat(org): carry task org across the State Store gRPC contract (archie-core-e5fn) 2026-09-26 11:43:10 +10:00
sam ee5339eb85 feat(org): idempotency key org/identity/owner/repo/number in one place, task uniqueness to match (archie-core-e5fn) 2026-09-26 11:43:10 +10:00
sam 16c52c1a16 feat(org): run the resumable default org upgrade before the State Store serves (archie-core-e5fn) 2026-09-26 11:43:10 +10:00
sam 4f5d135fe3 feat(org): Postgres tenant tables, org-aware store and resumable default upgrade (archie-core-e5fn) 2026-09-26 11:43:10 +10:00
sam 75f409141a feat(org): tenant boundary domain package — orgs, workspaces, memberships, agent assignments (archie-core-e5fn) 2026-09-26 11:43:10 +10:00