Commit Graph
2710 Commits
Author SHA1 Message Date
sam 15c9c0efc4 Merge crew/claude-997: 429 for rate-limited webhooks, agent profiles as a live resource, per-stage profile via workflow.call, harness needs enforced at dispatch 2026-09-27 05:27:09 +10:00
sam abb9f8a04e feat(harness): declare and enforce a workflow's harness needs at dispatch
docs/prds/external-agent-harness.md ("Contract"): workflow definition
validation must reject a stage whose needs the selected harness cannot
meet -- capture tools without an adapter, or a gate retry without a
resume verb. The step vocabulary had no static way to declare either.

Added task.WorkflowInterface.Needs{Captures, GateRetries}, parsed and
validated (GateRetries >= 0) the same way Inputs/Repository already are.
Only a Kit profile is checked against it: an image profile's built-in
agent loop always supports both, so Needs constrains only the harness path.

Two enforcement points, chosen for where each fact is actually known
without a network fetch at save time:
- Captures vs adapter is knowable from config alone (profile.Adapter, the
  agentexec.harnessAdapters table), so daemon.pinTaskProfile checks it
  right where it already resolves the profile (validateProfileMeetsNeeds),
  parking the task the same way an unconfigured profile already does.
- GateRetries vs the Kit's actual agent-sessions resume verb is only known
  once the Kit composition is fetched, which kitrun.Launcher.Launch
  already does -- so kit.ValidateNeeds runs there (new kitrun.Request.
  GateRetries field, threaded from the workflow's parsed Needs in
  daemon.runKitTask) rather than inventing an image-fetching workflow-save
  validator.

Deliberately NOT built: "a Kit profile named by no workflow is a
validation error" (external-agent-harness.md "Selection"). Read literally
("a Kit profile [that is] named by no workflow"), this is a
cross-resource check -- an agent-profiles entry unreferenced by anything
in workflow-definitions -- and no control-plane Definition validator can
see another resource today (Definition.Validate is a pure []byte->error
function by design, matching every existing resource kind). Building
that crossing is a real, separate piece of infrastructure with its own
ordering questions (does defining a Kit profile before its workflow now
fail?), not a two-line addition, so it is left for a follow-on rather
than bolted on here.

Also fixes a test bug caught by rerunning with a short -timeout per
instruction: TestRunKitTaskThreadsGateRetriesToTheLauncher's fake NATS
subscriber never responded, so runKitTask's call into runViaAgent blocked
forever waiting for a reply bounded by a zero (unconfigured) wall-clock
budget. Fixed by responding, matching the sibling test's pattern.
2026-09-27 05:25:39 +10:00
sam f12cb2478a docs(harness): pin per-stage profile override as workflow.call, not a mid-container switch
docs/prds/external-agent-harness.md's "Selection" says a stage may name a
different profile than its workflow's. Investigated before writing new
runtime code, per instruction not to build container-switching machinery:
a Kit task runs in one container provisioned for one profile (Container
layout), so a stage needing a different profile cannot be a mid-container
runner swap without inventing per-stage re-provisioning the PRD does not
ask for and Container layout argues against ("cost scales with the run,
not with how many agents it fans out to").

The requirement is already met by workflow.call plus per-task profile
resolution, both pre-existing: postgres.Store.StartCall stores the
callee's OWN Workflow name (not the caller's), and daemon.pinTaskProfile
resolves a task's profile purely from its own WorkflowDefinitionYAML.
Chaining these means a stage wanting a different profile is its own
callee workflow, started via workflow.call, exactly the way
event-automation.md already describes ("the callee runs as its own run
with its own agent profile").

No new production code, only a PRD clarification (dev-docs regenerated)
plus TestWorkflowCallGivesACalleeItsOwnProfile (daemon_test.go), which
closes the gap between two already-tested facts that nothing previously
stated together: TestWorkflowCallerConformance (staterpc) proves a
callee's Workflow field is independent of its caller's; this proves that
independence is what makes Kit-vs-image runner selection independent too,
composing a caller pinned to an image profile with a real StartCall-
created callee pinned to a Kit profile.
2026-09-27 05:25:39 +10:00
sam ed1325be77 feat(controlplane): agent profiles as their own live control-plane resource
docs/prds/external-agent-harness.md ("Selection"): a Kit profile is an org
resource that applies without a restart. Profiles were bundled inside
container-runtime-policies (ApplyMode: restart-required), which is correct
for Image/MaxConcurrency/Network but not for Profiles, which nothing
startup-builds: daemon.configFor reads config.Holder fresh per dispatch
(pinTaskProfile -> Containers.Profile).

Split Profiles into its own resource, AgentProfileKind ("agent-profiles"),
ApplyMode: live, seeded from [containers.profiles] and validated with the
same rules the file document uses (config.ValidateAgentProfiles, factored
out of ContainerConfig.ValidateProfiles so the two cannot disagree).
config.ContainerConfig.Profiles is now json:"-" so container-runtime-policies
never carries it (one authoritative resource per field, not two), and
runtime_config.go layers the new resource in by wholesale map replacement
(matching ModelRoleAssignmentsKind's precedent) rather than merging into the
file's map. Added Profiles to reload.go's Containers reloadable-subfield set
to match this.

Deliberately NOT built here: true per-org multi-tenant storage for this
resource (docs/prds/orgs-and-access.md places agent profiles under org
ownership, granted to workspaces). No existing control-plane resource kind
in this registry does real per-org storage yet either -- schedules,
workflow settings, etc. are all single instance-wide documents -- so
agent-profiles follows that same established, tested pattern rather than
inventing org-scoped storage unilaterally for this one resource. Org/
workspace grants over agent-profiles are a follow-on, not a redo: nothing
about this resource's shape needs to change to add that scoping later.
2026-09-27 05:25:38 +10:00
sam df389b7773 fix(webhook): reject rate-limited deliveries with 429 instead of chat prose or a false 202
Router.Route returned the chat-facing rate-limit prose with a nil error, so
the webhook handler had no way to tell a genuine dispatch outcome from a
rejection: DeliverTo "origin" echoed the prose as a 200 reply, and any
other route got a 202 for an event that was actually dropped before
reaching the LLM, so the source never retries it.

ChatReply now carries RateLimited, set by a new Router.RouteResult (Route
is now a thin wrapper over it) so chat channels that render Text to a
human are unaffected. The webhook handler checks it and answers 429,
matching the precedent internal/webhookguard.RateLimiter's other callers
already set.
2026-09-27 05:20:30 +10:00
sam 79fbd0b133 fix(archied): republish the config snapshot when live settings apply
applyWorkflowExecutionSettings swapped the running config with cfgHolder.Set,
so the dashboard's published projection kept the previous budgets until the
next reload. Route it through publishConfig like every other config change.

Fixes #1148
2026-09-27 05:09:24 +10:00
sam d90c2a6618 Merge crew/pi-dashboard-sweep: container document shape, systemd READY, capture route constant, contract tests 2026-09-27 05:04:57 +10:00
sam 8bcced0f47 Merge crew/codex-1184: provider secret resolution test, watch outage reporting, projection parity guard 2026-09-27 05:04:43 +10:00
sam 5a703e0eab test(controlplane): guard mirrored settings projections 2026-09-26 21:02:16 +10:00
sam 80c49242e8 fix(webui): mount the capture intake route from the receiver's constant
The bypass mux mounted captureintake's route from a local literal that
duplicated captureintake.Path, with a comment acknowledging the drift risk:
the kind of duplication that silently stops matching after a rename
(archie-core-1175). Reference the constant instead, so the receiver owns the
one spelling of the route and a rename there cannot unmount here.

The existing routing tests are the drift coverage: TestCaptureIntakeMountedOnBypassMux
drives Handler() with a POST to the real path, so a rename on either side that
stopped matching would fail it; no new test is needed. This gives internal/webui
its first infrastructure import -- the deleted literal existed so webui could
mount an http.Handler without depending on the receiver's concrete package, but
mounting by the owner's constant is exactly the boundary the intake cutover
drew: webui mounts and reads, captureintake owns the write path.
2026-09-26 20:44:40 +10:00
sam 26f0fc9838 fix(deploy): announce systemd READY from archie-gateway and archie-state-store
2fcc32d8 gave archied sd_notify: one READY=1 when the run loop starts serving.
RunGateway and RunStateStore serve contracts and health surfaces of their own --
the State Store even takes -ready-addr for a readiness HTTP listener -- but told
systemd nothing, so run as Type=notify either would hang until TimeoutStartSec
and be killed in a restart loop while perfectly healthy (archie-core-1174).

Both now announce at the moment each declares itself serving, the same notion
archied's runLoop announces beside healthSurface.markServing -- no second
definition of started. The Gateway announces from its serving tail
(serveGatewayListener, extracted from RunGateway), after the listener is bound;
the State Store announces after its optional surfaces have started, because a
readiness surface that fails to bind fails the boot and READY must not precede
it, and before serveStateStore, which begins accepting the moment it is called.
No unit file changes Type=notify for them; the helper already lives in package
archied beside both entry points, so no move was needed.

Tests follow the notify pattern the daemon's announcement already has: the
Gateway test drives its serving tail over a bound loopback listener with a
stub contract and judges READY against a real Snapshot RPC on the same socket;
the State Store test drives the real RunStateStore boot over a pgtest database
and waits for the datagram. Both failed red before the announcement existed --
no datagram within the window -- and pass after.
2026-09-26 20:39:56 +10:00
sam 488ed7a6ef fix(archied): report workflow settings watch outages 2026-09-26 20:38:48 +10:00
sam 8f4165256c test(store): a resource request ID is idempotent per kind
The per-kind request-ID dedup was fixed in e8f86622 for the SQLite store and
carried into the PostgreSQL rewrite (05ee5eff) from day one: ResourceByRequest
filters WHERE kind = $1 AND request_id = $2 and resource_history carries
UNIQUE(kind, request_id). What neither rewrite kept is the test that pins the
cross-kind half of the contract -- the deleted internal/store suite had one, and
the Postgres suite only covers same-kind replay.

Pin it, and prove the failure mode it guards against: with the kind filter
dropped from the query, a request ID reused across two kinds silently returns
the first kind's history row under the second kind's write -- wrong kind, wrong
value, no error (archie-core-1179's latent hazard). Restore the filter and both
kinds write their own revisions, with per-kind replay still idempotent.
2026-09-26 20:29:49 +10:00
sam af39a1bedb test(controlplane): refuse a resolved provider snapshot at the seed
archie-core-1000's failure mode was silent persistence: provider-settings seeded
from an already-resolved snapshot stored the process-local ARCHIE_PROVIDER_<hash>
env name with no api_key_ref, and once the resource outranked the file no later
boot had a reference left to resolve. The producer was fixed in b0ff54f6 and the
stored shape is refused in b5e4ee53, but nothing pinned the seed path's own
refusal: ImportConfig over a resolved snapshot must skip the kind, store nothing,
and report why -- the State Store still starts and the file document stays in
force -- rather than wedging on it or storing it.

Pin that contract, and prove the refusal comes from the guard: with
rejectBootDerivedProviderEnv removed the test fails with 'provider-settings was
stored from a resolved snapshot'.
2026-09-26 20:25:05 +10:00
sam b78722dc96 test(scheduling): pin the schedule interval's duration contract
scheduling.Duration landed in f29169c2 (archie-core-1169) with no test over the
JSON contract the schedules document and the jobs table both depend on: the
interval marshals the human string time.ParseDuration reads, unmarshals both
that string and the nanosecond count every schedule stored before the shape
existed carries, and refuses a non-duration string with ErrInvalidSpec. Pin it,
because a regression here fails silently in the worst direction -- the document
writes a count and an operator is back to typing 1800000000000 for one hour.
2026-09-26 20:18:41 +10:00
sam eb95be607d fix(control-plane): give the container document its own shape
container-runtime-policies seeded and validated config.ContainerConfig directly,
the last resource in the registry whose document was the internal config struct
(archie-core-1171). encoding/json fell back to that struct's Go field names --
Image, MaxConcurrency, LegacyEnabled, Profiles -- the only Go-cased keys the Web
UI still rendered, and case-insensitive matching let a lowercase misspelling
("maxconcurrency") decode as the field while "max_concurrency" was read back
half-applied.

Define the document here instead: containerRuntimePolicies with explicit
snake_case tags and agentProfile at the profile nesting level, the same shape
a45aebcf gave the channel document. The two durations stay config.Duration,
which already writes the string the file accepts; this leak was the keys only.
The mapping to config.ContainerConfig is one function the validator and the
runtime layering share.

Reading a legacy document is deliberate, so it stays strict about everything
else: DecodeRenamingLegacyKeys renames the seven Go-cased keys an existing store
holds (and the four at the profile level), drops LegacyEnabled -- the removed
containers.enabled switch, which had no runtime consumer and no current
counterpart -- and keeps DisallowUnknownFields, because swallowing an unknown
key decodes as unset and an unset image is a broken worker. A misspelling is
refused.

normalizeContainerPolicies, through the Definition.Normalize hook the channel
document already uses, rewrites a legacy document into the current shape on
write, so a stored document converges the next time it is saved. The layering
keeps its profile semantics -- a document that omits profiles (one stored before
they existed) inherits the file's, one that carries them replaces the file's --
and now checks both spellings of the key.

docs/development/control-plane.md's 'adding a database-owned field' still
instructed the untagged practice this class of issue removes; it now names the
wire-shape rule, the legacy-key tolerance and the Normalize hook instead.
2026-09-26 20:18:38 +10:00
sam 4c056d2dfb test(config): cover stored provider secret resolution 2026-09-26 20:15:32 +10:00
sam d7312143ee fix(archied): extract accessDeps to bring stateStoreDeps under the complexity limit
The equivalent fix was made in the crew/qc8p worktree but never committed there before the merge, so the merge silently carried the pre-fix version through despite a clean, no-conflict merge. Re-applied directly on main; task check clean.
2026-09-26 15:30:19 +10:00
sam 0219aee5ad Merge crew/qc8p: archie-core-qc8p 2026-09-26 15:27:12 +10:00
sam 5b9db382b4 test: loosen order- and count-pinned completeness checks, reference defaultContainerImage
Readiness probe test asserts each subsystem registers exactly once instead of
an ordered five-item list. Config descriptor test drops the exact-count check
and its stale settings.js rationale. Container defaults test references the
production constant.

Refs #1229 #1231 #1232
2026-09-26 15:25:02 +10:00
sam d425a8dd36 chore(notice): attribute the cedar-go dependency (archie-core-qc8p) 2026-09-26 15:23:32 +10:00
sam c8f9571448 feat(access): archied access reset --org/--instance on the State Store host (archie-core-qc8p) 2026-09-26 15:23:32 +10:00
sam 506416a406 feat(access): the chain at the dashboard, API and dispatch; denial records and health surfacing (archie-core-qc8p) 2026-09-26 15:23:32 +10:00
sam c2cb5ddfaf feat(access): the chain at the dashboard, API and dispatch, with denial records and health reporting (archie-core-qc8p) 2026-09-26 15:23:32 +10:00
sam 7c765dad9d feat(access): policy, denial and principal persistence on the State Store contract (archie-core-qc8p) 2026-09-26 15:23:32 +10:00
sam 7ecf357406 feat(access): cedar-go engine for the policy chain, shipped roles and cross-org forbid (archie-core-qc8p) 2026-09-26 15:22:01 +10:00
sam 219301a88e feat(access): policy schema, actions, request assembly and Authorizer contract (archie-core-qc8p) 2026-09-26 15:22:01 +10:00
sam e552de6ec3 docs(claude): dev-docs pages are mandatory pre-reading, not colour, and carry into subagent prompts 2026-09-26 14:50:58 +10:00
sam bcafff6941 docs(development): record today's two merge-surfaced bugs and the go-test-cache trap 2026-09-26 14:45:41 +10:00
sam 4cdc0dc7cb docs(claude): sharpen the test-quality bar against config/DB fixtures and shipped test helpers 2026-09-26 14:40:10 +10:00
sam e5627f470d fix(workflow): sort the call-cycle walk's DFS roots for a deterministic report
refuseCallCycles iterated calls (a map) in range order to pick its DFS root, so the same a<->b cycle reported as 'a -> b -> a' on one run and 'b -> a -> b' on the next -- flaky in CI (TestValidateCollectionWorkflowCalls/indirect_cycle failed ~40% of local runs) and a genuinely unstable error message for an operator reading it twice. Sort the roots.
2026-09-26 14:05:39 +10:00
sam 0c0d52a851 fix(agentworker): give the workflow-call test its own NATS server instead of the ambient default
TestExecuteTaskRequestStartsWorkflowCallCallee built its transport with no URL, so it silently fell through to nats.DefaultURL (127.0.0.1:4222). It passed locally against a developer's own NATS server and failed in CI with no servers available for connection. Start an ephemeral server per the sibling test's pattern and dial it explicitly.
2026-09-26 13:56:16 +10:00
sam fd9ebb9795 Merge crew/t2db-41: archie-core-t2db.41 2026-09-26 13:39:51 +10:00
sam 4cd60ad8ff fix(staterpc): drive the workflow-call conformance test's callee through running before completed 2026-09-26 13:39:25 +10:00
sam 26c199097d style(workflow): adopt gate formatting simplification in the workflow.call tests 2026-09-26 13:36:41 +10:00
sam d38ef5a4ec chore(docs): regenerate dev-docs index for workflow-calls 2026-09-26 13:36:41 +10:00
sam 211f6a5782 docs(prds): de-date the org-enablement sentence in workflow-calls 2026-09-26 13:36:41 +10:00
sam a7e7b95a85 test(agentworker): workflow.call callee start through the real State Store gRPC path (archie-core-t2db.41) 2026-09-26 13:36:41 +10:00
sam 262343539e docs(prds): record the workflow-caller pair and grant widening in the State Store contract (archie-core-t2db.41) 2026-09-26 13:36:41 +10:00
sam 484ea701fe feat(store): workflow.call contract — callee enqueue, parent-scoped status read, task-scoped grant widening (archie-core-t2db.41) 2026-09-26 13:36:41 +10:00
sam df58c518dc feat(workflow): workflow.call step with save-time input and cycle checks (archie-core-t2db.41) 2026-09-26 13:36:41 +10:00
sam 7b391b90df docs(prds): decisive design for workflow.call runtime and save-time checks (archie-core-t2db.41) 2026-09-26 13:36:40 +10:00
sam c90733e77b Merge crew/qdbo-5: archie-core-qdbo.5 2026-09-26 13:35:04 +10:00
sam 159519de6a feat(controlplane): re-layer model, repository, scheduling and provider settings live on a watch (archie-core-zfb0.1)
Flip provider-settings, model-role-assignments, repository-policies and
scheduling-policy to ApplyMode live, and give the daemon a watch per kind
(keepWatch's reconnection and backoff rules): on a new stored version the
watch re-runs boot.runtimeConfig over the published snapshot and republishes
through config.Holder, keeping last-known-good when the layered document
fails to validate and reporting through apply status.

The stored model-role assignments now own cfg.Models entirely (the map
merge would resurrect a role the store deleted on every re-layering), and a
provider or role update rebuilds the gateway chat runtime's provider set
(swapped runtime, read per turn) and the chat model manager's offered list.
2026-09-26 13:28:01 +10:00
sam a7023bb2c3 feat(statestore): enforce the execution transition table and map ErrIllegalTransition over the wire
Every caller-supplied status write (Transition, ParkTask, Requeue,
RetryTask, BeginRemediation) now checks the shared taskstate transition
table inside the write's transaction, after the from guard decides
staleness. An off-table pair returns the new storecontract.ErrIllegalTransition,
carried over gRPC through mapError/unmapError on (FailedPrecondition,
"illegal transition"). Claim, BeginRemediation and RecoverStale pin
their pairs in SQL; a test pins those pairs to the table so a query
edit cannot move a write off it. Requeue/RetryTask share one guarded
requeue body. (archie-core-qdbo.5)
2026-09-26 13:13:20 +10:00
sam fa19c459c7 fix(postgres): renumber org migrations past 0012, already taken by installed-packages 2026-09-26 11:44:21 +10:00
sam 44db11cdc4 feat(org): carry task org across the State Store gRPC contract (archie-core-e5fn) 2026-09-26 11:43:10 +10:00
sam e0e9a17bd8 feat(org): idempotency key org/identity/owner/repo/number in one place, task uniqueness to match (archie-core-e5fn) 2026-09-26 11:43:10 +10:00
sam fe4e2a2f6a feat(org): run the resumable default org upgrade before the State Store serves (archie-core-e5fn) 2026-09-26 11:43:10 +10:00
sam 47485d2751 feat(org): Postgres tenant tables, org-aware store and resumable default upgrade (archie-core-e5fn) 2026-09-26 11:43:10 +10:00