Commit Graph
16 Commits
Author SHA1 Message Date
sam 416df69b00 feat(egress): credential resolver backed by declared-and-granted run credentials
docs/prds/external-agent-harness.md (Credentials) and orgs-and-access.md:
egress.Resolver had no production implementation, and no run carried a
credential naming the org secrets its identity is granted, so a Kit's
required credential@1 was refused at the proxy and no harness run
authenticated (archie-core-egkf.11).

Two things, as scoped (this does not attempt #1026's full run-credential
consolidation replacing WorktreeGrantIssuer/StateStoreGrantIssuer -- that
PRD item is real, separate, and still open; this is deliberately narrower):

1. A run credential naming granted secrets: config.CredentialBinding maps a
   Kit credential@1 service to a secret; config.Config/IdentityConfig gain
   Org and GrantedCredentials (a per-identity allowlist of service names,
   reusing the existing per-identity config-override pattern DiffCapLines
   already uses, not a new Cedar/Authorizer integration -- KindSecret policy
   objects exist in internal/domain/access but assembling a full Principal
   is #1026's job, not a two-line addition here). config.ContainerConfig.
   BoundCredentials computes the declared (Kit) ∩ granted (identity) ∩
   org-matched intersection Sam's own rule requires: neither side widens
   the other, proven by a mutation-checked test flipping each of the three
   independently.

2. egress.GrantResolver: the production Resolver, holding only what a
   caller explicitly Grant-ed one run, replaced wholesale (not merged) on
   a repeated Grant, gone entirely on RevokeGrant. kitrun.Launcher.Launch
   computes the intersection per launch and resolves real values through
   the existing secret registry (SecretResolver), Grants it before the
   proxy session opens, and Release/error-path rollback (a new shared
   Launcher.release) always revokes it.

Course-corrected mid-implementation (flagged by Sam): credential bindings
first landed as a boot-time config.ContainerConfig.Credentials field
captured once into Launcher.Bindings -- exactly the "parses but is not a
resource" trap CLAUDE.md warns about, and inconsistent with agent-profiles
(#1207) making a Kit profile apply without a restart. Rebuilt as its own
live control-plane resource, CredentialBindingsKind ("credential-bindings"),
following the agent-profiles/container-runtime-policies split precisely:
config.ContainerConfig.Credentials is json:"-", [containers].credentials
is only its seed, and kitrun.Launcher now holds *config.Holder (not a
captured slice) and reads Containers.Credentials fresh on every Launch.
Org/GrantedCredentials were already read fresh per dispatch (d.configFor
inside runKitTask, never cached), so no equivalent fix was needed there.

Deliberately not built: identities and grants as their own State Store
resource (orgs-and-access.md's "Org owns ... identities and their grants").
Identities are entirely config.toml-defined today with no control-plane
resource at all; moving GrantedCredentials alone into one would put half
an identity in two places. That migration is identities' own scope, not
credential bindings'.
2026-09-27 06:14:22 +10:00
sam cc6d606de4 feat(controlplane): agent profiles as their own live control-plane resource
docs/prds/external-agent-harness.md ("Selection"): a Kit profile is an org
resource that applies without a restart. Profiles were bundled inside
container-runtime-policies (ApplyMode: restart-required), which is correct
for Image/MaxConcurrency/Network but not for Profiles, which nothing
startup-builds: daemon.configFor reads config.Holder fresh per dispatch
(pinTaskProfile -> Containers.Profile).

Split Profiles into its own resource, AgentProfileKind ("agent-profiles"),
ApplyMode: live, seeded from [containers.profiles] and validated with the
same rules the file document uses (config.ValidateAgentProfiles, factored
out of ContainerConfig.ValidateProfiles so the two cannot disagree).
config.ContainerConfig.Profiles is now json:"-" so container-runtime-policies
never carries it (one authoritative resource per field, not two), and
runtime_config.go layers the new resource in by wholesale map replacement
(matching ModelRoleAssignmentsKind's precedent) rather than merging into the
file's map. Added Profiles to reload.go's Containers reloadable-subfield set
to match this.

Deliberately NOT built here: true per-org multi-tenant storage for this
resource (docs/prds/orgs-and-access.md places agent profiles under org
ownership, granted to workspaces). No existing control-plane resource kind
in this registry does real per-org storage yet either -- schedules,
workflow settings, etc. are all single instance-wide documents -- so
agent-profiles follows that same established, tested pattern rather than
inventing org-scoped storage unilaterally for this one resource. Org/
workspace grants over agent-profiles are a follow-on, not a redo: nothing
about this resource's shape needs to change to add that scoping later.
2026-09-27 05:25:38 +10:00
sam d1cba5c1ab fix(control-plane): give the container document its own shape
container-runtime-policies seeded and validated config.ContainerConfig directly,
the last resource in the registry whose document was the internal config struct
(archie-core-1171). encoding/json fell back to that struct's Go field names --
Image, MaxConcurrency, LegacyEnabled, Profiles -- the only Go-cased keys the Web
UI still rendered, and case-insensitive matching let a lowercase misspelling
("maxconcurrency") decode as the field while "max_concurrency" was read back
half-applied.

Define the document here instead: containerRuntimePolicies with explicit
snake_case tags and agentProfile at the profile nesting level, the same shape
267d21a2 gave the channel document. The two durations stay config.Duration,
which already writes the string the file accepts; this leak was the keys only.
The mapping to config.ContainerConfig is one function the validator and the
runtime layering share.

Reading a legacy document is deliberate, so it stays strict about everything
else: DecodeRenamingLegacyKeys renames the seven Go-cased keys an existing store
holds (and the four at the profile level), drops LegacyEnabled -- the removed
containers.enabled switch, which had no runtime consumer and no current
counterpart -- and keeps DisallowUnknownFields, because swallowing an unknown
key decodes as unset and an unset image is a broken worker. A misspelling is
refused.

normalizeContainerPolicies, through the Definition.Normalize hook the channel
document already uses, rewrites a legacy document into the current shape on
write, so a stored document converges the next time it is saved. The layering
keeps its profile semantics -- a document that omits profiles (one stored before
they existed) inherits the file's, one that carries them replaces the file's --
and now checks both spellings of the key.

docs/development/control-plane.md's 'adding a database-owned field' still
instructed the untagged practice this class of issue removes; it now names the
wire-shape rule, the legacy-key tolerance and the Normalize hook instead.
2026-09-26 20:18:38 +10:00
sam 3186184964 feat(controlplane): re-layer model, repository, scheduling and provider settings live on a watch (archie-core-zfb0.1)
Flip provider-settings, model-role-assignments, repository-policies and
scheduling-policy to ApplyMode live, and give the daemon a watch per kind
(keepWatch's reconnection and backoff rules): on a new stored version the
watch re-runs boot.runtimeConfig over the published snapshot and republishes
through config.Holder, keeping last-known-good when the layered document
fails to validate and reporting through apply status.

The stored model-role assignments now own cfg.Models entirely (the map
merge would resurrect a role the store deleted on every re-layering), and a
provider or role update rebuilds the gateway chat runtime's provider set
(swapped runtime, read per turn) and the chat model manager's offered list.
2026-09-26 13:28:01 +10:00
sam 093e677ad8 fix(controlplane): reject stored boot-derived provider keys 2026-09-26 03:44:40 +10:00
sam 21f333a13e fix(controlplane): stored container policies replace file profiles; add profile image and workflow examples
A stored container-runtime-policies document that carries Profiles now
replaces the file's profiles instead of merging into them, so a profile
deleted in the dashboard stays deleted; one stored before profiles existed
still inherits the file's. Profiles drop their json tags so the stored
document keys them like its other fields.

Adds examples/profiles/Dockerfile.agent-net (agent image plus network
lookup tools) with a build-only agent-net compose service,
examples/workflows/firewall-investigate.yaml with a test that parses every
example against the registered step vocabulary, and names the postgres
service in the compose header.
2026-09-24 20:58:08 +10:00
sam ae552db7e0 refactor(store)!: remove the SQLite task store, the PocketBase event store and the SQLite session store
The State Store, event capture and Gateway conversations already serve from
PostgreSQL. This deletes internal/store and internal/infrastructure/edastore,
the SQLite gateway session store and openProductionTaskStore, and the -db
recovery path on a legacy task file; backup, restore, validate and rollback
work on the database database_url names.

- The binding-secret cipher moves to internal/infrastructure/bindingcipher and
  ToolCall to internal/infrastructure/postgres.
- NewSessionStoreMemory is now a pure in-memory store that passes the shared
  SessionStore conformance suite.
- Tests run against PostgreSQL through internal/infrastructure/postgres/pgstore;
  legacyimport/legacyfixture writes the legacy files the import and the boot
  gate read.
- The cron dialect is owned in internal/domain/scheduling, so
  github.com/pocketbase/pocketbase and pocketbase/dbx leave go.mod.
  modernc.org/sqlite stays for the one-time import (legacyread).
- The UI and Messaging architecture gates ban the Postgres and legacy readers
  in place of internal/store.
2026-09-24 14:43:50 +10:00
sam 4424071baf refactor(messaging): sever the Messaging Service from the store and workflow runtime
The PRD's deletion gate for the Messaging Service could not pass: go list -deps
./cmd/archie-messaging linked modernc.org/sqlite, internal/store,
internal/domain/workflow and internal/agentexec -- all through one import,
archiemessaging/run.go -> internal/app/controlplane, from which the service used
three symbols. The decision was to sever rather than amend the gate, because a
boundary you whitelist around is no longer a boundary (archie-core-1ng1, recorded
in docs/architecture/migration-decisions.md).

Measured before touching anything: internal/app/controlplane was the ONLY direct
import of the service that dragged those runtimes -- every other one was clean --
so severing this one dependency is sufficient for the gate.

Moved into the new internal/infrastructure/controlplanerpc: the generic resource
client (NewRPCClient, Query), the RuntimeChatConfig projection, the reader
interface (now the exported ResourceReader), the wire sentinels, the status
mapping (now ClientError), and the channel-settings document with its decode --
ChannelSettings, TelegramSettings, WebhookChannelSettings, EmailSettings,
RateLimitSettings, ChannelDuration, DecodeRenamingLegacyKeys.

internal/app/controlplane keeps the store-backed server, the Definition registry,
every kind's seed/validator/Normalize, the schema derivation, and its
resource-specific reads and watches. Its Client embeds the new one, so the read
path has one implementation while the workflow, persona and cronstore reads stay
with the processes that already link those engines. The documents are type-aliased
back into this package so the store-backed side reads them by the names it has
always used: one definition, not a copy on each side of the boundary.

The gate is now enforced by cmd/archie-messaging/architecture_test.go, measuring
the linked binary against the PRD's banned list, with one exception checked in
both directions: internal/domain/workflow/task, the Task status and view
vocabulary whose own dependency set holds none of the banned runtimes. The UI gate
carries the same entry for the same reason, and the fleet skill names it as the
precedent.

Verified: task check exit 0. The gate was red-proofed twice -- banning
internal/channels fails it, and removing the workflow/task exception fails it --
so neither the loop nor the exception is decorative.
2026-09-22 07:26:55 +10:00
sam 267d21a292 fix(controlplane): give the channel document its own shape
The channel-settings resource embedded internal/config types directly, so the
document the Web UI edits was a JSON projection of TOML-tagged structs:
encoding/json fell back to Go field names, and rate_limit.window was a bare
time.Duration. An operator saw "Window" over a field expecting a nanosecond
count -- 60000000000 for the window = "1m" the file example writes -- and
"ListenAddr" over the email addresses, while every sibling resource in the same
registry is snake_case.

Define the shape here instead: emailSettings and rateLimitSettings with explicit
tags, and channelDuration, which writes the string the file accepts and reads
that, a nanosecond count, and the Go-cased keys an existing store holds.

Reading a legacy document is deliberate, so it stays strict about everything
else. decodeRenamingLegacyKeys renames the four known legacy keys and keeps
DisallowUnknownFields, because swallowing an unknown key decodes as "unset", and
an unset rate limit is off -- the failure this shape exists to end. A
misspelling is refused.

normalizeChannels, through the Definition.Normalize hook schedules already uses,
rewrites a legacy document into the current shape on write, so a stored document
converges the next time it is saved. Reads tolerate it until then, so no
deployment has to re-save before the document can be read.

Verified: the controlplane, messaging and configuration suites, archied (the
loadRuntimeConfig consumer), go vet, go build ./..., golangci-lint run ./..., and
docs:check. The full task check is the merge train's, on this tip.

The same audit found container-runtime-policies embedding config.ContainerConfig
(seven fields, no json tags); tracked separately rather than folded in here.
2026-09-22 03:43:46 +10:00
sam 477b3738a3 fix(controlplane): carry parallel_tool_calls through the tool-settings projection
mcpServerSettings mirrors config.MCPServer field by field, and
runtimeToolConfigFrom rebuilds cfg.Tools from that mirror wholesale. The
parallel-tool-calls flag had no home in it, so on any deployment with a
control plane the file document's parallel_tool_calls = true was dropped and
every MCP server stayed serialized. The field is now carried through both
conversions -- the seed that writes the resource and the layering that reads
it back -- and the resource accepts it, so an operator can set it there too.

The field-by-field mirror is what let the field fall through, so the test
populates every MCPServer field by reflection and requires each one to be
projected and handed back, with an explicit allowlist for the fields the
projection deliberately leaves to the file document.
2026-09-22 01:57:58 +10:00
sam 2eba3487a8 fix(controlplane): make the scheduling policy carry the label its trigger pairs with
The scheduling-policy resource carried the dispatch trigger but not the
trigger-matching label, so validateScheduling could judge only half of
the pairing the file layer's validateDispatch enforces on the effective
document (a label-requiring trigger with no label matches every open
issue -- the GH#445 guard). The write path therefore stored trigger
label/either with no label whenever the file's own label was unset, and
boot then refused the layered document: a value the store blessed,
archied unstartable, the operator left with the offline rollback as the
only remedy (archie-core-7pyj).

The resource now carries the label: the seed copies the file document's
label, RuntimeConfig layers it with the trigger (a policy stored before
the field existed decodes nil and leaves the file's label in force, the
same absence-means-inherited shape a kind with no stored value has),
and validateScheduling refuses a label-requiring trigger with no label,
the pairing judged on the data the store actually holds. Once the store
carries a label it owns it, so a file that later loses the label can no
longer break a stored policy that depends on one. The parity test
gains both directions, and a legacy policy without the field is pinned
as an absence, not an erasure.
2026-09-22 00:32:30 +10:00
sam da7021c674 fix(controlplane): a kind with no stored value leaves the file's value in force
The fix-closure review found the previous pass moved the failure instead of
ending it. A seed the resource validator refuses leaves the kind ABSENT, and
RuntimeConfig treated an absent kind as fatal, so archied and the Gateway
refused to start with "control-plane resource not found" -- a database-named
error that editing config.toml cannot clear. Reproduced before the fix: every
kind in the new table failed that way.

Two changes, one on each side of the seam:

- Client.RuntimeConfig (and RuntimeChatConfig) now treat a kind with no stored
  value as "the file document's value stays in effect", which is what the
  skip-and-report comment already claimed. The version map records no version
  for such a kind, so nothing is reported as applied. A value the file layer
  also refuses still fails closed, at configuration.Validate, with a message
  naming the file's own field -- and the file is the fix.

- Which repository lists are valid can no longer differ between the layers.
  configuration.ValidateRepositories is now the single rule set (owner and name
  present, no duplicates, compilable test glob) and the repository-policies
  resource validator delegates to it. The file layer had no duplicate rule
  while the resource layer did, so a duplicate [[repos]] entry passed the loader
  and then vanished into a skipped seed.

Also, from the same review: the scheduling validator's "0s" carve-out is gone,
so a zero poll interval is refused at write time as configuration.Validate
refuses it (there is no defaulting behind a stored value); applyOverlayFile's
doc now says which overlay forms fold (a conf.d extra does not, and nothing
reads cfg.Extra); and plugin-settings' no-op validator is a named function with
the reason it accepts everything.
2026-09-21 22:12:32 +10:00
sam d3d6aedffb fix(statestore): read a kind the store does not hold as its seed, and stop validate touching the daemon's log
Four defects from the re-run contract review, all in the recovery path.

A store missing a control-plane kind was refused while a store with no
resources table at all was accepted -- two stances inside validateStore for
states the daemon cannot tell apart. The State Store seeds every kind from
its own config before it serves (RunStateStore -> ImportConfig), so a kind
the file does not hold yet is one archied reads the seed for: an upgrade that
adds a kind, and a validate before the State Store has started again, exited
1 with "read provider-settings: resource not found" for a store the daemon
boots on fine. StoredRuntimeConfig now reads a missing kind as the seed the
State Store would write for it (Definition.seededValue, shared with
ImportConfig so the two cannot drift), and the missing-table branch is the
same stance taken to its extreme.

defaultConfigPath() derived the daemon's config with os.UserConfigDir, which
disagrees with archied's configHome on darwin and rejects a relative
XDG_CONFIG_HOME outright -- leaving validate with an empty path and no
verdict, on a host where archied boots from relative-dir/archie/config.toml.
It is now the daemon's own rule, exported as archied.DefaultConfigPath and
used by the daemon's own -config default too, with a test pinning the
equality.

-h exited 2, the code documented as "the command line was wrong", while the
serve path in the same binary exits 0 and main.go advertises -h for these
subcommands. ErrHelp now returns 0.

validate resolved the boot config through loadConfig -> setupLogging, which
created cfg.Log.File's directory, opened the log, and wrote a line stamped
component="daemon" -- a diagnostic editing the deployment it inspects, and
rotating a log the daemon is writing. boot.stderrLog keeps setupLogging on
the stderr logger for the recovery path: no feed, no task-log registry, no
durable copy.

Tests: TestRecoveryValidateRunsTheGateBootRuns now asserts the unseeded and
the missing-kind stores are accepted (both exited 1 before), while the
bogus-policy case still refuses; TestRecoveryHelpExitsZero,
TestRecoveryValidateLeavesTheDaemonLogAlone and
TestRecoveryDefaultConfigFollowsTheDaemonsConfigHome are new. Each was run
red first, and the seed fallback and stderrLog were mutation-checked.
2026-09-21 22:04:42 +10:00
sam afde5890e0 fix(statestore): make validate run boot's gate, and stop backup replacing the database
Two P1 findings from review, both about a claim outrunning the code.

validate answered a different question from boot's. Boot refuses on
configuration.Validate over the file config with every stored resource
layered onto it (internal/app/archied/control_plane.go:27), which is a
stricter check than the write path's per-resource Decode: validateScheduling
never inspects dispatch.trigger and explicitly permits "0s", while boot
rejects an unknown trigger and a non-positive poll interval. A store holding
such a value printed "is a valid store ... N stored resources validate" and
exited 0 while archied exited 1 on it -- a false green pushing an operator
toward the blunt restore instead of the surgical rollback.

validate now runs both layers and reports both, in boot's order:
store.ValidateFile, then the write path's ValidateStored, then the config
gate. The layering itself is now one implementation: controlplane's
runtimeConfigFrom takes a resourceReader, which the gRPC client and a new
Server.StoredRuntimeConfig (reading its own store) both provide, so the
offline check cannot drift from the daemon's. validate takes -config /
-config-overlay (default: the path the serve subcommand uses), because the
question is about the stored values layered onto that document.

backup had no guard against -out naming the database. It renames its
snapshot over the target, so `backup -db F -out F` replaced a database a
serving State Store may still have open and orphaned the WAL of the file it
just unlinked -- the state Restore refuses up front and maintenance.go's own
comment calls unrecoverable. backup now refuses with the same message
Restore uses, which is what makes its exemption from the ownership lock
sound.

Claims corrected where they ran ahead of the code: ValidateStored now says it
is the write path's check and not boot's, the Restore comment no longer
promises "never a mixture" for the sidecars it removes first, the usage text
advertises validate's -config, and the architecture table drops "must be
stopped" for validate (nothing enforces it; read-only opens skip the lock by
design) and states which two layers validate reports.

Tests: TestRecoveryValidateRunsTheGateBootRuns fails on the pre-fix code
("validate accepted a stored policy the daemon refuses to boot with") and
covers a store the State Store never seeded; TestRecoveryBackupRefuses-
TheDatabaseAsItsOwnSnapshot fails on the pre-fix code ("backup over the
database itself succeeded"). The validate-accept test now seeds the store
through ImportConfig and asserts the reported count is non-zero, and the
stored-resource case uses a value both gates refuse instead of one only the
write path does.
2026-09-21 22:04:42 +10:00
sam 48367151f3 feat(controlplane): report which settings version each process is running
Every process that layers database-owned settings now publishes the version it
applied, and re-stamps it every 30 seconds so the report time doubles as a
liveness signal. A record older than 90 seconds belongs to a process that has
stopped, which is the only way this surface can tell a live process from one
that applied a version and then died.

RuntimeConfig returned the merged config and discarded the versions it read,
so no caller could report what it had applied. It now returns them, and
archied and archie-gateway report each kind through boot.runtimeConfig at boot
and on SIGHUP. A validation failure reports every kind with the error, because
the layering is all-or-nothing, and leaves the recorded version alone so a
rejected edit does not erase the fact that an older one is live. The daemon
also reports workflow execution settings as the watch applies them, and
archie-messaging reports channel settings at startup.

Reporting is best-effort by construction: the configuration a report describes
has already been applied, so a failed write is logged and never returned. That
keeps it off the reload's error path, where it would have raised a banner
saying a reload failed when its config took effect.

applystatus.New refuses a process name Processes() does not list. The reader
renders known names, so an invented one would be invisible rather than wrong,
which is the same thing the page means by a process that never started.

Relates to archie-core-pskb.
2026-09-21 15:49:54 +10:00
sam 02720c8e81 feat(controlplane): runtime administration over the State Store's gRPC server
The Web UI and messaging channels now read and write Archie's runtime
settings through one versioned, audited control-plane API hosted on the
State Store's existing gRPC server: catalog, query, command, watch. Each
feature owns its resource, validation and commands; storage only writes
versions and audit records.

Workflows move with it. Definitions become portable YAML compiled through
a closed, typed step registry and stored as a control-plane resource, so
the yaegi interpretation paths (internal/gate, wfeval, wfextract,
gateeval, gateextract, skillbuild) are deleted rather than carried.

The UI process reads definitions as data: the JSON projection lives in
internal/domain/workflow/task so a dashboard decodes the stored
collection without linking the workflow engine, which cmd/archie-ui's
deletion gate forbids. Shipped definitions ride the catalog descriptor
rather than a bespoke REST route, keeping the API to its four operations.

archie-messaging gains -state-store-target/-state-store-token and reads
channel settings from the database, failing closed when the State Store
is unreachable: the database is authoritative for those settings.
2026-09-21 08:17:42 +10:00