Commit Graph
179 Commits
Author SHA1 Message Date
sam a7d6cc8c9d refactor(messaging): sever the Messaging Service from the store and workflow runtime
The PRD's deletion gate for the Messaging Service could not pass: go list -deps
./cmd/archie-messaging linked modernc.org/sqlite, internal/store,
internal/domain/workflow and internal/agentexec -- all through one import,
archiemessaging/run.go -> internal/app/controlplane, from which the service used
three symbols. The decision was to sever rather than amend the gate, because a
boundary you whitelist around is no longer a boundary (archie-core-1ng1, recorded
in docs/architecture/migration-decisions.md).

Measured before touching anything: internal/app/controlplane was the ONLY direct
import of the service that dragged those runtimes -- every other one was clean --
so severing this one dependency is sufficient for the gate.

Moved into the new internal/infrastructure/controlplanerpc: the generic resource
client (NewRPCClient, Query), the RuntimeChatConfig projection, the reader
interface (now the exported ResourceReader), the wire sentinels, the status
mapping (now ClientError), and the channel-settings document with its decode --
ChannelSettings, TelegramSettings, WebhookChannelSettings, EmailSettings,
RateLimitSettings, ChannelDuration, DecodeRenamingLegacyKeys.

internal/app/controlplane keeps the store-backed server, the Definition registry,
every kind's seed/validator/Normalize, the schema derivation, and its
resource-specific reads and watches. Its Client embeds the new one, so the read
path has one implementation while the workflow, persona and cronstore reads stay
with the processes that already link those engines. The documents are type-aliased
back into this package so the store-backed side reads them by the names it has
always used: one definition, not a copy on each side of the boundary.

The gate is now enforced by cmd/archie-messaging/architecture_test.go, measuring
the linked binary against the PRD's banned list, with one exception checked in
both directions: internal/domain/workflow/task, the Task status and view
vocabulary whose own dependency set holds none of the banned runtimes. The UI gate
carries the same entry for the same reason, and the fleet skill names it as the
precedent.

Verified: task check exit 0. The gate was red-proofed twice -- banning
internal/channels fails it, and removing the workflow/task exception fails it --
so neither the loop nor the exception is decorative.
2026-09-22 07:26:55 +10:00
sam 7a8bb8e47b docs: drop one host's service names from port guidance
Five files named a specific host's service and the port it holds as the
reason a deployment might move the state store's listen address, including
config.example.toml, which ships to every operator. The guidance keeps its
point -- move the listen/target pair together when a port is already taken
on the host -- without naming a service the reader may not run, or have
heard of.

Also corrects the Taskfile's dev-stack summary, which still described the
state store as binding 127.0.0.1:9090. deployments/dev.toml pairs it with
9091, and has since the pair became configurable, so the comment described
a stack that does not start.
2026-09-22 04:09:16 +10:00
sam 0263bf4e0d fix(configuration): resolve the default config path in one place
Four binaries derived the default with os.UserConfigDir while the daemon
resolved it from its own config home. The two agree on Linux with an
absolute XDG_CONFIG_HOME and diverge everywhere else, so one deployment
booted from a different config file depending on which binary started.

Move the rule into the package that owns configuration resolution
(internal/infrastructure/configuration, which already owns xdgDataHome and
DefaultWorkDir) as DefaultConfigDir/DefaultConfigPath, and have every
-config default take it from there: archie-gateway, archie-messaging,
archie-state-store (serve and recovery) and archie-ui, plus the daemon's
own parseArgs, setup generator and drain-marker path.

Fence the claim at the source, since the resolved value is only observable
by starting a process: no source under cmd/ or internal/app/archied may
call os.UserConfigDir, and every -config registration must default to the
helper.
2026-09-22 01:57:49 +10:00
sam 9a45a35c27 fix(config): judge a database-owned setting at the layer that owns it
Review of the two previous commits found the split was incomplete in both
directions: the checks moved off the file load path were still enforced
anywhere the value could enter the system, and the fold reached only one of the
three overlay paths.

A stale database-owned TOML value could still stop archie-state-store. The
process seeds every control-plane resource from the same file config on a fresh
database, and the seed runs the resource's own validator; three of the five
values the loader now accepts aborted the boot there (verified by booting
RunStateStore before the fix). A seed that does not validate is now SKIPPED and
reported instead of fatal: the resource stays absent, so the file's value is the
one in effect and editing config.toml is still the fix -- the daemon and the
Gateway are where it fails closed, at boot.runtimeConfig.

The resource validators were weaker than configuration.Validate for exactly the
fields the split reclassified, so a value the file path used to refuse at load
became storable in the database, and then un-fixable by editing config.toml.
The trigger enum and the test-glob rule now come from the configuration package
rather than being restated, the container image is required there as it is in the
file, and a table test ties the two layers together value by value.

The directory form of -config-overlay kept the archie-core-e2e2 failure: it
decoded each overlay file straight into the populated config, so a map-valued
entry it only partly addressed still cleared the fields it did not name. Both
the overlay layer's main file and its feature files now fold, which is what
Loader.Dir's documented parity with Loader.Overlay has always claimed.

The overlay file's key report is derived from the apply's own view of the file as
well as the file decode's, because TOML matches keys case-insensitively and the
fold's yaml decode does not: a key only the file decode matched was consumed by
the report and dropped by the apply, a silent no-op that worked in the base file.

Tests: the state-store boot is asserted against the real process, the directory
overlay and the case-mismatched key each have one, and ImportConfig pins that a
refused seed is not stored.
2026-09-21 22:12:32 +10:00
sam be321d81bc fix(statestore): build the recovery control-plane server on the shared step vocabulary
After the rebase onto main, controlplane.NewServer takes the step vocabulary
it resolves definitions against (controlplane.ResourceStore, *workflow.Manager)
and returns (Server, error). Both offline call sites still passed the store
alone, so internal/app/archied stopped compiling -- and with it
cmd/archie-state-store, cmd/archied and cmd/archie-gateway.

validate and rollback now build their server through
openStateStoreControlPlane, the root helper the served path already uses, so
the offline checks resolve the same shared provider set
(internal/infrastructure/workflowsteps) as the process whose verdict they
predict, and the constructor's error is returned as a refusal rather than
dropped. No second vocabulary call site: the fwmp pin on
openStateStoreControlPlane covers these paths unchanged.

The test's seeding helper builds its server from the same shared set
(workflowsteps.NewManager) because it cannot call the unexported root helper;
an empty manager there would seed values the served path never validates
against. That is not merely a claim: with an empty manager the three
validate tests fail with `workflow "bootstrap" step 1: unknown type
"bootstrap.prepare"`, so they already pin that the offline path resolves a
live, shared vocabulary.
2026-09-21 22:07:41 +10:00
sam 08df95684c fix(statestore): read a kind the store does not hold as its seed, and stop validate touching the daemon's log
Four defects from the re-run contract review, all in the recovery path.

A store missing a control-plane kind was refused while a store with no
resources table at all was accepted -- two stances inside validateStore for
states the daemon cannot tell apart. The State Store seeds every kind from
its own config before it serves (RunStateStore -> ImportConfig), so a kind
the file does not hold yet is one archied reads the seed for: an upgrade that
adds a kind, and a validate before the State Store has started again, exited
1 with "read provider-settings: resource not found" for a store the daemon
boots on fine. StoredRuntimeConfig now reads a missing kind as the seed the
State Store would write for it (Definition.seededValue, shared with
ImportConfig so the two cannot drift), and the missing-table branch is the
same stance taken to its extreme.

defaultConfigPath() derived the daemon's config with os.UserConfigDir, which
disagrees with archied's configHome on darwin and rejects a relative
XDG_CONFIG_HOME outright -- leaving validate with an empty path and no
verdict, on a host where archied boots from relative-dir/archie/config.toml.
It is now the daemon's own rule, exported as archied.DefaultConfigPath and
used by the daemon's own -config default too, with a test pinning the
equality.

-h exited 2, the code documented as "the command line was wrong", while the
serve path in the same binary exits 0 and main.go advertises -h for these
subcommands. ErrHelp now returns 0.

validate resolved the boot config through loadConfig -> setupLogging, which
created cfg.Log.File's directory, opened the log, and wrote a line stamped
component="daemon" -- a diagnostic editing the deployment it inspects, and
rotating a log the daemon is writing. boot.stderrLog keeps setupLogging on
the stderr logger for the recovery path: no feed, no task-log registry, no
durable copy.

Tests: TestRecoveryValidateRunsTheGateBootRuns now asserts the unseeded and
the missing-kind stores are accepted (both exited 1 before), while the
bogus-policy case still refuses; TestRecoveryHelpExitsZero,
TestRecoveryValidateLeavesTheDaemonLogAlone and
TestRecoveryDefaultConfigFollowsTheDaemonsConfigHome are new. Each was run
red first, and the seed fallback and stderrLog were mutation-checked.
2026-09-21 22:04:42 +10:00
sam e7e5ddfba5 fix(statestore): make validate run boot's gate, and stop backup replacing the database
Two P1 findings from review, both about a claim outrunning the code.

validate answered a different question from boot's. Boot refuses on
configuration.Validate over the file config with every stored resource
layered onto it (internal/app/archied/control_plane.go:27), which is a
stricter check than the write path's per-resource Decode: validateScheduling
never inspects dispatch.trigger and explicitly permits "0s", while boot
rejects an unknown trigger and a non-positive poll interval. A store holding
such a value printed "is a valid store ... N stored resources validate" and
exited 0 while archied exited 1 on it -- a false green pushing an operator
toward the blunt restore instead of the surgical rollback.

validate now runs both layers and reports both, in boot's order:
store.ValidateFile, then the write path's ValidateStored, then the config
gate. The layering itself is now one implementation: controlplane's
runtimeConfigFrom takes a resourceReader, which the gRPC client and a new
Server.StoredRuntimeConfig (reading its own store) both provide, so the
offline check cannot drift from the daemon's. validate takes -config /
-config-overlay (default: the path the serve subcommand uses), because the
question is about the stored values layered onto that document.

backup had no guard against -out naming the database. It renames its
snapshot over the target, so `backup -db F -out F` replaced a database a
serving State Store may still have open and orphaned the WAL of the file it
just unlinked -- the state Restore refuses up front and maintenance.go's own
comment calls unrecoverable. backup now refuses with the same message
Restore uses, which is what makes its exemption from the ownership lock
sound.

Claims corrected where they ran ahead of the code: ValidateStored now says it
is the write path's check and not boot's, the Restore comment no longer
promises "never a mixture" for the sidecars it removes first, the usage text
advertises validate's -config, and the architecture table drops "must be
stopped" for validate (nothing enforces it; read-only opens skip the lock by
design) and states which two layers validate reports.

Tests: TestRecoveryValidateRunsTheGateBootRuns fails on the pre-fix code
("validate accepted a stored policy the daemon refuses to boot with") and
covers a store the State Store never seeded; TestRecoveryBackupRefuses-
TheDatabaseAsItsOwnSnapshot fails on the pre-fix code ("backup over the
database itself succeeded"). The validate-accept test now seeds the store
through ImportConfig and asserts the reported count is non-zero, and the
stored-resource case uses a value both gates refuse instead of one only the
write path does.
2026-09-21 22:04:42 +10:00
sam b8b477fb32 feat(statestore): offline backup, restore, validate and rollback
archie-state-store gains the offline recovery commands
docs/prds/runtime-control-plane.md ("Bootstrap, migration, and recovery")
asks for. They work on the task database file directly -- <db_path>-tasks.sqlite,
not db_path -- and need neither archied nor the Web UI, which matters because
the failure they exist for is the one that stops archied starting:

  backup   -db F -out F     consistent snapshot (VACUUM INTO), verified as a
                            usable store before it is published; safe against
                            a serving store
  restore  -db F -from F    replace the database with a snapshot, dropping the
                            replaced file's -wal/-shm
  validate -db F            readable, not corrupt, schema no newer than this
                            binary, every stored resource decoding under the
                            definition that owns it
  rollback -db F -kind K    replay an earlier revision of a stored resource
           [-revision N]    through the ordinary replace, recorded as a new
                            revision

No rollback RPC is added and none is needed: a revision carries its own value,
so restoring one is the same replace the dashboard and the channels already
use, by hand, with the store stopped.

"The State Store is stopped" is checked rather than assumed. The serving
process now takes an exclusive flock on <db>.lock for its whole life
(store.AcquireOwnership), and restore/rollback refuse with
`task database is owned by another process` while it holds it. backup is the
deliberate exception: the update installer runs inside the process it is
replacing and cannot stop the State Store.

scripts/archie-update-install takes its pre-flight snapshot through
`go run ./cmd/archie-state-store backup` from the release being installed
instead of its own sqlite3 shell, so the installer and an operator run one
implementation. The watchdog's refusal semantics are unchanged: an update that
cannot back up still must not start.

Docs: safe-change-and-recovery.md's "Recovery: not yet built" now describes
what exists.
2026-09-21 22:04:41 +10:00
sam 31622ec5c2 refactor(config): delete the runtime config overlay
The control plane replaces this, rather than extending it
(docs/prds/runtime-control-plane.md, "First implementation"). Until now both
paths existed: overlay kept dashboard edits in its own SQLite file beside
archie.db and layered them over the file config at boot and on every reload,
which is the duplicate configuration authority the PRD set out to remove.

Gone with it: the --no-config-overlay flag and ARCHIE_SKIP_CONFIG_OVERLAY
recovery hatch, which existed because a bad overlay row could brick the boot;
ReloadStatus.OverlayUnavailable; and the reload controller's overlay source.
The -config-overlay file overlay is a different thing and stays.

The dashboard's three write seams went too. UpdateConfig, ResetConfig and
UpdateRepoField were assigned only in internal/webui's own tests, so in
production they answered 503 and ConfigView.Editable was permanently false
(archie-core-j28m). Removing Editable would otherwise have flipped the SPA's
controls on, since configEditable read a missing field as true, so the config
pages become what they already were in practice: a read of what each process
is running with. Configuration is written through the control plane.

apply.go survives this deliberately; archie-core-xrux carries the question of
whether its map merge is still guarding the file overlay path.
2026-09-21 08:57:55 +10:00
sam 924b02a1b0 feat(controlplane): runtime administration over the State Store's gRPC server
The Web UI and messaging channels now read and write Archie's runtime
settings through one versioned, audited control-plane API hosted on the
State Store's existing gRPC server: catalog, query, command, watch. Each
feature owns its resource, validation and commands; storage only writes
versions and audit records.

Workflows move with it. Definitions become portable YAML compiled through
a closed, typed step registry and stored as a control-plane resource, so
the yaegi interpretation paths (internal/gate, wfeval, wfextract,
gateeval, gateextract, skillbuild) are deleted rather than carried.

The UI process reads definitions as data: the JSON projection lives in
internal/domain/workflow/task so a dashboard decodes the stored
collection without linking the workflow engine, which cmd/archie-ui's
deletion gate forbids. Shipped definitions ride the catalog descriptor
rather than a bespoke REST route, keeping the API to its four operations.

archie-messaging gains -state-store-target/-state-store-token and reads
channel settings from the database, failing closed when the State Store
is unreachable: the database is authoritative for those settings.
2026-09-21 08:17:42 +10:00
sam e0b6aa6105 feat(workflow): post line-anchored review comments with suggested fixes (#891)
* feat(workflow): post line-anchored review comments with suggested fixes

Bead archie-core-q9au (docs/prds/inline-review.md). A review's findings reached
the operator only as a flat list in the PR body, so the author had to map every
finding back to its line by hand and apply mechanical fixes by hand. Now the
line-anchored findings are also posted as inline review comments, carrying a
one-click suggestion when the reviewer could state the fix.

Domain (internal/domain/workflow):
- ReviewFinding gains Suggestion; Validate rejects one on anything but a
  confirmed finding, so the agent's record_finding call is refused with
  something it can act on. Validate splits into validateAnchor /
  validateClassification / validateDisposition, which also brings it back under
  the cyclop limit the gate enforces.
- StagePostReviewComments runs after StageOpenPR in the implement workflow. It
  posts only line-anchored findings (Line > 0); a whole-file finding has no line
  to attach to and stays in the body list, which remains the complete record.
  It is best-effort: the PR is open and the body already lists every finding, so
  a failed comment is logged, never a park -- the same reasoning as OpenPR's
  existing best-effort LinkBranch call.
- The suggestion fence is gated on Verdict == confirmed where the fence is
  written, because it is a one-click apply: a wrong suggestion breaks the code
  and the author has to notice.

Forge (internal/forge): ReviewCommentWriter + GitHub/Gitea implementations,
type-asserted like PullRequestReader so a forge that cannot is skipped rather
than faked (NoopForge is pinned as not satisfying it). GitHub loops one
CreateComment per comment and joins the failures, so one refused anchor cannot
drop the rest; Gitea carries them on a single COMMENT-state review.

Two deliberate departures from the PRD's sketch, both recorded in it:
- The interface is batch (CreateReviewComments), not one call per comment:
  Gitea's only inline shape is one submitted review, and the set then crosses
  NATS once instead of once per finding.
- The head SHA is resolved inside each forge implementation rather than by the
  caller. The stage runs in archie-agent, which holds no forge credentials and
  has no PullRequestReader; the daemon-side forgerpc handler is the only place
  that does.

Transport (internal/forgerpc): workflow.Forger gains the method, so the agent
reaches the daemon over the new create_review_comments subject; the server
type-asserts the writer and answers an incapable forge with an error the stage
logs. This is the one proxied method whose absence on an image skew does not
park a task.

The reviewer prompt and record_finding schema ask for a suggestion on a
confirmed, mechanically fixable defect only.

Regenerating the wfextract symbol table -- the documented way a new exported
workflow symbol reaches interpreted stage plugins -- also picked up symbols that
were missing from the committed table (the Status* and Source* constants,
ReviewCheck, Store, Task, and the _Store wrapper). Nil-guards were re-applied to
every wrapper afterwards, as that file's header requires.

Evidence: task check exit 0 (incl. 229 UI tests). Three mutations of the new
stage logic each fail a test that names the behaviour -- dropping the
confirmed-only gate, dropping the Line > 0 filter, and letting a posting failure
escape -- so the tests discriminate rather than passing vacuously.

* fix(build): drop the Dockerfile pin assertion

The guard fetched golangci-lint's go.mod at image build time to fail the build
when GOLANGCI_LINT_VERSION and GOFUMPT_VERSION disagreed. It only ever tripped
on self-inflicted pin drift, but it added a network fetch and a new failure
mode to the release build, which is not where that belongs.

The pins stay correct (golangci-lint 2.13.2, whose vendored gofumpt v0.11.0
equals GOFUMPT_VERSION) and the comment above them still states the invariant,
so the reasoning survives without the build-time machinery. The pin change
itself reaches the agent image through the normal release path: CI rebuilds
and pushes archie-agent on an archie/vX.Y.Z release, and a gateway-only
release deliberately does not.

* feat(workflow): add the remediate workflow for PR review remediation

Implements step 5 of docs/prds/pr-review-remediation.md: a remediate
workflow that resumes the task's existing worktree onto its PR branch
(worktree.Manager.Resume, not Prepare's refresh), runs a builder agent
against the review's actionable comments, commits and pushes to the
same branch, and replies via the forge's ReplyToReview. Task.RetryCount
doubles as the remediation round counter, capped by the repo's
max_retries; the round-cap stage parks the task and posts one comment
when it's hit.

Task gains a ReviewPayload field (JSON-encoded review unit) with full
store/proto/staterpc plumbing. workflow.Forger gains Comment and
ReplyToReview, proxied to archie-agent via a new forgerpc RPC. Resume
and its Dir lookup are added as an unexported optional Trees capability
rather than widening the exported interface, which is already at
golangci-lint's interfacebloat cap and is projected into wfextract's
Yaegi symbol table for repository-authored stage plugins.

The daemon-side reaction consumer that dispatches into this workflow
(subscribing ARCHIE_REACTIONS, grouping comments by review, enforcing
per-task serialization) is tracked separately in archie-core-8li9.4.

* feat(messaging): extract standalone messaging service binary and process composition

* feat(messaging): adapt channel frontends to consume Gateway ChatContract

* feat(messaging): remove in-process channels and Router from daemon bootstrap

Telegram, email and webhook now run only in the Messaging Service, which
reaches this process through the Gateway's ChatContract. The daemon builds
no channel and no gateway.Router, so the wiring that existed only to serve
them goes with it: setupGateways, setupEmailGateway, setupWebhookGateway,
setupTelegramGateway, buildTelegramRouter, makeTelegramReload and the
release-announcement path helper.

The seams those paths owned move to where they can still be honoured:

- Channel ValidateConfig runs in archiemessaging.compose, which now returns
  an error, so a configured-but-invalid front-end fails the service rather
  than being dropped silently.
- The inbound rate limiter moves to the Gateway process, which owns the only
  Router and therefore the only place a per-sender budget applies to every
  channel's turns.
- The daemon's gateway readiness probe becomes a ContractProbe against the
  Gateway. It cannot observe a channel lifecycle it no longer runs, and a
  channel-count probe would have reported every daemon degraded.
- telegramSetup is now chatSetup (in chat_setup.go), trimmed to the fields a
  chat turn runner actually reads.

Telegram operator features that were wired only in the daemon (/update,
release announcements, version reporting, SIGHUP reload) are not yet wired
in the Messaging Service: archie-core-8cda.6.6. Channel lifecycle reporting:
archie-core-8cda.6.8. The Gateway process builds no worktree manager, so
review_pr is unadvertised on every channel: archie-core-8cda.6.7.

* feat(messaging): re-wire the Telegram operator surface in the Messaging Service

setupTelegramGateway satisfied eight telegram.Gateway seams from the daemon's
own process. Each is resolved by where the fact it needs is owned, not by
where it used to be built; the decision table is in
docs/architecture/migration-decisions.md.

Wired here: /update (built from this service's own [chat.telegram] check and
install commands), the update-report relay (same sha256(bot_user)[:8] path the
daemon wrote, so a pending report is found rather than written afresh),
show_tool_calls, and /restart's config reload, which re-resolves the token and
allowlist from this service's own file and refuses a reload that resolves to
an empty token.

/version is sourced from the Gateway: ChatSnapshot gains a Version field
carrying the Gateway's rendered build block, read per call and bounded by the
dependency timeout. cmd/archie-messaging deliberately takes no version
ldflags, so it cannot report a partial upgrade as a matched one.

RunningVersions and ReleaseAnnouncements stay nil. Both need a component's
real running version, and the only versions this process can obtain are a
different binary's stamps -- reporting them would manufacture exactly the
false success the update check exists to catch, and the announcer without
parseable versions is a no-op that reads as configured. archie-core-8cda.6.9
carries the contract that unblocks them. Dangerous stays nil: no composition
has ever supplied a command authority.

Enrich is dropped from the update service: componentInstallTypeEnricher read
[nats], which this service must not decode, so the check command's own
reported install type stands.

* fix(workflow): guard inline review comments against head drift

Two findings from the review of q9au.

A comment's line numbers describe exactly one revision of the branch, but the
forge resolved the pull request's head at posting time -- after the review had
already read the worktree. A push in that window (a collaborator, a retry) meant
the findings were anchored to whatever now occupied those lines, mislabelling
instead of failing. The revision the review measured now travels with the
comment set: OpenPR records it from the same worktree read it already performs
(TaskContext.ReviewedHeadSHA), StagePostReviewComments threads it through
Forger to forgerpc, and the forge refuses the whole set once the head it reads
has moved off it -- which the best-effort stage logs, the PR body remaining the
record. An unmeasured revision posts unverified rather than dropping every
finding, so a worktree that cannot report one degrades instead of regressing.

Suggestion was documented single-line only, because GitHub applies a suggestion
fence up to its first line and a multi-line replacement would silently corrupt
the file. Only the verdict was validated, so a multi-line suggestion reached the
renderer and posted as that truncated fence. Validate now rejects an embedded
newline, which makes record_finding refuse it with feedback the model can act on
instead of the reviewer discovering the corruption later.
2026-09-19 23:37:23 +10:00
sam 04d4d8f1d8 chore(memory): delete internal/memory (unification slice 5) (#889)
Deletes internal/memory (Manager, builtin.Provider, memory_edit tool) now
that slices 2-4 built the CRUD engine family and its per-turn read/write
seams. Also deletes internal/infrastructure/memory/builtin/provider.go (the
memory_edit adapter over the old package -- its markdown store already
relocated in slice 2), internal/tools/provider/memory (the old tool
registration, redundant with slice 4's memory_create/update/delete/list),
config.Memory.{Provider,ProviderConfig} and the validateMemory rejection
that existed only to complain about Provider, and webui.MemoryStatus/
MemoryProviderHandle plus its dashboard page -- never wired in production
(s.Memory was always nil), confirmed by /api/capabilities always reporting
"memory": false.

Rewrites ARCHITECTURE.md's "Memory engine family" section to describe the
current engine (internal/domain/memory contract, internal/infrastructure/
memory builtin engine + its builtin/ markdown store, the per-turn read seam
in gateway/turn_memory.go, the per-turn write tools in
gateway/turn_memory_tool.go) instead of the deleted Manager. Fixes two
docs/architecture/configuration.md rows that described the deleted
manager's literals and SaveConfig mechanism.
2026-09-19 13:43:16 +10:00
sam e0beb68978 feat(config): let the State Store and Gateway listen addresses be configured
Both gRPC listeners took their bind address from a -listen flag default alone,
with no config key. The port a service binds and the port its clients dial were
therefore two separate facts -- the flag default, and [services.<name>].target --
with nothing keeping them in step.

On a host that already owned the default the effect read as a code fault rather
than a port collision: the store died with "bind: address already in use" while
every client reported "state store: connection error ... error reading server
preface: EOF". Cockpit binds 9090 by default, so this is not hypothetical.
Verifying the dashboard on such a host needed a hand-written overlay plus a
matching -listen, two things to keep in step by hand.

- [services.state].listen and [services.gateway].listen now name the bind
  address. Precedence is -listen, then config, then a default applied by the
  loader. An empty result is a startup ERROR, never a fallback: net.Listen reads
  "" as "any free port", so defaulting there would bind the service somewhere no
  client looks and still report success.
- Both flag defaults become "" so configuration can win, mirroring how archie-ui
  already prefers [web].listen over its flag.
- Gateway listen defaults to the same constant as the gateway target, so a
  default deployment's client and server cannot drift onto different ports.
  [services.state].target still has no default: an empty target stays a startup
  error in the daemon and gateway, which no longer serve a local store.
- ServiceConnection gains the field. Its doc now says clients read only Target
  and TargetToken, and Listen belongs to the process that owns the service --
  the previous wording claimed the struct starts no listeners while carrying one.
- deployments/dev.toml states each port once, beside the target it must match, so
  `task dev` is retargetable on a host that owns 9090 by editing one overlay.

Tests. Unit: config defaults, explicit values, and precedence. Process: the
existing real-process smoke test now takes its listen from configuration instead
of -listen, so the end-to-end wiring is proven by the test that already starts
the binary, with no extra process. Both were verified load-bearing by mutation --
inverting the precedence fails the unit test, and making the app layer ignore
config fails the process test with "state: no listen address".

Verified: task check exit 0. Also documented in config.example.toml, the
configuration architecture table, and the `task dev` summary.
2026-09-18 19:15:02 +10:00
sam 2ef962912a feat(tasks): attribute events, changes and config to the attempt that produced them
The events table had no attempt column, so two attempts of the same task were
indistinguishable: a retried task's stage events interleaved into one timeline and
nothing could say which run produced a given result.

- events gains an `attempt` column through the existing presence-gated
  migrateTasks loop rather than a version bump, so user_version stays 3 and an
  existing archie.db is migrated in place. A legacy-database test builds the
  pre-change schema and proves both that existing rows survive unattributed
  (attempt = 0) and that a new value round-trips.
- Event.attempt = 11 and ReadTaskLogRequest.stage = 9 are field additions to
  existing messages: no new RPC, no new table, and no change to any canonical error
  message that crosses the state-store wire.
- Every task event the workflow engine emits now carries the attempt. Producers
  that have no attempt yet (task_queued, binding_dispatch_failure) stay
  unattributed rather than guessing one.
- Changed files are captured as a durable changes_captured event at commit and push
  time, reusing the existing three-dot diff primitive: paths from FilePatches(),
  counts from Stats(), base_sha recorded as the merge base actually diffed rather
  than the base tip, a 200-file cap with totals kept complete. Consumption goes
  through an unexported optional interface so workflow.Trees -- the yaegi capability
  surface for interpreted stage plugins -- is not widened. A capture is also taken
  once the pull request exists, so the capture can link it.
- The per-attempt configuration is captured as a durable config_captured event
  carrying the non-secret config.TaskConfig, instead of a new table plus two new
  state-store RPCs: events are already this system's per-attempt provenance stream,
  so a table would have given one feature two provenance mechanisms. Redaction is
  proven against canary secrets. Measured payload: 539 bytes typical, 2333 bytes at
  20 models.
- Captures carry a schema marker and are refused unless their key set matches the
  declared payload shape, so drift surfaces as "recorded but could not be read"
  rather than as a capture of zeroes.
- Three read endpoints serve the attempt rail, the changed-file captures and a raw
  debug view, mapping to UI-owned response types rather than dumping store types.
- The task-log read gains a stage filter, bound to the stage logger for that
  stage's own lines; agent and tool output records no stage.

A task-scoped grant may already insert events on its own task, so this provenance
is exactly as trustworthy as the rest of that task's event stream. That limit is
documented rather than papered over.
2026-09-18 18:32:59 +10:00
sam ee608d81fb feat(configuration): add archied setup, the caller the first-run flow was built for
docs/architecture/configuration.md decides that "install.sh must stop generating
config.toml; archied setup owns it", and gives the reason: the installer
hand-rolling TOML already drifted from the schema once, and "the durable fix is
that the code writing the config is the code that reads it". setup.Run,
tomlwrite, configtemplate.Example and terminalprompt were all built for that and
had no production caller between them. This adds the caller.

setup.Run's own doc comment states the obligations, and they are the shape of
installValidatedConfig: render the edits, prove the result loads through a real
configuration.Loader, and only then install the file and commit the secrets. A
validation failure returns before either write, so a bad config leaves neither a
config nor a secret on disk. The rendered bytes are proven from a temp file, so
a load failure cannot have already installed anything. The config is written
0600 via same-directory temp and rename.

Parameterisation is typed, not matched against prompt text. A flag-to-prompt
mapping would have to match strings, which is the brittleness removed in
9a4f11c, so setup.Params is consulted before each question instead and the
prompts are only reached for answers nothing supplied. Run stays a wrapper
passing the zero value, so the package's existing tests keep passing untouched.
No environment variable is consulted anywhere: flags and prompts are the only
two ways an answer arrives.

--defaults is a known-valid unattended baseline, not "each prompt's own default".
That cannot work, and the test proves it: the provider step's first option is
OpenAI, whose model prompt hard-errors on an empty answer. The baseline therefore
covers every required answer and leaves optional ones to answer themselves as
"skip" -- ollama is the only keyless provider, so an unattended install boots
without needing a secret.

Two gaps closed after review rather than left as accepted work:

- A cloud provider was not configurable without a terminal, because the provider
  API key had no typed parameter: unattended setup could only ever pick the
  keyless self-hosted provider. Params.ProviderAPIKey and -provider-api-key fix
  that. The key goes to the env file and TOML carries only the secret reference.
- The write order was unpinned. Inverting installValidatedConfig's two writes --
  committing secrets before the config -- left every test green, so nothing held
  the invariant the SecretSink contract states. The new
  TestInstallValidatedConfig_ConfigWriteFailureLeavesNoSecret fails on that
  inversion; the existing validation-failure test does not cover it.

Also fixed three lint failures in the first pass (gocyclo on RunSetup, and two
ineffectual `choice` initialisers): RunSetup's flag surface, the flag-to-Params
mapping, the prompter choice and the existing-config read are now separate
functions, which is the real fix for the complexity rather than a suppression.

The setup flow no longer hand-rolls its own answer format: -telegram-user-ids
reuses setup.ParseTelegramUserIDs, the same parser the prompt path uses, so the
allowlist cannot be parsed two ways.

Found by an adversarial package sweep (temp tracker: config-1,
repo-root-deployment-2, .local/issue-tracker/). Plan of record:
.local/issue-tracker/design-archied-setup.md. install.sh still generates its own
config; that is phase 2 and is not claimed here.
2026-09-15 02:52:24 +10:00
sam 3b60ec23ba Merge pull request #827 from samcharles93/fix/webui-config-projection-chat-setup
fix(webui): count every chat front-end in the published projection
2026-09-15 00:26:55 +10:00
sam 03775e6fa6 test(archie-ui): drive the projection through the real producer in the smoke suite
The real-process suite hand-wrote the ConfigView document into the store, so it
asserted on a shape it built itself and could not have caught a producer that
forgot to publish a field. Both values GitHub #821 named -- chat.show_tool_calls
and the setup operator name -- arrive in the extracted process through that
projection and nowhere else.

seedStore now renders the document with webui.BuildConfigView, the same function
the daemon calls, from a config.Config, and a helper republishes between turns
the way a config reload does. The suite additionally streams a turn through the
fake Gateway's tool event and asserts the resulting frames twice: with
show_tool_calls on the frames carry the tool, and with it off they do not. That
is the browser-visible behaviour the issue described, asserted on what the UI
process actually sends.

Verified the new assertions are sensitive rather than vacuous: emptying
ConfigView's chat section makes the "on" case fail on a missing tool frame and
the checklist case fail on an empty operator, which is the original bug.
2026-09-14 10:55:55 +00:00
sam 0b7e3a86b3 fix(webui): publish per-identity forges so multi-identity task rows keep their links
Task rows in a deployment configuring [[identities]] rendered no forge links.
ConfigView carried the default identity's forge alone, so resolveForge
(api_tasks.go) withheld the links entirely rather than pointing every row at
the wrong forge -- production behaviour since the UI process cutover
(archie-core-8cda.5.4), where the dashboard runs in a process with no
config.Holder. archie-core-ml30 then removed webui.forgeConfigForTask and its
test, which resolved a task's forge from config.Config and had no reachable
caller once the holder was gone.

Fix at the producer, not the renderer. ConfigView gains `identities`: each
configured identity's forge type and host, plus the repositories it owns. That
is the whole answer to "which forge owns this task", and it is a projection the
daemon already rebuilds and republishes on boot and on every reload, so nothing
new is needed on the publish side. Only coordinates and repository names cross
-- an identity's token is a secret reference and stays out, which the new test
asserts alongside the existing leak guard.

resolveForge ports the matching rules recovered from c65f9fb's
forgeConfigForTask onto that projection: an exact identity name first, then
repository ownership, and an ambiguous repository -- two identities claiming the
same owner/name -- falling back to the default forge rather than inventing a
link. Two deliberate departures from the recovered shape: it indexes the
identity list once per request instead of rescanning config per task, since the
projection supplies repo slices it can key on directly; and a document that
reports multi-identity but carries no per-identity forges still withholds,
because that is what an older daemon published and guessing there is the
original bug. The recovered code's first-wins scan for a duplicated identity
name is preserved.

Tests: the projection's per-identity forges and their secret-freedom; the three
matching rules and both fallbacks through resolveForge; every row of a
multi-identity GET /api/tasks carrying the right forge's link; and the real
archie-ui binary driven over HTTP against a multi-identity projection, which
fails on the pre-fix resolver with three unlinked rows.
2026-09-14 10:36:29 +00:00
sam 408b8c6bff test(archied): stop racing state-store startup against a wall clock
TestRunStateStoreAcceptsEnvironmentToken wrapped the whole of RunStateStore --
SQLite schema init and the serve loop -- in a 200ms context deadline, so
whether it passed depended on how fast the disk was. Measured before the fix:
6 failures in 12 consecutive runs. A gate that fails half the time teaches
people to ignore it, and the failure it reported ('store: init schema:
context deadline exceeded') named a time budget, not a defect.

Cancellation is the assertion, so the test now cancels and then asserts:
the call must not return before cancellation (that would mean startup refused
the config, the thing this test exists to catch) and must return within a
generous grace once cancelled. The graces bound 'did startup reject us', not
'machine speed'. Measured after: 12 passes in 12 runs.

Also applies the canonical gofumpt and 'go fix' output to
internal/infrastructure/staterpc/dial.go, cmd/archie-ui/main_test.go and
internal/releaseupdate/check_script_test.go. task check runs gofumpt -w over
the tree, so leaving these unformatted made the gate rewrite unrelated files
mid-run and report dirty state for changes it had made itself.
2026-09-14 08:09:14 +00:00
sam 19062a702d refactor(webui): take configuration off the dashboard server
webui.Server declared Cfg *config.Holder with eight read sites, none of
which ran in the UI process: composition leaves it nil there. The field
survived because the daemon kept a webui.Server it never served, using it
to render the published ConfigView snapshot and to hold the readiness
probes (archie-core-ml30).

Split at the producer. LocalConfigView becomes webui.BuildConfigView, a
pure function over a ConfigViewInput the configuration owner assembles;
the DTO and its schema stay where the UI renders them. The daemon builds
that input from its own state (boot.configViewInput) and publishes the
result. Readiness reads the daemon's Holder, channel manager and chat
contract, and the registry it produces is a daemon field, not a field on
a server with no listener. Boot now owns one Holder instead of seeding a
second and replacing it in buildDaemon; operator retry caps read it too,
so they follow a reload rather than a boot-time snapshot.

Server loses Cfg, ConfigProvenance, LastReload, ConfigOverrides and
SetProvenance. The nil-guarded branches they fed went with them, each
checked first against what the UI process actually serves:

- chat.show_tool_calls and the /api/setup checklist were degraded, not
  dead. ConfigView gains a chat section carrying show_tool_calls, the
  operator name and whether a channel is configured, and both surfaces
  read the projection. The end-to-end suite asserts the checklist renders
  from the published snapshot.
- /api/logs takes its path as a process-local option. The history is on
  the daemon's disk, so a dashboard elsewhere cannot read it by learning
  the path; that route still needs a contract.
- handleChannels drops its config-derived fallback, superseded by the
  status manager, which already carries the descriptions and detail.
- trustForwardedHeaders keeps only the process's own setting, which the
  UI process already sets.
- resolveForge keeps the published-projection path it already had.

ConfigView.Editable moves to handleConfig, decided by whether this
process holds UpdateConfig. It describes the writer, not the document, so
the write contract (archie-core-j28m) makes the page editable by wiring
one field. The write handlers and their seams stay for it.

overlay.Store gains Keys: Snapshot nests dotted keys for decoding, so the
overridden-key list was publishing section names ("budgets") where the
dashboard marks and resets rows by stored key ("budgets.max_steps").

Multi-identity forge links move to archie-core-pv6t: forgeConfigForTask
resolved a task's forge from config.Config, had no reachable caller once
the holder was gone, and belongs in the projection rather than the
rendering package.
2026-09-11 23:57:38 +10:00
sam af2c8e1f41 fix(staterpc): attach the bearer token to State Store streams
The capture batch reads moved onto server-streaming RPCs (StreamCaptures /
StreamUndispatchedCaptures) to get past gRPC's 4 MiB unary cap, but the
client interceptor suite attached the bearer token to unary calls only:
Dial installed WithUnaryInterceptor and nothing for streams, while
TaskGrants.StreamInterceptor requires the administrative token. On every
token-protected (non-loopback) listener -- every container-mode deployment
-- the daemon's binding dispatch and the dashboard's capture inspector got
PermissionDenied from the stream interceptor, and the unary fallback those
reads replaced is no longer taken, so there was no working path.

- Add StreamClientTokenInterceptor, mirroring gatewayrpc's, and install it
  alongside the unary one in Dial, the single client dial path the daemon,
  the dashboard and the agent all use.
- Route agenttransport's State Store dial through staterpc.Dial, deleting
  the duplicate dial/fail-closed logic that installed only the unary
  interceptor -- a second place for this class of bug to hide.
- Regression tests: the grants harness now dials through the production
  Dial so client-side wiring cannot drift again;
  TestCaptureStreamsCarryTheAdminToken reads both list surfaces (5 MiB
  batch) through a token-protected listener;
  TestStateStoreRealProcessTokenProtectedCaptureStream does the same
  against the real binary; TestUpdateRejectsNilTask covers the Update
  guard left untested by the earlier finding fix.
2026-09-11 22:43:34 +10:00
sam 21710f1147 test(ui): smoke the archie-ui process against live dependencies
The in-process suite proved the composition; nothing proved the process.
This builds the real binary, points it at a State Store and a Gateway on
loopback gRPC through an operator-shaped config.toml, and drives it over
HTTP: the token exchange and its 401s, the embedded SPA, a chat stream
relayed frame by frame, an operator task action crossing to the Gateway
and refused without the CSRF header, unauthenticated capture intake, the
ratified degradations, readiness dropping when a dependency dies while
liveness holds, and a clean SIGTERM.

It found one regression, fixed here at the producer. Task rows lost their
forge links in the extracted process: taskURLs read config.Holder, which
is nil once the dashboard stops sharing the daemon's process, so every
click-through to the issue and the PR silently disappeared. The forge is
now resolved once per request from whichever source that process owns,
the daemon's configuration or the daemon's published projection.

A multi-identity deployment keeps no links rather than gaining wrong ones:
the projection carries the default identity alone, so it now says so and
the renderer withholds.
2026-09-11 20:10:59 +10:00
sam 6f1ad5c5b5 test(ui): enforce the UI process deletion gate on every check
The severance that cut archie-ui's package closure from 68 to 26 was a
one-time measurement: nothing re-ran go list -deps, so the next import of
a type sitting beside a runtime would have relinked that runtime silently
and left the suite green.

TestUIProcessLinksNoDaemonRuntime re-measures the closure and fails on any
package in a banned category, with the categories named as the PRD names
them. Its two exceptions are checked in both directions so the allowlist
cannot decay into an open prefix. database/sql is matched exactly rather
than as a subtree: database/sql/driver arrives from google/uuid's
Valuer/Scanner and brings no driver with it.

Verified by violation: one blank import of internal/store relights eleven
banned packages across three categories.
2026-09-11 20:00:41 +10:00
sam 536fa20254 feat(webui): deliver live dashboard events by polling the State Store cursor
archie-core-za9f. The UI process owns no event bus, so after the .5.4 cutover
the activity feed would show history at page load and then go silent: live
delivery is a Server.Broadcast call made only from the daemon's bus
subscription.

Decided against the push-subscription RPC the bead assumed, on three findings:

* store.Open sets SetMaxOpenConns(1), so writes serialise and the event id is
  assigned in commit order. The gap-skipping failure that normally rules out
  polling an autoincrement cursor cannot happen here, and SSE catch-up already
  depends on the same property.
* sseStream.drain treats a broadcast as a wakeup, not a payload: it fills the
  persisted gap via catchUp and delivers the event only if catch-up missed it,
  deduplicating on the since watermark. A poller substitutes for the bus with
  no change to sse.go and no new SSE semantics.
* The events table is the only complete fan-out point. ArchiveTask writes an
  audit event inside its transaction and daemon.go inserts binding-dispatch
  failures directly, neither touching the bus. A push hub needs a hook at every
  write site and misses any added later.

So: no proto change, no new auth surface, no backpressure policy. The pump
primes past existing history (a client's own catch-up serves that), then ticks
EventsSince into Broadcast. One pump serves every browser. A failed fetch
leaves the watermark intact, so a State Store restart costs latency, not
events.

Wired into archieui.Run in the background so priming a large events table
cannot delay the listener, with -event-poll-interval (default 1s).

Decision recorded in migration-decisions.md; ui-service-boundary.md's /events
route-owner row moves to RESOLVED, and /api/logs splits out of it as still
unresolved and host-local.
2026-09-09 02:53:32 +10:00
sam 0ec4ececd5 feat(gateway): authenticate the gRPC contract and allow off-loopback listeners
The Gateway had no transport boundary: RunGateway refused any non-loopback
listen address and serveGateway registered no interceptor, so archie-ui was
colocation-only while the State Store already had a token-authenticated
equivalent.

gatewayrpc gains a gateway-token metadata credential with constant-time
comparison and both unary and stream interceptors on each side. Stream
coverage is required here and not in staterpc: the Gateway's Stream RPC is
server-streaming. Dial fails closed on a non-loopback target with no token,
mirroring staterpc.Dial. gatewayServerOpts applies the same rule server-side:
loopback stays insecure, non-loopback requires a token, malformed errors.

The daemon resolves [services.gateway].target_token then GATEWAY_TOKEN;
archie-ui passes the token its config projection already carried, so a
distributed dashboard is now possible.

Closes: archie-core-juqx
2026-09-09 00:27:03 +10:00
sam 3d611f18ab feat(archieui): standalone UI process over remote Gateway and State Store
Add cmd/archie-ui and internal/app/archieui: a second composition root that
serves the dashboard and the compiled SPA against the two contracts the daemon
already dials, so the UI stops needing the daemon's in-process objects.

The composition root lives outside internal/app/archied deliberately. That
package pulls 88 internal packages (daemon, forge, containers, NATS, secrets,
workflow); reusing its boot the way RunGateway and RunStateStore do would make
the PRD's deletion gate unreachable. archieui links 41, and cmd/archie-ui 42.

Configuration ownership follows docs/prds/ui-service-boundary.md: the process
reads the shared file, but only [services.gateway].{target,target_token},
[services.state].{target,target_token} and [web].{listen,
trust_forwarded_headers}, with flags taking precedence. Options has no field
for anything else in that file to land in, and no config.Holder.

compose() sets only the contract-backed and process-local fields. Every daemon
runtime handle stays nil on purpose and its route degrades through the
existing 501/503/empty paths. Captures, CaptureLimiter and BindingDispatcher
are withheld for the duration of the seam: the daemon's dashboard still serves
POST /webhooks/capture/{source}, and two intakes are not permitted.

Readiness is the UI's own registry over its two dependencies via a new bounded
readiness.ContractProbe. The existing GatewayProbe was unusable here: with no
channel manager to inject it reports "no gateway channels configured", i.e.
permanently degraded.

The State Store's fail-closed non-loopback rule and the two dial paths move to
staterpc.Dial/TargetIsLoopback and gatewayrpc.Dial, so the server side and
every client share one implementation instead of three copies.

The Gateway remains loopback-only with no auth interceptor, so this process is
scoped to being colocated with it; filed as archie-core-juqx.

Refs: archie-core-8cda.5.2
2026-09-08 22:54:26 +10:00
sam 33751abe4e test(staterpc): verify State Store end-to-end and preserve context error identity (#775)
* test(state-store): real-process smoke + deadline/sentinel error contract

Add end-to-end verification for archie-core-8cda.4.7, the .4.7 State Store
verification stage against docs/prds/state-store-contract.md (rev. 2c):

- Real-process smoke coverage: build ./cmd/archie-state-store, run it against
  a minimal config on loopback, poll the readiness surface, and drive the
  State Store gRPC contract through staterpc.NewClient exactly as a consumer
  does. Covers the combined task/event/capture view, error-sentinel fidelity
  across the wire, the found=false-not-error convention, single-owner SQLite
  (the binary owns the one archie.db file; the consumer never opens a store
  file), per-call deadline behaviour, restart/recovery (SIGTERM + reopen on
  the same db), and the remote capture/mapping/binding surfaces.

- Fix a real deadline-identity gap in unmapError: a cancelled or
  deadline-exceeded context surfaced by gRPC-Go as a *status.Error was
  re-wrapped into a sanitised internal-style error, breaking
  errors.Is(err, context.Canceled/DeadlineExceeded). The agent's deadlineStore
  bounds each Store call with context.WithTimeout and the workflow consumer
  (workflow.go) uses errors.Is(err, context.DeadlineExceeded) to mark a stage
  interrupted rather than failed, so the re-wrap would wrongly park an
  interrupted task. Unmap Canceled/DeadlineExceeded codes back to the standard
  context sentinels.

- Pin the error-mapping contract in staterpc: context-identity preservation,
  sentinel round-trip fidelity for every store sentinel, and the sanitisation
  rule (an Internal error must not leak raw detail or map to a sentinel).

* test(state-store): assert single-owner SQLite composition (daemon/gateway require target)

The single-owner invariant (docs/prds/state-store-contract.md §12 step 7/8) is
that only archie-state-store opens archie.db; the daemon and gateway resolve
their task-store surface from openStateStoreAdapter, which requires
[services.state].target and opens no local store. This test pins that
composition root so a future change cannot silently reintroduce a local
*store.Store in the daemon or gateway path (dual-store ownership).

* test(state-store): skip real-process smoke tests under -short

The smoke tests build and exec the real archie-state-store binary, which is
slow enough that a -short run (e.g. a focused edit loop) should skip them.
task check (go test ./... -count=1) does not use -short, so they still run in
the full gate.

* test(state-store): fix subprocess log race under -race

The smoke test captured the subprocess's stdout/stderr in a strings.Builder
that the os/exec writer goroutine and the test reader goroutine touched
concurrently, racing under -race. Replace it with a mutex-guarded buffer, the
standard pattern for reading a running subprocess's output.

* test(state-store): satisfy lint (noctx, errorlint) on new tests

- exec.Command -> exec.CommandContext, http.Get -> NewRequestWithContext +
  Client.Do, so the subprocess and readiness polling honour the test context
  (noctx).
- errorlint: compare the raw non-status error via errors.Is rather than != on
  a wrapped error.

* docs(deployments): document State Store readiness verification and single-owner invariant

The deployment runbook described how to launch archie-state-store but not how
to verify it is healthy before starting archied. Document the -ready-addr
readiness surface (/healthz, /health/detailed state_db component) and the
single-owner invariant: one task SQLite file owned by archie-state-store, with
archied/archie-gateway never opening archie.db directly (state-store-contract
§12 step 7/8).

* fix(state-store): preserve context deadline/cancel identity in mapError

The symmetric server-side gap to the unmapError fix: if a store query fails
because the caller's context expires mid-call (§6 StoreTimeout), the handler's
mapError folded context.Canceled/DeadlineExceeded into codes.Internal. The
client would receive a sanitised internal error with no deadline identity, so
the agent's workflow consumer (errors.Is(err, context.DeadlineExceeded)) could
not distinguish an interrupted stage from a failed one.

Map a context cancel/deadline error to its matching gRPC code so unmapError
rehydrates it back to the standard context sentinel, completing the §6/§7
round-trip. Pin both directions with TestMapErrorPreservesContextIdentity.
2026-09-07 05:47:36 +10:00
sam 3a0dfa924d refactor(state-store): cut daemon/Gateway over to the remote State Store (archie-core-8cda.4.6) (#774)
* refactor(state-store): delete NATS storerpc; agent store is gRPC-only (archie-core-8cda.4.6)

The agent is fully on the gRPC State Store contract, so the legacy NATS
storerpc path is dead (docs/prds/state-store-contract.md §12 step 4):
- delete internal/storerpc (Server, Client, subjects) and its tests
- drop the storerpc registration from registerTaskRPCServers and the
  now-unused store param
- make the agent transport's State Store URL required and Store() return
  only the gRPC staterpc.Client (no NATS fallback)
- rework agentworker/agent transport tests to back the store with a real
  gRPC State Store server instead of the deleted NATS storerpc server

The agent binary now fails fast when STATE_STORE_URL is absent. Agent store
calls (workflow.Store) go over gRPC to the archie-state-store service.

* refactor(state-store): delete daemon/gateway in-process store ownership (archie-core-8cda.4.6)

The daemon and gateway no longer open archie.db in-process: the single SQLite
file is owned by the standalone archie-state-store process, and both consumers
dial its gRPC State StateStore contract (docs/prds/state-store-contract.md
§12 step 7, no dual-store ownership).

- openStores no longer opens the production task store or seeds b.st; it only
  wires secrets/forge and the gateway conversation store
- openStateStoreAdapter now requires [services.state].target (the old local
  default is deleted) and resolves b.stateStore as the remote *staterpc.Client
- delete the in-process startStateStoreServer (the multiplexed gRPC server and
  its per-task bearer-token registry) from the daemon composition
- RunGateway now resolves the same remote State Store adapter; its session
  SQLite stays Gateway-owned and untouched
- daemon.ConnectedStateStore carries the configured target plus the resolved
  bearer token, which containerEnv injects as STATE_STORE_URL / STATE_STORE_TOKEN
  so archie-agent authenticates to the same remote service
- readiness store probe reads b.stateStore

PRD-consistent choice (noted): with the in-process server gone, per-task token
generation is removed; the daemon/gateway/agent all present the single
[services.state].target_token (or STATE_STORE_TOKEN secret) that the standalone
store validates.

* docs(webui): require [services.state].target in config and deployment templates (archie-core-8cda.4.6)

After the in-process store path is deleted, archied and archie-gateway dial the
standalone archie-state-store gRPC service via [services.state].target and never
open archie.db themselves (docs/prds/state-store-contract.md §12 step 7).

- config.example.toml documents [services.state].target as required, with the
  loopback vs Docker-bridge gateway + target_token rule
- deployment templates set [services.state] per topology: bridge gateway +
  token for container-mode agents, loopback for host-only
- deployments/README.md and systemd-user-service.md describe start commands
  for both topologies and drop the stale "not yet consumed by the daemon" note

* style: gofumpt alignment for State Store daemon literal; appease noctx/staticcheck (archie-core-8cda.4.6)

* docs(state-store): refresh stale in-process references post-cutover (archie-core-8cda.4.6)
2026-09-07 05:06:12 +10:00
sam e6a2e5b3b4 feat(state-store): stand up archie-state-store binary and own archie.db (archie-core-8cda.4.3) (#771) 2026-09-07 03:41:32 +10:00
sam 2b102b04a2 feat(state-store): generalize storerpc as the State Store gRPC contract (archie-core-8cda.4.2) (#770)
* feat(state-store): generalize storerpc as the State Store gRPC contract

Implements archie-core-8cda.4.2 against docs/prds/state-store-contract.md
rev. 2c: generate protobuf/gRPC client and server adapters for every
ratified store contract (proto/state/v1/state.proto, 40 RPCs, one
StateStoreService), add internal/infrastructure/staterpc with a bufconn
conformance suite exercising both local and remote adapters, serve the
service in-process (bridge address + mandatory per-task bearer token when
agent containers exist, loopback-only otherwise), and flip archie-agent's
workflow.Store to a long-lived gRPC client when the daemon injects
STATE_STORE_URL/STATE_STORE_TOKEN (falling back to the legacy NATS
storerpc path otherwise -- one agent process is never both).

Also drops the *sql.Tx parameter from store.BindingDispatcher.RecordDispatch
(a required wire-blocker fix: a transaction cannot cross a gRPC boundary,
and production always passed nil).

fix(store): drop RecordDispatch's unusable *sql.Tx parameter
feat(config): add services.state contract-adapter selector

* fix(state-store): apply no-mistakes review decisions

- Drop StreamTokenInterceptor (unused; the State Store service has no
  streaming RPCs and nothing calls it).
- Make StateStoreTokens.Validate compare tokens in constant time
  (crypto/subtle) per the auto-fix hardening.ing finding.
- Amend config.example.toml / config.Services.State doc text so it frames
  [services.state].target as the ratified surface consumed by the upcoming
  standalone archie-state-store binary (.4.3), not active today.
2026-09-07 03:01:33 +10:00
sam 10e76e7528 feat(gateway): consolidate task actions and repair daemon default path
Phase 1 (extract the Gateway Service), first stage. Repairs the
half-migrated working tree so the default (inproc) daemon path works
again while keeping the WIP's direction:

- chatTaskActorAdapter is contract-only (no HTTP detour); nil contract
  returns ErrChatCapabilityUnavailable instead of panicking.
- openStores opens the conversation store for both processes again;
  the standalone gateway no longer owns it exclusively. Phase 1 keeps
  the daemon's Telegram/email routers and session-memory curator
  reading the same SQLite file; remote conversations arrive when a
  process that cannot open the file needs them.
- setupLLMAndChat composes the local gateway (setupGatewayChat) in
  default mode and a remote gRPC client in mode=remote; the returned
  contract is never nil, and Run handles the error.
- taskactions.Service is now the single task-mutation implementation
  shared by the dashboard (webui TaskActionService), the daemon's local
  gateway actor, and the NATS responder. The daemon registers
  taskactions.Register so the standalone Gateway's client can forward
  actions; transport follows the repo's natsrpc pattern.
- uncommitted sessions.proto / SessionStoreService gRPC contract
  deferred: it has no wired consumers (scope creep for this stage).
- task_action_adapter_test.go rewritten against the contract chain.

task check passes; archied and archie-gateway both build.
2026-09-06 01:35:15 +10:00
sam 79375f33e9 feat(gateway): extract ChatContract service 2026-09-05 23:06:16 +10:00
sam 8c19555b4a feat(playbooks): standalone lint tool for playbook binding dirs
Fourth slice of docs/prds/eda-playbook-engine.md (t2db.12): the design
doc's pre-merge collision layer, as a gopls-shaped standalone binary
(cmd/archie-playbooks) rather than a Taskfile wrapper -- one binary, one
schema source, mode-based entrypoints.

- cmd/archie-playbooks: subcommand dispatch table from day one (lint now;
  an lsp/serve mode later slots in without restructuring), flag parsing
  only per organisation.md's cmd/ rule.
- internal/app/archieplaybooks: the lint mode's composition -- collects
  directories, calls workflow.LoadPlaybookDirs (the daemon's own loader),
  reports findings. No second validation path: the linter agrees with
  runtime startup validation by construction.
- Reuses LoadKindWorkflowsYAML/LoadLabelWorkflowsYAML/LoadPlaybookDirs
  directly; collision/malformed/invalid bindings exit non-zero.
- Known limitation recorded: findings are file-granular, not
  line-granular (yaml.Unmarshal into map discards line numbers; a
  file:line diagnostic needs a yaml.Node upgrade, out of scope here).
- Taskfile lint:playbooks target; PRD records the shipped shape and the
  file-granular caveat.
2026-09-03 03:07:27 +10:00
sam 5fa010c9c2 fix(plugin): add missing CLAUDE.md reference to plugin engine rule
Test TestPluginEngineRuleIsCanonicalAndSharedByAgentInstructions was
missing entirely, letting CLAUDE.md drift out of sync with
ARCHITECTURE.md#plugin-engine-rule-strict. Added the test and the
missing reference.
2026-08-31 21:25:20 +10:00
sam b2da85ead1 refactor: go modernisation fixes 2026-08-29 00:20:05 +10:00
sam 8120dc0760 feat(curator): track and expose recent activity per curator
Add a small bounded activity tracker (activityTracker, in
internal/domain/curator/activity.go) fed directly from the runtime's
existing emitRun point: last-run time, last-run action count, and up to
20 recent actions per curator, newest first. No persistence layer -- it
resets on restart, same as the rest of the runtime's in-memory state.
Registry.RecordActivity/Activity expose it.

Closes archie-core-1786637489932-6-1f803e02.
2026-08-28 22:28:48 +10:00
sam 96c82cd4e0 feat(tasks): allow rejecting a task from any non-terminal state
An operator could only reject work that was waiting on a human decision, so a
task sitting in review (PR open) or running had no way to be refused even
when it was clearly dangerous.

- expose reject in taskstate.Actions for every non-terminal state (queued,
  running, waiting_human, parked, pr_open) so both the dashboard and the chat
  tool surface it; terminal states still only expose archive
- make rejectTask transition from the task's actual status to closed_wont_do
  and interrupt running execution first, so nothing keeps working against a
  task being closed out
- keep the distinct stop (recoverable → parked) semantics; reject is the
  terminal escape hatch
- update the lifecycle contract tests to the new intent
2026-08-28 00:19:27 +00:00
sam 2395254559 build(deps): upgrade github.com/samcharles93/ai-sdk to v0.1.29 2026-08-27 21:40:01 +00:00
sam 48133e68fe feat(logging): cursor pagination and filtering for task_logs
task_logs could only read the newest N entries with no way to walk the
rest, so a task's log beyond the page cap was unreachable and the agent
had no honest way to say what was missing. Add logging.Page -- a
byte-offset cursor over the tail-most 8 MiB scan window -- plus
Query.Since/Until time bounds, and wire the gateway's task_logs tool to
expose level/since/until/after_id with cursor and more_available in the
result.

The reader now reads the scan window into a single in-memory buffer and
walks it by exact slice offsets. Offsets into a contiguous buffer are
trivially correct, which an incremental reader over the file is not:
bufio.Reader buffers ahead, so its underlying file position runs one
buffer past its logical position and cannot be used as a cursor. The
same rewrite removes f.Read in favour of f.ReadAt (full-or-error), since
a silent short read would drop the newest entries and still report
success.

The scan window is a documented hard ceiling: for a log over 8 MiB the
oldest bytes are unreachable by any cursor -- that is what Truncated
reports, and send_file remains the path for a complete archive. The
tool description teaches the model the difference between more_available
(more this call did not return) and truncated (older entries the scan
cap hides).

Closes archie-core-vq92 (#625).
2026-08-26 02:02:36 +10:00
sam 4b340d2b56 fix: harden task and chat execution
Task runs now preserve real model context limits, provider usage, baseline-repair spend, commit identity, durable agent-finish telemetry, and useful timeline details without imposing a token cap. Dashboard forge links are canonical per owning GitHub or Gitea identity. Chat turn claims acquire SQLite writer intent before reading, eliminating the concurrent SQLITE_BUSY snapshot race.
2026-08-25 05:48:17 +10:00
sam a27fb5f36e fix(update): remove source checkout dependency
Dashboard-initiated updates also retain a configured Telegram restart-report route, while falling back to the web UI report when Telegram is unavailable.
2026-08-25 03:28:37 +10:00
sam 901864a9df merge: task_action chat tool (crew/taskact) 2026-08-25 02:05:25 +10:00
sam e5e0316475 feat(telegram): send local files as uploads, not fetchable URLs
The agent could produce a file -- a transcript, a log dump, a screenshot --
and had no way to hand it over. Asked to send one it offered a "download"
link; nothing arrived and it had no signal that anything had failed. The
defect was not that it lied: success and silent non-delivery were
indistinguishable to it.

Three places made a local file undeliverable, and each is fixed here:

  - liveReply.Media skipped any attachment with an empty URL, so a local
    file was dropped before it ever reached a sender.
  - telegramMediaSender built every attachment as models.InputFileString,
    which tells Telegram to FETCH the string as a URL. A host path is not
    fetchable, so the send did nothing while reporting success. It now
    chooses by what the attachment carries: URL -> fetch, Path -> upload
    via models.InputFileUpload over an open file.
  - Nothing let the agent ask for a send at all. New send_file tool
    (internal/tools/sendfile) validates the file and returns a
    MultimodalResult naming it; the turn's channel delivers it.

Meaning changes, called out because the names did not:

  - MediaAttachment gains Path, the alternative to URL. Exactly one is
    meaningful for an outbound send.
  - tools.MultimodalResult.URLs used to mean remote-hosted media only; it
    now means media this result wants delivered, which may be local. JSON
    name unchanged so existing producers keep working.
  - errNoURL is now errNoSource: having neither source, not having no URL,
    is what makes an attachment undeliverable.

Failures are visible at every layer, which is the actual fix. send_file
errors on an absent, irregular, unreadable, oversized or confinement-
refused path, so the model sees it and can report it. The sender checks
Telegram's ceilings (10 MB photos, 50 MB otherwise) before the request and
reports the real size rather than leaving a 400 nobody reads. A delivery
that still fails now appends "could not send <name>" to the reply instead
of an empty link, which read as a delivered file.

Exposing a public URL for local files was explicitly rejected: it trades a
missing feature for a hosting and access-control problem.

send_file resolves paths through builtin.ResolveReadable -- the read tool's
own policy, exported rather than reimplemented -- so sending cannot reach
what reading cannot, whatever [chat] unrestricted_filesystem is set to.
2026-08-25 01:43:15 +10:00
sam 64da3952aa feat(gateway): add task_action chat tool for operator task management
Adds the task_action tool to internal/gateway/task_tools.go and wires it via
chatTaskActorAdapter in cmd/archied. This enables operators in chat to execute
lifecycle actions (abandon, archive, retry, stop, cancel, approve, reject) on
tasks using the same action path and state machine as the dashboard.

Enforces identity scoping so an instance cannot mutate another identity's
tasks, verifies status permissions via internal/taskstate, and classifies the
tool as mutating with required human approval.
2026-08-25 01:19:21 +10:00
sam 1f05dbb33c feat(deployment): default to managed NATS workers
Autonomous workflows now always hand a complete task to a scoped archie-agent container. Embedded NATS is the default and external NATS changes only broker placement; legacy agent and containers.enabled inputs decode for migration but cannot select an execution path. The daemon flushes task-event subscriptions before a worker can publish, and runtime updates snapshot and restore the configured worker image.
2026-08-24 14:56:57 +10:00
sam d3da153768 refactor(agentexec): remove legacy execution paths
Autonomous workflows now have exactly one production boundary: a task-scoped archie-agent receives a full-task core-NATS request and runs the worker-local stage loop. The legacy [agent] section remains decode-only for migration; it no longer selects runtime behavior.
2026-08-24 14:56:57 +10:00
sam 5303ef65db fix(daemon): require full-task worker handoff
Autonomous tasks no longer execute workflow stages inside archied; missing worker capability parks before worktree preparation.

A transition to parked now atomically makes its detail the task park reason, so daemon-side failures remain visible in task APIs and the dashboard.
2026-08-24 14:56:57 +10:00
sam 9d73bc7157 feat(nats): expose embedded broker to managed workers 2026-08-24 14:56:57 +10:00
sam cc53e5258d fix(telegram): compact tool output and drop turn caps
Render completed ToolCallEvents as Telegram-safe streaming blocks
with compact name/status and a short cleaned preview. Summarize
file contents and listings instead of dumping source or quoted
JSON. Collapse repeated legacy turn-budget failures.

Remove max_tokens turn gating and the 200000 turn_budget_chars
default so absent or commented config cannot restore an aggregate
cap. Keep per-result max_result_chars and spill.

Confine find/grep discovery to the active workspace/worktree.

Refs #604
2026-08-23 16:40:16 +00:00
sam 25af279d7a feat(releaseupdate): enrich components with install type and reference
Component.Installed/Available come from an operator-owned external
check command, which reports what version exists but has no way to
know how THIS process/deployment actually installed it -- native
binary, container, or embedded process. Two sources for that already
exist and were previously unused for this: internal/installtype
(this process's own build-time stamp) and daemon.AgentStatus (an
archie-agent version observed at runtime, landed in the prior commit).

Add Component.InstallType/Reference and a Service.Enrich hook that
fills them in per component ID after the check command runs, without
overwriting whatever a check command already reported for an ID it
doesn't recognise. Wire the actual sources in cmd/archied via
componentInstallTypeEnricher: the daemon's own installtype.Type(),
the agent's most recently observed install type, and -- for NATS, a
third-party dependency this repo doesn't build -- "embedded" or
"external" (with its URL) read straight from [nats] configuration,
which is already the deployment profile's own record of how NATS is
run. This is wired into the single Service construction shared by
Telegram and the dashboard, so both surfaces get it for free.

Part 2/3 of archie-core-1786751948782-3-837a8fd0 point 4. Closes
archie-core-1786751948782-3-837a8fd0.2.
2026-08-23 23:28:52 +10:00