Commit Graph
113 Commits
Author SHA1 Message Date
sam 3edbd6b0a7 feat(releaseupdate): transactional self-update through archied (#1633)
archied update install verifies the release zip, checks every binary
reports the release, pulls the agent image by digest, backs up the
database in-process and stages the binaries before handing off to
archied update watchdog in its own process. The watchdog restarts units
in release order, requires archied healthy for 120s and otherwise rolls
back binaries and image policy, restoring the database only when a
migration ran. Boot recovery resumes the frozen watchdog.

Deletes scripts/archie-update-install, scripts/archie-update-watchdog,
the updater's --from-source and --auto, CommandInstaller,
ScriptInstaller and the update-image and update-topology subcommands.
2026-10-12 03:21:02 +11:00
sam 0d78df69a3 fix(prreview): restore the benchmark runner on configured models and public PRs 2026-10-11 04:11:40 +11:00
sam 0b9769dbf2 feat(releaseupdate): archied update check finds releases through go-selfupdate; archie-update-check is deleted 2026-10-07 01:12:11 +11:00
sam ee2c9cda97 feat(releaseupdate): install from release artifacts with journaled rollback, topology check and channels 2026-10-05 17:02:29 +11:00
sam 823d60e1dc feat(presence): services card on the status page and archied status 2026-10-04 03:29:05 +11:00
sam 31715f1a50 refactor(archieplaybooks): fold archie-playbooks into archied playbooks lint|serve 2026-10-04 01:20:33 +10:00
sam 9f4a9e8249 refactor(statestore): own package for the State Store service; access reset moves to archie-state-store 2026-10-04 00:46:11 +10:00
sam 2cb5ae3418 feat(access): archied access reset --org/--instance on the State Store host (archie-core-qc8p) 2026-09-26 15:23:32 +10:00
sam 2a4e764718 refactor(indexing): delete the workspace codesearch indexer
The index built google/codesearch over the chat workspace on every daemon
start, for the process lifetime, through a child-process helper and a
SQLite metadata DB -- and grep consulted it only for searches confined to
the workspace root, which the analysed sessions showed was a small
fraction of searches. The compute was continuous; the benefit was not.

Removes internal/indexing, the workspace-codesearch helper subcommand,
the [indexing] config section, and the GrepIndex accelerator seam from
the grep tool, which returns to its direct walk. google/codesearch
leaves go.mod with it.
2026-09-23 00:36:38 +10:00
sam 3f17821cd8 chore: cleanup incorrect phrasing on package comments. 2026-09-22 21:36:53 +10:00
sam ee608d81fb feat(configuration): add archied setup, the caller the first-run flow was built for
docs/architecture/configuration.md decides that "install.sh must stop generating
config.toml; archied setup owns it", and gives the reason: the installer
hand-rolling TOML already drifted from the schema once, and "the durable fix is
that the code writing the config is the code that reads it". setup.Run,
tomlwrite, configtemplate.Example and terminalprompt were all built for that and
had no production caller between them. This adds the caller.

setup.Run's own doc comment states the obligations, and they are the shape of
installValidatedConfig: render the edits, prove the result loads through a real
configuration.Loader, and only then install the file and commit the secrets. A
validation failure returns before either write, so a bad config leaves neither a
config nor a secret on disk. The rendered bytes are proven from a temp file, so
a load failure cannot have already installed anything. The config is written
0600 via same-directory temp and rename.

Parameterisation is typed, not matched against prompt text. A flag-to-prompt
mapping would have to match strings, which is the brittleness removed in
9a4f11c, so setup.Params is consulted before each question instead and the
prompts are only reached for answers nothing supplied. Run stays a wrapper
passing the zero value, so the package's existing tests keep passing untouched.
No environment variable is consulted anywhere: flags and prompts are the only
two ways an answer arrives.

--defaults is a known-valid unattended baseline, not "each prompt's own default".
That cannot work, and the test proves it: the provider step's first option is
OpenAI, whose model prompt hard-errors on an empty answer. The baseline therefore
covers every required answer and leaves optional ones to answer themselves as
"skip" -- ollama is the only keyless provider, so an unattended install boots
without needing a secret.

Two gaps closed after review rather than left as accepted work:

- A cloud provider was not configurable without a terminal, because the provider
  API key had no typed parameter: unattended setup could only ever pick the
  keyless self-hosted provider. Params.ProviderAPIKey and -provider-api-key fix
  that. The key goes to the env file and TOML carries only the secret reference.
- The write order was unpinned. Inverting installValidatedConfig's two writes --
  committing secrets before the config -- left every test green, so nothing held
  the invariant the SecretSink contract states. The new
  TestInstallValidatedConfig_ConfigWriteFailureLeavesNoSecret fails on that
  inversion; the existing validation-failure test does not cover it.

Also fixed three lint failures in the first pass (gocyclo on RunSetup, and two
ineffectual `choice` initialisers): RunSetup's flag surface, the flag-to-Params
mapping, the prompter choice and the existing-config read are now separate
functions, which is the real fix for the complexity rather than a suppression.

The setup flow no longer hand-rolls its own answer format: -telegram-user-ids
reuses setup.ParseTelegramUserIDs, the same parser the prompt path uses, so the
allowlist cannot be parsed two ways.

Found by an adversarial package sweep (temp tracker: config-1,
repo-root-deployment-2, .local/issue-tracker/). Plan of record:
.local/issue-tracker/design-archied-setup.md. install.sh still generates its own
config; that is phase 2 and is not claimed here.
2026-09-15 02:52:24 +10:00
sam 5fa010c9c2 fix(plugin): add missing CLAUDE.md reference to plugin engine rule
Test TestPluginEngineRuleIsCanonicalAndSharedByAgentInstructions was
missing entirely, letting CLAUDE.md drift out of sync with
ARCHITECTURE.md#plugin-engine-rule-strict. Added the test and the
missing reference.
2026-08-31 21:25:20 +10:00
sam 2395254559 build(deps): upgrade github.com/samcharles93/ai-sdk to v0.1.29 2026-08-27 21:40:01 +00:00
sam 48133e68fe feat(logging): cursor pagination and filtering for task_logs
task_logs could only read the newest N entries with no way to walk the
rest, so a task's log beyond the page cap was unreachable and the agent
had no honest way to say what was missing. Add logging.Page -- a
byte-offset cursor over the tail-most 8 MiB scan window -- plus
Query.Since/Until time bounds, and wire the gateway's task_logs tool to
expose level/since/until/after_id with cursor and more_available in the
result.

The reader now reads the scan window into a single in-memory buffer and
walks it by exact slice offsets. Offsets into a contiguous buffer are
trivially correct, which an incremental reader over the file is not:
bufio.Reader buffers ahead, so its underlying file position runs one
buffer past its logical position and cannot be used as a cursor. The
same rewrite removes f.Read in favour of f.ReadAt (full-or-error), since
a silent short read would drop the newest entries and still report
success.

The scan window is a documented hard ceiling: for a log over 8 MiB the
oldest bytes are unreachable by any cursor -- that is what Truncated
reports, and send_file remains the path for a complete archive. The
tool description teaches the model the difference between more_available
(more this call did not return) and truncated (older entries the scan
cap hides).

Closes archie-core-vq92 (#625).
2026-08-26 02:02:36 +10:00
sam 64da3952aa feat(gateway): add task_action chat tool for operator task management
Adds the task_action tool to internal/gateway/task_tools.go and wires it via
chatTaskActorAdapter in cmd/archied. This enables operators in chat to execute
lifecycle actions (abandon, archive, retry, stop, cancel, approve, reject) on
tasks using the same action path and state machine as the dashboard.

Enforces identity scoping so an instance cannot mutate another identity's
tasks, verifies status permissions via internal/taskstate, and classifies the
tool as mutating with required human approval.
2026-08-25 01:19:21 +10:00
sam 1f05dbb33c feat(deployment): default to managed NATS workers
Autonomous workflows now always hand a complete task to a scoped archie-agent container. Embedded NATS is the default and external NATS changes only broker placement; legacy agent and containers.enabled inputs decode for migration but cannot select an execution path. The daemon flushes task-event subscriptions before a worker can publish, and runtime updates snapshot and restore the configured worker image.
2026-08-24 14:56:57 +10:00
sam d3da153768 refactor(agentexec): remove legacy execution paths
Autonomous workflows now have exactly one production boundary: a task-scoped archie-agent receives a full-task core-NATS request and runs the worker-local stage loop. The legacy [agent] section remains decode-only for migration; it no longer selects runtime behavior.
2026-08-24 14:56:57 +10:00
sam 9d73bc7157 feat(nats): expose embedded broker to managed workers 2026-08-24 14:56:57 +10:00
sam cc53e5258d fix(telegram): compact tool output and drop turn caps
Render completed ToolCallEvents as Telegram-safe streaming blocks
with compact name/status and a short cleaned preview. Summarize
file contents and listings instead of dumping source or quoted
JSON. Collapse repeated legacy turn-budget failures.

Remove max_tokens turn gating and the 200000 turn_budget_chars
default so absent or commented config cannot restore an aggregate
cap. Keep per-result max_result_chars and spill.

Confine find/grep discovery to the active workspace/worktree.

Refs #604
2026-08-23 16:40:16 +00:00
sam 9e46e31759 fix(worktree): bind publication to dispatch grants
Worktree RPC used to accept repository coordinates from the sandbox. It now accepts only an opaque per-dispatch grant, and the daemon derives the repository path and branch from grant-owned state.

Repository worktree paths also move from an ambiguous flattened key to encoded owner/repo components. Origin-matched legacy clones migrate safely; mismatched legacy paths are preserved.
2026-08-22 22:06:54 +10:00
sam 1c975b8348 fix(container): persist npm cache for per-task MCP servers
Ephemeral task containers re-resolve and re-download npx-launched MCP
server packages from the npm registry on every task, since each task
gets a fresh container with no persistent cache -- a worse amplifier
of a registry outage than the daemon-restart case d4bb914b already
fixed.

Extract the daemon's npm-cache env logic into mcp.NpmCacheEnv, shared
by both the daemon's own long-lived npx MCP servers and archie-agent's
per-task ones. Mount a new archie-cache-mcp-npm Docker volume at
/data/cache/mcp-npm on every task container regardless of ecosystem
(MCP servers are daemon-wide config, not tied to the task repo's
language), and point npx-launched per-task stdio transports at it.

Closes archie-core-qk8m.
2026-08-22 21:13:33 +10:00
sam c32319968c fix(mcp): persist npx package cache across daemon restarts
config.docker.toml launches MCP servers with `npx -y <pkg>@<version>`,
and the container has no persistent npm cache, so every daemon restart
re-resolved and re-downloaded the package from the registry -- slow,
and it turns a registry outage or a yanked version into a daemon
outage. It directly amplified the crash loop observed on 2026-07-30,
since each restart repeated the whole download.

configuredMCPProvider now points npx-launched servers (matched by base
name, so an absolute-path npx still qualifies) at NPM_CONFIG_CACHE
under the daemon's own persistent work_dir, plus
NPM_CONFIG_PREFER_OFFLINE=true so a warm cache skips the registry
round trip entirely. Does not cover a package's own separate
downloads (e.g. Playwright/Puppeteer fetching a browser binary via
its postinstall script).

Closes archie-core-1786637490923-43-d4bb914b.

The identical, arguably worse instance in
internal/app/agentworker/mcp_providers.go (every per-task ephemeral
container, not just daemon restarts) needs a persistent Docker volume
that doesn't exist yet, so it's filed separately as
archie-core-qk8m rather than folded into this scoped fix.
2026-08-19 02:08:33 +10:00
sam 7c409aadb5 feat(update): stream live progress and roll back failed updates automatically
The /update command went silent between "starting" and one final message,
gave no version summary, and required manual intervention on failure.

CommandInstaller now streams the installer script's stdout live instead of
buffering it, and parses a structured result (previous/installed versions,
per-component status) from a sentinel line. Because the reference install
script hands the daemon restart off to a detached systemd unit specifically
so restarting archied.service doesn't kill the process that requested the
update, the outcome is reported in two phases: the synchronous build/install
result comes back through Install() itself, and the post-restart health
check/rollback outcome is written by a new watchdog script and relayed by
whichever archied process boots next, via a small JSON handoff file cleared
once relayed.

Also fixes three defects an adversarial review pass found in the first
draft: the caller's request/handler context was passed straight into the
install child process, so a concurrent /restart or a browser disconnect
could SIGKILL it mid binary-copy; Telegram and the web dashboard each
tracked "install in progress" with their own private flag despite sharing
one Service, allowing two installs to race the same .prev backups; and the
watchdog reported rolled_back:true even when no backup existed to restore,
which would have told an operator to stand down while the daemon was still
broken.

internal/webui gains an unauthenticated /healthz the watchdog polls after
restarting archied.
2026-08-19 01:47:29 +10:00
sam edf38eb3e4 refactor: extract daemon bootstrap into phased wiring 2026-08-16 03:12:18 +10:00
sam a47ca4ec78 refactor(archied): extract config overlay boot helper and thread ctx 2026-08-16 01:38:11 +10:00
sam 9d6a11f491 fix(agentworker): forward workflow events across the NATS worker boundary
A workflow run through the container/NATS path never wired
workflow.TaskContext.Bus, so every tc.Emit call (stage_start,
stage_finish, outcome, parked) was a silent no-op. archied's own
in-process path sets Bus and always worked; anything dispatched to an
archie-agent worker did not, which meant the dashboard's per-task
timeline showed "No events yet" for every task regardless of status
or stage -- confirmed live across a Working task and two differently
parked tasks.

Bridges the gap the same way system logs already cross this boundary:
ForwardTaskEvents relays a per-task in-process *events.Bus to a new
NATS subject (SubjectForEvents), and a new daemon-side subscriber
(subscribeAgentEvents) republishes decoded events on the daemon's own
Bus, landing them in the same persist-and-broadcast pipeline an
in-process workflow already used.

Closes #518
2026-08-15 10:34:25 +10:00
sam cad3cf745a fix(config): de-duplicate disabled-forge predicate, export ForgeDisabled
cmd/archied carried a copy of the none/off/disabled predicate because the
configuration package's version was unexported. Two copies of a validation
predicate drift: a fourth disabled alias added in one place would silently
leave the other accepting it.

Export ForgeDisabled in internal/infrastructure/configuration as the single
definition, referenced by both validateIdentities and resolveForge, and
delete the duplicated copy. TestForgeDisabled pins the exact alias set
(including case-variant and empty-string guards); the resolveForge
degradation test now asserts on the warn log so the silent disabled path is
distinguishable from the token-unavailable fallback (verified to fail under
a re-inlined 'none'-only copy).
2026-08-15 01:43:31 +10:00
sam 7ee68d080a feat(archied): add -version so the installed build can be read from a shell
gatewayVersion and runtimeVersion were reachable only through the chat
/version command, which needs a running daemon and a configured channel.
An update-check adapter has to answer "what is installed?" before it can
compare against what is available, and had no way to ask.

Prints one "name version" pair per line so a shell adapter can parse it
without a JSON dependency.
2026-08-10 02:48:59 +10:00
sam 479ed95a91 fix(config): close review blocker on the PATCH path; deep-copy snapshots; atomics for shared state
Second adversarial pass findings, fixed:

BLOCKER -- PATCH silently dropped every dotted key. The UI sends
dotted keys ('budgets.max_steps') but ApplyOverlayValues fed them
straight to yaml.Unmarshal, which cannot decode a dotted key as a
struct field and silently ignored it: 200 OK, zero effect, and the row
still persisted (poisoning later reloads with an unvalidated value).
The reload path worked only because overlay.Snapshot nests first. Fix:
applyDottedOverlay nests dotted keys via overlay.Nest before decode --
the same nesting Snapshot performs -- and is pinned by
TestApplyDottedOverlayNestsBeforeDecode, which fails against the old
behaviour.

MAJOR -- shallow copies let overlay decode mutate the published
snapshot. Holder.Get and *doc are shallow; yaml decodes maps in place,
so a crafted nested update (or a failed boot overlay apply) mutated
shared maps of the published config before validation. Fix: config.Clone
deep-copies every reference field (Models, Providers, Repos, Identities,
Dispatch.Labels, Agent.Env, Chat/Telegram slices, Tools.MCPServers,
Memory.ProviderConfig, WebFetch.Enabled pointer, Extra), used by the
PATCH path (next := d.Cfg.Get().Clone()) and by Loader.ApplyOverlay,
which now never mutates its input (clones Config and the provenance
chain internally), so boot and reload pass the base document directly
and a failed apply leaves it exactly as it was. Pinned by
TestConfigCloneDeepCopiesReferenceFields.

MAJOR -- data races on captured main.go variables. bootOverlayErr and
currentProvenance were plain values written by the SIGHUP goroutine and
the PATCH/Reset handlers and read by every GET /api/config; -race passed
only because no test exercised the composition root concurrently. Both
are now atomic.Pointer; the LastReload closure and the provenance
dedup read them atomically.

MINOR -- MaxConcurrency removed from the reloadable allowlist. The
dispatchers re-read it but the startup-built container pool captures it
(container/pool.go:94,163), so 'every consumer re-reads it' is false and
a change only partially applies. It now warns requires-restart; the
criterion comment on reloadableFields documents the pool capture.

MINOR -- ResetConfig now resolves and validates the target state BEFORE
deleting the row, so a broken file cannot leave the store and the
running config disagreeing; the provenance chain no longer grows
unboundedly (the runtime-overlay origin replaces, not appends).

Also: holder.go no longer references a file watcher that does not exist.

Verified: task check clean; -race clean on cmd/archied, webui, daemon,
config; new findings from the second adversarial pass are closed.
2026-08-10 01:23:34 +10:00
sam 171e1b2686 feat(config): surface overlay state, shadowed keys, and per-row reset
Three gaps closed, all in the 'your change didn't take effect and
nothing told you' family:

1. MaxRetries joins the reloadable allowlist, and the criterion is
   fixed on reloadableFields: a field is reloadable when EVERY
   consumer re-reads it. MaxRetries is the worked example -- its only
   consumer is the webui maxRetriesFor handler, which re-reads per
   request (api_tasks.go:380,383); nothing in the daemon reads it, so
   warning 'requires restart' was spurious. Chat.Telegram.TokenEnv is
   documented as the counter-example (webui reads it per request but a
   startup-built telegram bot consumes it, so it is not reloadable).
   Pinned by TestChangedNonReloadableFields alongside the comment.

2. Boot-time overlay degradation is now surfaced. config.ReloadStatus
   gains OverlayUnavailable; main.go records the reason when the
   overlay store cannot be opened or its values fail validation, and
   /api/config carries it (merged into LastReload). Cleared by a
   successful dashboard write or reset. The settings page renders a
   warning banner, so a daemon running on file config alone says so
   where the operator is looking, not only in logs.

3. Overlay-shadowed file edits are visible and recoverable.
   ConfigView.Overridden lists the dotted keys the runtime overlay
   currently sets, so the UI marks those rows (their file value is
   shadowed until reset). POST /api/config/reset exposes the
   store's Delete as a per-row reset: it deletes the row and
   republishes file + remaining overlay, which is also the answer to
   'my file edit isn't taking effect'. The dashboard can now remove
   every override it can create.

Verified: task check clean; -race clean on cmd/archied, webui,
infrastructure/configuration; new handler tests cover the reset
200/400/503 paths and the overridden list.
2026-08-10 01:09:05 +10:00
sam da2d93a830 feat(config): runtime config overlay store and PATCH /api/config
The second half of #348's storage layer: dashboard-edited overrides
live in their own SQLite file, separate from the task and conversation
stores so each file owns its user_version and migrator with no
contention (configDBPath -> cfg.DBPath + "-config.sqlite", matching
the taskDBPath / conversationDBPath convention).

Store (internal/infrastructure/configuration/overlay):
- config_overlay(key TEXT PRIMARY KEY, value TEXT NOT NULL,
  updated_at, updated_by), own migration, WAL.
- Set refuses DeniedKeys (db_path, work_dir) at write time as well as
  at the API layer; the map is exported so the dashboard can show the
  reason instead of silently omitting the edit affordance.
- Snapshot JSON-decodes each value and nests dotted keys
  ("budgets.max_steps" -> map[b][m]) ready to decode into Config.

Loader (ApplyOverlay / ApplyOverlayValues):
- Decodes a nested overlay map with the same field-level precedence as
  a file overlay, records the runtime-overlay origin, then applies
  defaults and validates. Used at boot, on SIGHUP reload (ReloadController
  gained an optional WithOverlay source) and by the PATCH path.

Boot and reload:
- A broken overlay (unopenable store, unreadable snapshot, values that
  fail validation) degrades to file config alone -- it must never brick
  the daemon. --no-config-overlay / ARCHIE_SKIP_CONFIG_OVERLAY=1 skips
  the store entirely; the flag is now a real gate, not a log-only stub.

PATCH /api/config (webui):
- handleConfigUpdate decodes {"updates": {dotted.key: value}}, calls
  the composition-wired UpdateConfig seam, and classifies errors via
  webui sentinels: ErrConfigUpdateInvalid -> 400 (denylisted key,
  failed validation, unparseable value), ErrConfigUpdateUnavailable ->
  503 (overlay skipped by recovery flag), anything else -> 500.
- The seam (main.go) applies updates to a COPY of the published
  config, validates the materialised result, persists to the overlay,
  then publishes through the same path as reload -- it never calls
  Set directly from the handler, since web.Cfg is the daemon's own
  Holder.
- ConfigView gains Locked (dotted key -> reason) so the UI disables
  denylisted rows explicitly.

Verified: task check clean; new tests cover the store round-trip,
denied keys, loader overlay layering/validation/provenance, reload
with and without overlay, and the PATCH handler's 200/400/503 paths.
2026-08-10 00:54:23 +10:00
sam d8d4c93239 fix(config): share one config Holder between daemon and dashboard; re-apply model catalog on reload
Adversarial review findings (fresh reviewer, no memory of the code):

1. (major) web.Cfg was a second Holder the reload never swapped, so
   /api/config showed boot-time values forever after the first reload
   while the daemon ran the new config -- the exact 'code means
   something different from what it says' failure. web and d now
   share one Holder (web.Cfg = d.Cfg after construction), so a reload
   swaps both views atomically.

2. (minor) the model catalog merge ran once at boot; publishing the
   raw reloaded document dropped catalog-discovered providers from
   the running config even though the file was unchanged. The apply
   closure re-runs applyModelCatalog on the reloaded document before
   publishing (idempotent; a catalog that failed to load merges to
   identity).

3. (minor) changedNonReloadableFields parameter 'new' shadowed the
   predeclared identifier (predeclared lint finding). Renamed to
   'next'; the finding is gone from the branch diff.

Also: tightened --no-config-overlay help text to state the file
-config-overlay still applies (it gates the future DB overlay only),
and documented that the controller's apply callback must be non-nil.

task check clean; -race clean on cmd/archied and internal/daemon.
2026-08-10 00:30:26 +10:00
sam f544dd0978 feat(config): SIGHUP-triggered reload with requires-restart warnings
The daemon can now re-read its configuration on SIGHUP (or on a
future in-process trigger) and republish it through the Holder,
closing the first half of issue #348: 'config can change at runtime,
safely, and can't brick the daemon.'

Reload path:
- cmd/archied/reload.go: ReloadController re-runs loader.Resolve on
  the same paths main.go passed at startup (Resolve is pure -- no
  reload state added to the loader). apply is called only with a
  Document that passed validation; a failed reload records the error
  and its timestamp and leaves the running config untouched. That
  Set-not-called-on-error property is the safety core and is pinned
  by TestReloadValidationFailureDoesNotApply.
- SIGHUP listener in main.go runs reloadLoop(ctx, ch, controller, log);
  the loop is extracted as a testable seam (syscall.SIGHUP is Unix-only,
  fine given the Linux target).

PollInterval is genuinely reloadable now, not just re-read:
the three ticker sites (Run, per-identity goroutine, shared
maintenance loop) re-read the interval each tick and Reset the ticker
on change. A reloaded poll_interval takes effect on the next cycle.

Diff-and-warn so an operator never has to read the source to know
whether an edit took effect: changedNonReloadableFields compares the
old and new snapshots against an allowlist of re-read fields
(PollInterval, Repos, DiffCapLines, Budgets, Dispatch, and the
Containers sub-fields VolumeTTL + MaxConcurrency). Anything else that
changed logs Warn 'requires a restart'; a fully-applied reload logs
Info. The allowlist is deliberate: a field added to Config later
defaults to requires-restart, the safe direction.

NATS split-brain closed: containerEnv now hands containers the
endpoint the daemon's own client connected with at startup
(Daemon.ConnectedNATS, captured at construction) instead of the live
config, so a reloaded nats.url cannot point new containers at a
server the daemon is not publishing on. A nats change is therefore
classified requires-restart, and CLAUDE.md's 'hands its own nats.url
verbatim' note is updated to the meaning it now has (same today,
deliberately different after a reload).

Validation guard: poll_interval <= 0 is rejected in validate.go
beside the VolumeTTL guard. applyDefaults fills zero with 60s, so
this fires on an explicit negative -- which would panic
time.NewTicker/Reset at boot and, after the per-iteration reset,
on a SIGHUP mid-run.

Observability: config.ReloadStatus (last_error, last_error_at,
last_reload_at) is recorded by the controller and surfaced in
/api/config via Server.LastReload, so a stale running config is
visible in the dashboard, not just the logs. Rendering waits for the
UI step.

Recovery hatch: --no-config-overlay flag plus ARCHIE_SKIP_CONFIG_OVERLAY=1
(configuration.SkipOverlay), logged at startup. No overlay source
exists yet; the flag is the documented escape hatch the DB overlay
(step 3 of #348) will consult.

webui.Server.ConfigProvenance becomes atomic.Pointer so a reload can
swap it via SetProvenance independently of Cfg; the accepted skew
(new config alongside old provenance) is documented on the field.

Verified: task check clean; -race on daemon, webui, cmd/archied,
config, infrastructure/configuration.
2026-08-10 00:19:46 +10:00
sam b6f0aea930 refactor(daemon,webui): make Daemon.Cfg and webui.Server.Cfg reload-safe
Daemon.Cfg, webui.Server.Cfg, and telegramSetup.Cfg were all held by
value (or as a raw shared pointer in webui's case), so a future reload
path could not swap the running configuration -- the consumers held
frozen snapshots made at construction. This is the prerequisite for
issue #348.

Add config.Holder, a small RWMutex-backed snapshot holder:
  - Get returns a value copy under RLock; the copy shares its
    reference-type fields with the published snapshot, so a doc
    comment pins the immutability invariant (a published Config is
    immutable; reload constructs a fresh one).
  - Set replaces the snapshot under WLock.
  - Get is nil-safe; Set is not.

Convert every consumer to read through the accessor. The convention
'cfg := h.Get() once per logical operation' is documented on the
type so the next reader knows that inline d.Cfg.Get().Foo is a code
smell (a Set could land between two consecutive Gets).

Special cases:
  - webui.Server.Cfg: was *config.Config, a raw shared pointer read
    from handler goroutines. Convert to *config.Holder. The
    pre-existing 'cfg := s.Cfg' aliasing bug at api_config.go:295
    (which was *trying* to copy but only copied the pointer) now
    becomes 'cfg := s.Cfg.Get()', which is the value copy the
    read-only view always wanted.
  - daemon_test.go: one test mutated d.Cfg.DiffCapLines in place,
    which the Holder forbids. Replace with the documented
    read-modify-write pattern (prev := d.Cfg.Get(); prev.X = Y;
    d.Cfg.Set(prev)).
  - telegramSetup.Cfg: matches the daemon's API for consistency
    even though setup runs before the daemon is alive.

TaskContext.Cfg in internal/domain/workflow/workflow.go is
deliberately NOT converted -- a task that starts under one config
should finish under it, and a config changing between TDD and gate
stages is a bug, not a feature. Snapshotted at dispatch, as before.

Verified: go test ./... (61 packages), go vet, -race on
internal/daemon and internal/config.
2026-08-09 23:29:52 +10:00
sam 61b4b7b0cf refactor(store): replace legacy persistence with SQLite
Task and event persistence now uses a dedicated SQLite sidecar derived from the configured state prefix, matching the independently persisted conversation store. The configured path now means a state-path prefix rather than a database file so fresh SQLite state never attempts to interpret the previous storage format.

Remove the retired implementation and dependency completely, strengthen SQLite migrations and concurrency behavior, restore backend-neutral coverage, and align configuration, operational guidance, UI labels, and deployment documentation with the new storage and host-daemon topology.
2026-08-09 22:27:25 +10:00
sam 3afbab8201 refactor(gateway): persist conversations in sqlite 2026-08-09 19:47:33 +10:00
sam e4385ef266 feat(webui,gateway): add /api/tasks/{id}/logs endpoint and task_logs chat tool
Adds two read surfaces for per-task persisted log history so an operator
can answer "why did task N park?" without needing to watch docker ps in
real time — the deployment model documented in CLAUDE.md.

/api/tasks/{id}/logs — built on logging.Tail the same way /api/logs
already is, defaults attempt to the tasks current Attempt from the store.

task_logs chat tool — lets Archie read task logs from Telegram or webui
chat. Identity enforcement matches chatTaskListerAdapter: a model bound
to one identity cannot read another identitys logs, and forge tasks
(empty identity) are denied.

internal/logging: add TaskRegistry.Path() for read-side access without
threading baseDir separately through every caller.

Fixes from adversarial review: strict identity check (no empty-identity
bypass), non-null Entries when disabled, component doc accuracy, and
adapter tests covering the security boundary.

Ref: #455
2026-08-09 19:14:35 +10:00
sam 951ddbc9f4 feat(daemon): consume agent system logs into per-task sinks and the dashboard feed
Closes the loop #451-#453 opened. archie-agent (#453) now publishes its
own logs to agentexec.SubjectForSystem(taskID) over NATS, and a task
log sink (#452) exists to persist them, but nothing on the daemon side
subscribed, demuxed, or wrote anything -- the messages had nowhere to
land.

- agentexec: SubjectSystemWildcard + TaskIDFromSystemSubject, the
  inverse of SubjectForSystem, for a daemon subscribing to every
  task's system subject with one wildcard subscription.
- logging: TaskRegistry manages one open TaskSink per running task
  attempt, mirroring every write onto the live dashboard Feed the same
  way a locally-written log would appear. Nil-receiver-safe, matching
  every other optional composition-wired Daemon field.
- daemon: Daemon.process opens a task's log sink for its own duration
  (covering both the sandboxed-container and in-process dispatch
  paths uniformly) and closes it on every return path.
- cmd/archied: constructs the registry, subscribes core NATS to
  SubjectSystemWildcard, and wires TaskLogs into both Daemon and
  webui.Server.
- webui: archiving a task now removes its log directory too, so
  archived tasks' logs don't accumulate forever. Documented as a
  known, accepted race: a task can reach terminal status (satisfying
  ArchiveTask's precondition) before process() itself returns and
  closes the sink, so a very late system-log write can race an
  archive's Remove. Not corrupting -- the write lands on a
  soon-to-be-unlinked inode and is discarded -- and not worth fixing
  here: it would mean tying Close to the store transition instead of
  to process()'s return, a larger change than this issue's scope.

Adversarial review (fresh subagent) found and fixed two real issues in
the first draft: a doc comment accidentally spliced between two
functions during an extraction meant to fix a lint regression, and a
genuine test gap on openTaskLog (added, using TaskRegistry.Write's own
true/false contract as the observable -- no Store/Forge/Trees fakes
needed, since openTaskLog only touches TaskLogs and Log).
2026-08-09 18:42:54 +10:00
sam 4131f05036 feat(webui,gateway): wire config, channel, log-feed, and work-intake seams
Composition root work tying the config resolver, channel manager, log
feed, and executable workflow registry into the running daemon and
dashboard server: web.Channels/LogFeed/Workflows/ConfigProvenance,
channel-reload wiring, TaskStopper for the Stop action, and
SpawnRequest.Body/WorkRequests threading dashboard "Start work" requests
through the same admission path chat uses. Touches all three features at
once because this is where they're composed, not owned by any single one.

Refs #53, #54, #55
2026-08-09 01:26:58 +10:00
sam 6aa3772ce9 feat(gateway): tool-approval gate, session tools for chat, and /delete command
Adds a human-consent gate for tools marked RequiresApproval, exposes
session management as LLM-invokable chat tools, and gives the operator
/delete to retire conversations permanently.

**Tool-approval gate (internal/tools, internal/agentexec)**

- ApprovalRequester contract lives in internal/tools (not gateway) so
  agentexec can import it without pulling in the chat-adapter stack.
- ToolSetOptions.Approval wires the approver through to toolExecute.
- BuildToolSetFrom omits RequiresApproval tools when no approver is
  configured; dispatch paths (invokeTool, tool_call bridge) refuse
  them too, so the invariant holds at every indirection.
- The gate fails closed: only Approved/PermanentlyApproved pass;
  an unknown decision value from a faulty approver does not execute.
- Tool-level timeout (2 min) returns a plain error, not a context
  cancellation, so the model can report "approval timed out" instead
  of the turn aborting.
- BuildApprovalDescription callback lets a tool surface the specific
  resource being acted on ("Delete session \"abc123\"") rather than
  showing a static tool description.

**Session tools (internal/gateware/session_tools.go)**

- session_list (idempotent), session_resume (mutating), session_title
  (mutating), session_delete (mutating + RequiresApproval).
- Scoped to the current platform + channel: GetByChannel replaces
  List() so a model in chat A cannot list or manipulate chat B.
- session_title validates the session belongs to this channel before
  writing; session_resume/session_delete resolve refs against the
  channel-scoped list.
- Shared limit helper extracted to tools.ListLimit.

**Telegram approval adapter (internal/channels/telegram/approval.go)**

- telegramApprover implements ApprovalRequester via the three-button
  inline keyboard (Approve/Permanent/Deny). 2-min token expiry.
- Own map + mutex + "approval:" callback prefix — separate from the
  dangerous-command system.
- Permanent approval uses exact-match (not prefix) lookup keyed by
  action + resource description, so permanently approving "delete
  session abc123" only covers that one session, not every call to
  session_delete for 24h.

**Session lifecycle**

- /delete <session-id>: removes a session and its history. Refuses
  prefixes shorter than 4 chars even when unique. Exact ID wins over
  an ambiguous prefix. sessionTracker.forget sweeps every channel+
  thread pointing at the deleted session.
- Extracted resolveSessionRef and shortSessionID for /resume and
  /delete to share.

**Wiring**

- submitTurn builds the approver onto ctx before the LLM turn.
- chatGenerateOptions reads approver from ctx, passes it through
  ToolSetOptions.
- SessionTools built per turn on the current message, combined with
  taskTools into a single toolset.
2026-08-08 16:42:59 +10:00
sam 1081a6c3ed feat(daemon): identity-scoped isolation + gateway session titles and /delete
Multi-identity daemon isolation:
- Split runIdentities into per-identity poll loops plus one shared
  maintenance-and-drain loop, so reconcilePRs and storage cleanup run in
  multi-identity mode and the global max_concurrency / per-repo
  serialization holds across identities.
- Make the poll path identity-aware: each identity polls with its own
  forge client, bot user, label, trigger, and ack reaction.
- Make allowConcurrentFor task-based and identity-aware via repoFor.
- Roll back storage mounts when container acquisition fails; log
  Store.Update failures after branch set.

Container-mode RPC identity routing:
- forgerpc/worktreerpc gain identity-scoped subjects (SubjectFor,
  RegisterFor) and identity-carrying clients.
- registerTaskRPCServers registers one server pair per identity (plus
  root), so a container-mode task owned by a non-root identity is served
  by its own forge client and worktree manager.

Gateway/Telegram:
- LLM-proposed session titles generated in the background after turns
  (TitleGenerator + chatTitleGenerator), persisted by the gateway.
- /delete session command with prefix guard and tracker forget.
- Shared session display helpers (shortSessionID, sessionDisplayTitle,
  resolveSessionRef).
2026-08-08 01:32:35 +10:00
sam 7e896c9411 refactor: migration of workflow into it's own domain 2026-08-07 00:24:05 +10:00
sam 308666b410 chore: remove legacy terminology and dead code
- Reword docs and comments to drop vague "legacy" qualifiers where they no longer describe deprecation status
- Remove unused code: RecentEvents, MessageEvent.ToLegacy, index setState, stderr redaction tool, telegramOnlyCommands
- Delete or comment out tests covering removed or superseded behavior
2026-08-06 22:55:26 +10:00
sam e10cbbc92c refactor: pre-cleanup. 2026-08-06 01:40:15 +10:00
sam 36d00963a5 feat(curator): trigger accounting — primary-input wake path (archie-core-035)
A completed chat turn is published as turn_completed from the single
responder choke point shared by the telegram and web gateways
(redeliveries return earlier and never emit). The runtime wake path
forwards ONLY primary-input kinds to input-driven curators; curator-
produced activity (curator_run/action/error and any future derived
kinds) never appears on that path, so a curator that writes memories
cannot schedule itself again — the feedback-loop guard, with the
rationale recorded in the regression test so it is not removed as
redundant.

The bus subscription is bounded and dropping, so the wake path can
never backpressure the chat publisher; a dropped event only delays a
run to the next check-in.
2026-08-05 15:00:29 +10:00
sam 4ece064026 feat(curator): engine family contract, registry, and runtime (archie-core-rfp, archie-core-89x)
internal/curator is a conforming engine family per the strict plugin
engine rule: typed CuratorEngine contract (Lifecycle + domain ops),
owning Registry (registration, discovery, start/health/stop with
failure isolation, reverse-order shutdown, ctx-bounded stop), and a
narrow typed Registrar — each curator receives a capability-filtered
view of the host at Bind, never the daemon.

Runtime: one loop per curator on an injectable clock (fake-clock
testable), best-effort wake nudges with the trigger decision kept in
Check (a dropped nudge only delays to the next check-in, never
corrupts accounting), per-pass timeout and a max-concurrent-passes
slot so curator bursts cannot starve chat turns, per-pass panic
recovery, and bounded shutdown even when a pass ignores cancellation.
curator_run / curator_action / curator_error ride the in-process
event bus so activity stays attributable.

Wired in cmd/archied/main.go (constructed beside memory, Start after
daemon startup, shutdown ordered before memory/LLM teardown) and
exposed as daemon.Daemon.Curators.
2026-08-05 14:54:12 +10:00
sam 68ba3456ee feat(webui): chat from the dashboard
Brings the chat surface to the Web UI: /api/chat with its own handlers
and tests, a self-contained ui/src/chat feature folder, and the shared
plumbing to reach the same router Telegram uses.

Includes a dangerous-command approval path (internal/gateway/dangerous.go,
internal/webui/dangerous.go, and the Telegram counterpart) so /rollback
and /stop are gated the same way on both surfaces, persona handling for
web sessions, and release-update formatting split out of the Telegram
service so both can render it.

web_parity_test.go pins that the two surfaces expose the same commands.

Authored in a parallel session; committed here so it lands with the rest
rather than sitting uncommitted while other work builds on top of it.
task check passes with it in place.

Screenshots used while building are gitignored (*.png) rather than
committed, matching the call made on the earlier design reference.
2026-08-05 10:48:51 +10:00
sam fa6f4aef5f fix(gateway,daemon): repair regressions found by the adversarial review
Six defects, four of them introduced by the fixes in dbacb96, 44d43ef,
d5e016e, de1a899 and 4d44ef7. This is the third round in this codebase
where a round of fixes broke something the next round caught.

SANDBOX SILENTLY DROPPED. startContainers degrades a failed container
pool to nil, and the daemon's only test was `if ContainerPool != nil`.
So `containers.enabled = true` plus a missing agent image ran every
task's agent loop in-process, with the daemon's own authority and the
host filesystem -- the one thing [containers] exists to prevent -- while
logging only that containers were disabled. Nil now means two things and
they are distinguished: SandboxRequired parks the task instead. The
daemon still stays up; chat and the dashboard are unaffected.

SUMMARY AT THE WRONG END OF HISTORY. compressedHistory returns [head,
summary, tail], but ordering is by timestamp, not slice position. The
retained messages keep their original timestamps -- that is what
preserves their identity -- so the summary, being new, was clamped to
now and sorted after the tail it was meant to precede. The model read
the recent conversation and was only then told earlier context had been
summarised. Confirmed on both backends: index 23 of 24.

The monotonic clamp belongs to the append path, where an inbound message
must not sort ahead of the one that provoked it. ReplaceMessages is a
rewrite primitive and now honours explicit placement, and the summary
takes a timestamp inside the span it replaces.

A COMPRESSION SUMMARY ANSWERED A QUESTION. Following from the above,
PriorReply saw a bot-attributed record immediately after an unanswered
message and replayed it as the answer: a redelivered question was
"answered" with the compression banner and the model never ran. The
user's question was silently swallowed.

RECENCY STILL DIVERGED. sessionRecency preferred LastActiveAt whenever
set; SQLite used MAX(last_active_at, created_at). Those agree only when
LastActiveAt is the later of the two, so a clock step or an older record
made the caller and the store order the same sessions differently.

LASTACTIVEAT WAS NEVER UPDATED. Touch had no caller, so "resolve the
most recently active session" meant the most recently created one -- an
operator who moved to an older session with /topic and worked there was
dropped back into the newer, abandoned one on restart, which is the bug
d5e016e claimed to fix, with the direction reversed.

/branch WITH NO ACTIVE SESSION PANICKED, slicing an empty parent ID for
its default title. Pre-existing, but reachable: email and webhook call
Route directly without resolving first, and the email path runs in a
goroutine with no recover, so one message took the daemon down.

The sandbox guard has no test. The honest reason is that exercising it
needs a daemon with a forge and worktree the current harness does not
build, and a test that re-states the condition proves nothing -- which
is the same false confidence that let these regressions through.
2026-08-05 04:24:16 +10:00
sam 1f2e7a42eb refactor: one definition of what approving or declining a task means
The dashboard and chat both let an operator approve or refuse work, and
they had drifted. Declining from the dashboard left the task
closed_wont_do; cancelling the same task from chat left it rejected. Two
operators looking at one queue saw different states for the same
decision, and "rejected" -- which reconcilePRs also uses for a pull
request closed without merging -- stopped meaning one thing.

internal/gateway deliberately does not import internal/store, so it kept
a hand-synced copy of the status strings with a comment asking future
editors to keep them aligned. That is the same defect one level down,
and it is how the drift happened.

internal/taskstate is a leaf package with no dependencies holding the
lifecycle vocabulary and the rules for operator actions. gateway, store
and the dashboard all import it, so gateway stays decoupled from store
without a copy, and CheckApprove/CheckRetry/CheckDecline have one
implementation. Declined is named for what it is -- a human said no --
and stays distinct from Rejected, which is something archie observed.

Chat cancel now records Declined. That is the behaviour change: same
decision, same state, whichever surface the operator used.

Rules were preserved rather than tidied while unifying. A parked task
still cannot be declined, which both surfaces already refused, even
though "not terminal" would have allowed it -- widening a rule under
cover of a refactor is how you ship a change nobody reviewed. The
comment says where to change it if that is ever wanted.

TestDashboardAndChatAgreeOnTerminalStates drives both real surfaces
against one store, so this cannot drift apart again without a red test.

Also removes a second rule for the dash-top-standalone class: renderSetup
already reconciles it from the data at the top of every call, and a later
branch cleared it again. Harmless, but it reads as two sources of truth
for one bit of layout state.
2026-08-05 04:15:39 +10:00
sam 170fe0250b fix: close the duplicate-work loop and the dead paths around it
Applies the four-reviewer pass over the forge/dashboard-actions work.
The through-line is that several capabilities were wired but inert, and
one of them could cost real work.

DUPLICATE WORK. "Closes #N" was dropped from PR bodies in favour of
LinkBranch, which is sidebar linkage on Gitea and a no-op stub on
GitHub -- neither closes anything, and nothing else did either. So a
merged task left its issue open, labelled and assigned; the next poll
re-enqueued it, and once the dashboard's Clear removed the task row that
became a second implementation and a second PR for work already merged.
reconcilePRs now closes the issue on merge, and a rejected task closes
its issue too. A PR closed without merging deliberately leaves the issue
open: that usually means "try again".

MAX RETRIES. Configured, defaulted and reported by /api/config, and
enforced nowhere -- markDead was unreachable and EffectiveMaxRetries had
no callers, so the dashboard's retry incremented retry_count forever.
The retry action now retires a task to dead at the cap.

DEAD ON ARRIVAL. Tasks() never selected plan or source, so the
"Decision required" panel could not render and chat tasks -- whose issue
number is synthetic -- linked to forge issues that do not exist.

COSMETIC FAILURE, FATAL RESULT. OpenPR failed the whole stage when
LinkBranch errored, so the pull request was never opened and the task
parked; a retry re-links the same branch, which Gitea answers with a
conflict, so every retry parked again. It is best-effort now.

INVISIBLE ACTIONS. Approve/retry/reject wrote store rows and nothing
else, so the timeline and activity stream never showed that a human had
intervened. They emit events now.

Also: store write failures answer 500 rather than 409; the dashboard
surfaces the server's actual reason instead of "409 Conflict"; action
buttons stop propagation (acting on a task no longer also expands it),
disable in flight, keep per-task errors instead of one shared string,
and confirm before rejecting.

Removed: the maybeRetryParked shim and markDead with their now-vacuous
tests, WaitingTasks (no consumers; its tests were rewritten to assert
the same routing through TaskByID), write-only Daemon.Runtime, and
dir/dirs response bloat from /api/skills. skillSource checked workDir
before sharedDir, mislabelling every shared skill when the two are equal.
DefaultRoots now dedupes roots differing only by a trailing separator.

The air build rule claiming to keep ui/dist current never ran -- it
required one file to satisfy include_dir and include_file together, and
ui/ is excluded from the walk. Deleted with the comment that promised it.
task test:ui joins task check and CI; it ran nowhere before, so the
dashboard suite could rot silently.

Docs corrected where they described behaviour that no longer exists:
parks no longer comment on the issue, state labels no longer drive
retries, and forgerpc's doc claimed four proxied methods while listing
three of five.
2026-08-05 03:54:25 +10:00