Commit Graph
337 Commits
Author SHA1 Message Date
sam 39d8b067ec feat(agentexec,workflow): surface tool calls on the task timeline
archie-core-467 wired tool-call visibility into the interactive chat
gateway, but a workflow task's own transcript never showed what its
tools actually did -- only stage/agent-finish events existed, so
there was no way to confirm an agent's summary matched what it really
ran.

Adds a ToolCallReporter, invoked once per completed call from
toolExecute (the single choke point every tool call passes through
before reaching a model), reported via a new ToolSetOptions.OnToolCall
hook. Runner.Run grows a report parameter to carry it in: InProcessRunner
wires it through, since it executes in the same process as its tools;
SubprocessRunner and NATSRunner accept and ignore it, since their tool
calls happen in a different process with nothing here to observe them.

The workflow engine turns each report into a tool_call event on the
task's own Bus (agent.go, implement.go), riding the same NATS
event-forwarding archie-core-518 already built to get container
events onto the daemon's timeline.

Scoped to the central tool registry (read/write/edit/grep/shell/...)
built via BuildToolSet; capture tools (structured finish output) and
run_go_script build their ToolSet entries directly and are not yet
covered.
2026-08-15 12:53:41 +10:00
sam 8f73c1febb fix(agentworker): back off before refetching a nak'd stage message
On a stage-handle failure, the main loop logged the error, called
Nak (which requests redelivery), and looped straight back into
FetchStage with no delay -- unlike the fetch-error branch three lines
above, which already backs off for a second. A deterministic failure
(a broken gate, a malformed request) therefore refetched and failed
again immediately, flooding the log with one "handle failed" line per
loop iteration instead of per genuine retry -- hundreds of identical
lines within the same second in production.

Adds the same time.Second backoff after Nak.
2026-08-15 12:53:31 +10:00
sam a37f126f79 fix(logging): resolve error attrs to their message string, not {}
SystemLogHandler and FeedHandler both json.Marshal Entry.Fields
directly rather than through slog's own JSON handler, so a logged
`error` value went straight into a map[string]any as the raw
interface. encoding/json has no Error()-aware special case, and the
concrete type behind almost every Go error (errors.errorString,
fmt.wrapError, ...) holds only unexported fields, so it marshalled to
`{}` and silently discarded the message everywhere it was logged --
including on every container/system-log line shipped to the daemon
and shown in the webui, which is where this was actually noticed: a
flood of `"err":{}}` with no way to see what had actually failed.

attrValue now resolves an error attr to its .Error() string before it
reaches the map. It also guards against the classic Go footgun where
a non-nil error interface wraps a nil concrete pointer (var e *T; var
err error = e) -- a naive err != nil check would call Error() on a
nil receiver and crash the one thing that must never crash: logging
itself.
2026-08-15 12:53:24 +10:00
sam 37792c9ec0 fix(worktree): force checkout so retry recovers from a dirty worktree
refresh checked out the task branch without Force before hard-resetting
to base. A prior attempt's leftover uncommitted changes -- a killed
container, or a stage that edits files before ever reaching a commit --
made go-git's default MergeReset checkout refuse to reconcile them, even
with HEAD already on the correct branch and the target commit unmoved.
The fallback then made it worse: any checkout failure was assumed to
mean the branch didn't exist yet, so it retried with Create: true, which
failed with a much more confusing error since the branch did exist --
"a branch named ... already exists" -- masking the real cause and
parking the retry with the same unhelpful message every time.

Reproduced directly against a live deployment's actual broken worktree
(modified tracked file, untracked file, branch already checked out) and
confirmed retry then succeeds and correctly restores to base.

Force is safe unconditionally here: refresh hard-resets to base a few
lines later regardless of what checkout leaves behind.

Closes #523
2026-08-15 11:22:22 +10:00
sam 54be74ae89 feat(releaseupdate): stamp install type at build time, fail closed on unknown
/update could report success at the correct target version while the
archie-agent container running it never actually changed: the update
mechanism trusted the configured installer's exit code and reported
whatever version number the release claimed, with nothing verifying
the two agreed.

The first piece of the fix: a new internal/installtype package
reports how a binary was built for distribution (Type(), defaulting
to "unknown"), stamped once per release artifact via
"-ldflags -X .../installtype.buildType=<value>" rather than guessed
at runtime by probing the environment -- restic and Tailscale both
treat "which install shape is this" as a build-time decision for the
same reason: a probed signal is guessable, a compiled-in one is not.

releaseupdate.Service.Install now refuses to run the configured
Installer at all when InstallType is unset or installtype.Unknown, so
an unstamped or misconfigured build can no longer silently attempt an
update it has no way to know is even the right kind for how this
instance was deployed. Wired through every real archied build path so
the gate doesn't just exist in principle: Taskfile.yml's build task
(binary), Dockerfile.archied (container), and install.sh (binary) all
now stamp it; cmd/archied/telegram_setup.go wires
Service.InstallType from installtype.Type().

Configuration-page display of install type/version and post-update
verification against the actually-running reference (not just the
installer's reported version) remain open -- this lands the
correctness gate, not the full observability surface.

Refs #522
2026-08-15 10:34:46 +10:00
sam a253d3043b fix(webui): filter turn_completed noise from the Live Activity SSE feed
turn_completed fires on every chat turn so input-driven curators can
wake on it, but it carries no task correlation (TaskID always 0,
Detail a raw session ID) -- on the dashboard's "Live activity" panel
this drowned real task-lifecycle events: 27 of the last 30 rows in a
live sample were content-free turn_completed noise.

Excluded at the SSE boundary (sseVisible), not at publish, so
curators reading the Bus directly are unaffected. since still
advances past a filtered event exactly as it would for a sent one --
EventsSince only ever returns events with id > since, so leaving
since behind a filtered event would make the next catchUp refetch it
forever.

Closes #521
2026-08-15 10:34:33 +10:00
sam 9d6a11f491 fix(agentworker): forward workflow events across the NATS worker boundary
A workflow run through the container/NATS path never wired
workflow.TaskContext.Bus, so every tc.Emit call (stage_start,
stage_finish, outcome, parked) was a silent no-op. archied's own
in-process path sets Bus and always worked; anything dispatched to an
archie-agent worker did not, which meant the dashboard's per-task
timeline showed "No events yet" for every task regardless of status
or stage -- confirmed live across a Working task and two differently
parked tasks.

Bridges the gap the same way system logs already cross this boundary:
ForwardTaskEvents relays a per-task in-process *events.Bus to a new
NATS subject (SubjectForEvents), and a new daemon-side subscriber
(subscribeAgentEvents) republishes decoded events on the daemon's own
Bus, landing them in the same persist-and-broadcast pipeline an
in-process workflow already used.

Closes #518
2026-08-15 10:34:25 +10:00
sam 8ae23164bb refactor(webui): decompose handleSSE to cut cognitive complexity
handleSSE mixed connection registration, backlog catch-up, and live
broadcast relay into one function with three nested closures --
gocognit flagged it at 52 against a limit of 30. Extracted each
concern into its own top-level piece: sseSince (resume-point
resolution), registerSSEConn (subscribe/unsubscribe), and a new
sseStream type carrying the per-connection state (send, catchUp,
sendPage, drain) that the closures used to capture implicitly.

Behavior is unchanged -- catchUp/sendPage's page-boundary and
target-reached logic is the same control flow the original closure
had, just named and independently testable. handleSSE itself is now
a straight-line sequence of calls.

Added direct unit tests against sendPage/catchUp for the boundary
cases a full HTTP round-trip test doesn't exercise: a target reached
mid-page vs. at a page boundary, a target of zero (drain everything),
a write failure stopping the page immediately, and backlog spanning
more than one EventsSince fetch.
2026-08-15 02:24:43 +10:00
sam 95bb4b5bb6 fix(telegram): use NewRequestWithContext to satisfy noctx
Pre-existing lint finding, unrelated to any behavior change.
2026-08-15 02:19:57 +10:00
sam c2b6823eba fix(telegram): guarantee no message is stranded on gateway stop/restart
A daemon shutdown or /restart cancels turnCtx without any guarantee
that the goroutine owning a liveReply ever runs again to notice and
clean up after itself -- a full process exit gives it no chance at
all. Gateway now tracks every liveReply mid-turn in a registry and
Stop drains it, marking each as interrupted so nothing is left ending
in the "typing" cursor forever.

finalize and abandon now run their bodies inside a per-reply
sync.Once (the terminal field): a turn finishing naturally and a
concurrent Stop draining the same reply were reaching it from two
goroutines at once, both sending their own edit for the same message
with the outcome decided nondeterministically by which HTTP response
Telegram processed last -- including a complete real answer getting
silently overwritten by the restart marker. Only one side's body now
ever runs; the other blocks on the winner and then does nothing.

The drain itself is bounded (liveDrainTimeout, default 5s): a single
stalled Telegram API call must not hang gateway, and therefore
daemon, shutdown indefinitely. Entries still in flight past the
deadline keep running in the background rather than blocking Stop.

A liveReply created in the narrow window between Stop claiming the
registry and its own registration call is abandoned immediately
instead of being registered into (and lost from) a registry nothing
will drain again.

Regression tests cover the mutual-exclusion race directly (20
iterations of finalize and abandonRestarted racing on the same
reply, asserting exactly one outcome ever wins), the bounded drain
against a deliberately hung API call, and the post-stop registration
gap.
2026-08-15 02:19:53 +10:00
sam ddfc8a7b91 fix(config,webui): make show_tool_calls one cross-channel setting
show_tool_calls gated Telegram's tool-activity narration only; the
web dashboard's SSE chat stream narrated every completed tool call
unconditionally with no way to turn it off, leaking tool names and
parameter shapes to anyone with dashboard access even when an
operator had already opted out for Telegram.

Moved ShowToolCalls from TelegramConfig to ChatConfig so it is one
setting for every channel, and wired the webui stream to it:
chatStreamSink now carries a showToolCalls flag (snapshotted once per
turn, mirroring how Telegram's liveReply snapshots it) and ToolCall
no-ops when it's off.

BREAKING: an existing [chat.telegram] show_tool_calls = true in
config.toml now parses as an unrecognized key under [chat.telegram]
and is silently ignored (TOML decoding is non-strict) rather than
erroring -- move it to the top-level [chat] table.
2026-08-15 01:57:06 +10:00
sam c9ae131317 docs(agent): correct markSafe's doc comment to its actual existence check
The comment claimed a mount was checked before touching the
operator's global gitconfig; the code only ever checked os.Stat
(existence + directory-ness). A directory that merely exists at the
mount path -- a stale leftover, never actually bind-mounted this run
-- already satisfied the guard and got added to safe.directory,
exactly what the comment said it prevented. A real mount check would
need platform-specific mountinfo parsing for an inert failure mode
(the entry does nothing unless a git repo later exists at that exact
path), so this corrects the contract to match the code and pins it
with a test instead of changing behavior.
2026-08-15 01:48:39 +10:00
sam b6505f0232 docs(telegram): note the live-edit timestamp trade-off on liveReply
The finished reply is delivered by editing the message created at the
first streamed frame, and editMessageText does not change a
message's date, so a long tool-using turn's answer keeps its
turn-start timestamp and can sort before conversation that produced
it. Delete-and-resend would fix the ordering but reintroduce the
flicker/reorder streaming exists to avoid, so this documents the
accepted trade-off rather than changing behavior.
2026-08-15 01:48:33 +10:00
sam d43416f2fe fix(webui): style tool-call failure from a structured field, not a string prefix
chatStreamEvent gained a Failed bool, set from ToolCallEvent.Err !=
"" -- the field the rest of the codebase already treats as
authoritative -- rather than the browser sniffing summary.startsWith
"failed:" on rendered text. A successful tool whose own output starts
with that literal string (a grep hit, a log line read back) was being
styled as a failed step.
2026-08-15 01:48:29 +10:00
sam cad3cf745a fix(config): de-duplicate disabled-forge predicate, export ForgeDisabled
cmd/archied carried a copy of the none/off/disabled predicate because the
configuration package's version was unexported. Two copies of a validation
predicate drift: a fourth disabled alias added in one place would silently
leave the other accepting it.

Export ForgeDisabled in internal/infrastructure/configuration as the single
definition, referenced by both validateIdentities and resolveForge, and
delete the duplicated copy. TestForgeDisabled pins the exact alias set
(including case-variant and empty-string guards); the resolveForge
degradation test now asserts on the warn log so the silent disabled path is
distinguishable from the token-unavailable fallback (verified to fail under
a re-inlined 'none'-only copy).
2026-08-15 01:43:31 +10:00
sam baef64163a fix(telegram): keep tool activity visible and mark failed/empty turns
Three more P2s from the same code-review pass: the live frame's
single clamped buffer mixed tool lines and answer text chronologically,
so early tool lines silently scrolled out of the frame once the answer
grew past liveBodyMaxRunes and only reappeared at finalize (which
already led with tools separately). Tool lines are now tracked
exclusively in toolLines; a new framedText helper keeps the tool block
whole and clamps only the answer tail, used consistently by the live
frame, an abandoned frame, and the finished message.

A tool-only turn with no reply text used to finalize as a bare,
immutable tool line indistinguishable from a real (if terse) answer;
finalText now appends an explicit "(no response)" marker in that case.

A provider error used to funnel into the same clean-partial path as a
deliberate /stop, so a failed turn was indistinguishable from a
successful short one. abandonFailed appends a "turn failed" marker
before dropping the cursor; /stop still goes through the unmarked
abandon path.

Regression tests cover all three paths plus the degenerate case where
a tool block alone exceeds the frame budget, verified red-then-green
against the pre-fix code.
2026-08-15 01:25:20 +10:00
sam 240c91a117 fix(telegram): harden live-reply delivery against /stop and multi-byte content
Three P1s from code review of PR #476: finalize sent the authoritative
reply through the same cancellable turnCtx a /stop kills, so a stop
landing mid-finalize stranded the live message with its cursor
forever; abandon had the identical hazard, saved only by its one
caller wrapping the context correctly. Both now strip cancellation
internally (context.WithoutCancel) rather than trusting callers to
remember.

Tool activity lines were composed unescaped into the same Markdown
body as the model's own reply, so a stray '*', '_', or backtick in
tool output/parameters could unbalance the whole message and take the
reply's formatting down with it. ToolCall now runs name, parameters,
and summary through bot.EscapeMarkdown before joining the line.

splitLongMessage sliced oversized lines on byte offsets, cutting
mid-rune for any multi-byte UTF-8 line crossing the 4000-byte bound
(CJK, emoji) and producing invalid UTF-8 parts. It now counts and
slices on runes throughout.

Regression tests cover all four paths, verified red-then-green
against the pre-fix code.
2026-08-15 01:06:27 +10:00
sam d28ca099c5 refactor(agent): move worker runtime into application layer
cmd/archie-agent used to own worker construction, NATS lifecycle, task execution, MCP setup, and workflow wiring. It now owns process input only and invokes internal/app/agentworker, while transport-specific NATS, boot-brief, and git-safety mechanics live under infrastructure.
2026-08-15 00:43:12 +10:00
sam dd287d3336 Merge pull request #478 from samcharles93/chore/bump-ai-sdk-v0.1.21
chore(deps): bump ai-sdk to v0.1.21
2026-08-15 00:29:27 +10:00
sam 566beff1d6 chore(deps): bump ai-sdk to v0.1.21
ai-sdk v0.1.21 wraps the cancellation cause in aborted-stream errors
("%w: %w") instead of formatting it, so errors.Is(err, context.Canceled)
is reliable again. The Telegram channel was compensating with a comment
explaining why errors.Is could not be trusted.

The code was already correct; only the comment was obsolete. It now
states the real reason for context.WithoutCancel(turnCtx): turnCtx is
dead on both stop and fault paths, so the cursor-drop edit needs a live
context. The runtime effect of the bump is that a /stop landing inside
the step loop is now logged as "chat turn stopped" rather than
"route failed".

Refs ai-sdk-0pn
2026-08-15 00:24:33 +10:00
sam 0bbd6e75de fix(chat): harden inline tool call streaming
Telegram streaming callbacks now enqueue cancellable rendering rather than performing network I/O while draining the authoritative model stream. Final edit failures fall back to normal split delivery.

Completed tool events now correlate inputs by call ID and expose only bounded, conservative parameter shapes. Tool-call visibility is snapshotted per reply so reloads cannot change an in-flight turn.
2026-08-14 04:52:48 +10:00
sam 0d73f2fa5a feat(chat): show tool calls inline and stream replies into one message
Chat returned only the final answer, so a wrong tool choice, a
hallucinated action, or a footgun like `systemctl stop` against
`restart` was invisible from the conversation. Only text deltas were
threaded through the turn, and tool calls executed inside Generate with
nowhere to report.

Replace the delta callback with gateway.TurnStream, which carries text
fragments and completed tool calls on one ordered path from the model
loop to the channel. The sequence is the point: a tool line that
overtakes the sentence it interrupted describes a turn that never
happened. Text-only callers use gateway.DeltaFunc and are unaffected.

A tool is reported on its result, never on its call  --  the call part
carries no outcome yet, so reporting both would list every tool twice,
once with nothing to say.

Telegram renders it behind [chat.telegram] show_tool_calls, off by
default: this is debugging visibility, and it narrates every internal
step to provide it. The dashboard always shows it, being the operator's
own observability surface; its tool lines are live-only and are not
persisted with the conversation.

The Telegram live reply changes mechanism, and with it what a frame
MEANS. It used to stream through sendMessageDraft: drafts are ephemeral
30-second previews, so a tool call longer than that outlived the
preview, and successive frames were observed concatenated with their
cursors intact. It now sends one real message and edits it. An update
used to be "the next preview of the answer"; it is now "the whole
answer as it currently stands, replacing what is there". The cursor is
appended at render time and never stored, so exactly one exists and
always at the end, and a frame too long for one message is cut to its
tail rather than rejected outright.

A stopped or failed turn now edits the cursor away instead of leaving
it blinking on a message nothing will finish. That cleanup runs on a
live context because turnCtx is already dead, and it runs on both
failure branches: ai-sdk raises an aborted stream as ErrAborted with
ctx.Err() formatted in rather than wrapped, so errors.Is cannot be
trusted to tell a /stop from a fault.

Closes #467
2026-08-14 02:36:12 +10:00
sam d1ce92043d Merge pull request #465 from samcharles93/alert-autofix-5
Potential fix for code scanning alert no. 5: Open URL redirect
2026-08-12 19:22:11 +10:00
sam 9b52cc4923 fix(webui): preserve encoded redirect path characters 2026-08-12 19:19:48 +10:00
sam a56f95d967 forge/github: implement CreateIssue to un-stub the method
Replaces the 'not yet implemented' stub with a real call to
go-github's Issues.Create, mapping title, body, and labels.
2026-08-12 13:49:20 +10:00
samandCopilot Autofix powered by AI 0ba27935bd Potential fix for code scanning alert no. 5: Open URL redirect
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-08-12 11:40:21 +10:00
sam c06dd9c668 fix(gateway): harden session recovery paths
Repair future-dated conversation history on SQLite open and cap newly supplied timestamps so one bad platform clock cannot pin a session indefinitely.

Make message and turn identifiers injective while preserving legacy redelivery and crash-recovery compatibility, retire idle turn lanes safely, and pin the existing compression and partial-undo behavior with focused regressions.
2026-08-10 02:03:19 +10:00
sam 3802ed473d fix(config): preserve nil slices in Clone to avoid spurious reload warnings
Clone normalized nil slices to empty non-nil (make([]T, 0)), so after a
dashboard PATCH the cloned published config carried [] where the file
decodes to nil -- a subsequent SIGHUP reload's diff-and-warn compared
old [] against fresh nil and reported 'Repos changed; requires a
restart' for a field that had not changed at all. The exact warning
fatigue the allowlist exists to prevent. cloneRepos/cloneIdentities/
cloneMCPServers/cloneStringSlices now return nil for nil input (the
append-based copies already preserved nilness).
2026-08-10 01:41:20 +10:00
sam 479ed95a91 fix(config): close review blocker on the PATCH path; deep-copy snapshots; atomics for shared state
Second adversarial pass findings, fixed:

BLOCKER -- PATCH silently dropped every dotted key. The UI sends
dotted keys ('budgets.max_steps') but ApplyOverlayValues fed them
straight to yaml.Unmarshal, which cannot decode a dotted key as a
struct field and silently ignored it: 200 OK, zero effect, and the row
still persisted (poisoning later reloads with an unvalidated value).
The reload path worked only because overlay.Snapshot nests first. Fix:
applyDottedOverlay nests dotted keys via overlay.Nest before decode --
the same nesting Snapshot performs -- and is pinned by
TestApplyDottedOverlayNestsBeforeDecode, which fails against the old
behaviour.

MAJOR -- shallow copies let overlay decode mutate the published
snapshot. Holder.Get and *doc are shallow; yaml decodes maps in place,
so a crafted nested update (or a failed boot overlay apply) mutated
shared maps of the published config before validation. Fix: config.Clone
deep-copies every reference field (Models, Providers, Repos, Identities,
Dispatch.Labels, Agent.Env, Chat/Telegram slices, Tools.MCPServers,
Memory.ProviderConfig, WebFetch.Enabled pointer, Extra), used by the
PATCH path (next := d.Cfg.Get().Clone()) and by Loader.ApplyOverlay,
which now never mutates its input (clones Config and the provenance
chain internally), so boot and reload pass the base document directly
and a failed apply leaves it exactly as it was. Pinned by
TestConfigCloneDeepCopiesReferenceFields.

MAJOR -- data races on captured main.go variables. bootOverlayErr and
currentProvenance were plain values written by the SIGHUP goroutine and
the PATCH/Reset handlers and read by every GET /api/config; -race passed
only because no test exercised the composition root concurrently. Both
are now atomic.Pointer; the LastReload closure and the provenance
dedup read them atomically.

MINOR -- MaxConcurrency removed from the reloadable allowlist. The
dispatchers re-read it but the startup-built container pool captures it
(container/pool.go:94,163), so 'every consumer re-reads it' is false and
a change only partially applies. It now warns requires-restart; the
criterion comment on reloadableFields documents the pool capture.

MINOR -- ResetConfig now resolves and validates the target state BEFORE
deleting the row, so a broken file cannot leave the store and the
running config disagreeing; the provenance chain no longer grows
unboundedly (the runtime-overlay origin replaces, not appends).

Also: holder.go no longer references a file watcher that does not exist.

Verified: task check clean; -race clean on cmd/archied, webui, daemon,
config; new findings from the second adversarial pass are closed.
2026-08-10 01:23:34 +10:00
sam 171e1b2686 feat(config): surface overlay state, shadowed keys, and per-row reset
Three gaps closed, all in the 'your change didn't take effect and
nothing told you' family:

1. MaxRetries joins the reloadable allowlist, and the criterion is
   fixed on reloadableFields: a field is reloadable when EVERY
   consumer re-reads it. MaxRetries is the worked example -- its only
   consumer is the webui maxRetriesFor handler, which re-reads per
   request (api_tasks.go:380,383); nothing in the daemon reads it, so
   warning 'requires restart' was spurious. Chat.Telegram.TokenEnv is
   documented as the counter-example (webui reads it per request but a
   startup-built telegram bot consumes it, so it is not reloadable).
   Pinned by TestChangedNonReloadableFields alongside the comment.

2. Boot-time overlay degradation is now surfaced. config.ReloadStatus
   gains OverlayUnavailable; main.go records the reason when the
   overlay store cannot be opened or its values fail validation, and
   /api/config carries it (merged into LastReload). Cleared by a
   successful dashboard write or reset. The settings page renders a
   warning banner, so a daemon running on file config alone says so
   where the operator is looking, not only in logs.

3. Overlay-shadowed file edits are visible and recoverable.
   ConfigView.Overridden lists the dotted keys the runtime overlay
   currently sets, so the UI marks those rows (their file value is
   shadowed until reset). POST /api/config/reset exposes the
   store's Delete as a per-row reset: it deletes the row and
   republishes file + remaining overlay, which is also the answer to
   'my file edit isn't taking effect'. The dashboard can now remove
   every override it can create.

Verified: task check clean; -race clean on cmd/archied, webui,
infrastructure/configuration; new handler tests cover the reset
200/400/503 paths and the overridden list.
2026-08-10 01:09:05 +10:00
sam da2d93a830 feat(config): runtime config overlay store and PATCH /api/config
The second half of #348's storage layer: dashboard-edited overrides
live in their own SQLite file, separate from the task and conversation
stores so each file owns its user_version and migrator with no
contention (configDBPath -> cfg.DBPath + "-config.sqlite", matching
the taskDBPath / conversationDBPath convention).

Store (internal/infrastructure/configuration/overlay):
- config_overlay(key TEXT PRIMARY KEY, value TEXT NOT NULL,
  updated_at, updated_by), own migration, WAL.
- Set refuses DeniedKeys (db_path, work_dir) at write time as well as
  at the API layer; the map is exported so the dashboard can show the
  reason instead of silently omitting the edit affordance.
- Snapshot JSON-decodes each value and nests dotted keys
  ("budgets.max_steps" -> map[b][m]) ready to decode into Config.

Loader (ApplyOverlay / ApplyOverlayValues):
- Decodes a nested overlay map with the same field-level precedence as
  a file overlay, records the runtime-overlay origin, then applies
  defaults and validates. Used at boot, on SIGHUP reload (ReloadController
  gained an optional WithOverlay source) and by the PATCH path.

Boot and reload:
- A broken overlay (unopenable store, unreadable snapshot, values that
  fail validation) degrades to file config alone -- it must never brick
  the daemon. --no-config-overlay / ARCHIE_SKIP_CONFIG_OVERLAY=1 skips
  the store entirely; the flag is now a real gate, not a log-only stub.

PATCH /api/config (webui):
- handleConfigUpdate decodes {"updates": {dotted.key: value}}, calls
  the composition-wired UpdateConfig seam, and classifies errors via
  webui sentinels: ErrConfigUpdateInvalid -> 400 (denylisted key,
  failed validation, unparseable value), ErrConfigUpdateUnavailable ->
  503 (overlay skipped by recovery flag), anything else -> 500.
- The seam (main.go) applies updates to a COPY of the published
  config, validates the materialised result, persists to the overlay,
  then publishes through the same path as reload -- it never calls
  Set directly from the handler, since web.Cfg is the daemon's own
  Holder.
- ConfigView gains Locked (dotted key -> reason) so the UI disables
  denylisted rows explicitly.

Verified: task check clean; new tests cover the store round-trip,
denied keys, loader overlay layering/validation/provenance, reload
with and without overlay, and the PATCH handler's 200/400/503 paths.
2026-08-10 00:54:23 +10:00
sam 31fdbddd61 fix(config): complete the reloadable allowlist and pin it against ForTask
The allowlist drifted before the chunk shipped. ForTask (config.go:457-465)
is the authoritative per-dispatch snapshot and carries Models, Budgets,
Dispatch, DiffCapLines, Notify and Forge.Host -- I allowlisted only three.
And the poll path reads cfg.Label and cfg.BotUser live per cycle
(daemon.go:359,390 and :368,381) off the same per-poll config as
Dispatch.Trigger. Five fields would have reported 'requires a restart'
while actually taking effect immediately: the safe direction, but exactly
the warning-fatigue failure that kills the feature (an operator who is
told to restart for a label change that already applied learns the
warning is noise).

The fix is enforcement, not a bigger list: TestReloadableFieldsCoverForTaskSnapshot
reflects over TaskConfig and fails if any field ForTask carries is not in
the allowlist (Forge maps to the Forge.Host sub-field entry). A field
added to ForTask later now fails the test instead of silently producing a
wrong warning. Label and BotUser are not carried by ForTask -- they come
off the poll path -- so they are allowlisted by hand with a comment, and
the same test pins them explicitly.

Also documents on webui.Server.Cfg that in production it is the daemon's
OWN Holder (web.Cfg = d.Cfg), so Set publishes to the running daemon: a
config-mutating handler must apply to a copy, validate the materialised
config, persist, then Set -- never Set directly.

task check clean.
2026-08-10 00:47:30 +10:00
sam f544dd0978 feat(config): SIGHUP-triggered reload with requires-restart warnings
The daemon can now re-read its configuration on SIGHUP (or on a
future in-process trigger) and republish it through the Holder,
closing the first half of issue #348: 'config can change at runtime,
safely, and can't brick the daemon.'

Reload path:
- cmd/archied/reload.go: ReloadController re-runs loader.Resolve on
  the same paths main.go passed at startup (Resolve is pure -- no
  reload state added to the loader). apply is called only with a
  Document that passed validation; a failed reload records the error
  and its timestamp and leaves the running config untouched. That
  Set-not-called-on-error property is the safety core and is pinned
  by TestReloadValidationFailureDoesNotApply.
- SIGHUP listener in main.go runs reloadLoop(ctx, ch, controller, log);
  the loop is extracted as a testable seam (syscall.SIGHUP is Unix-only,
  fine given the Linux target).

PollInterval is genuinely reloadable now, not just re-read:
the three ticker sites (Run, per-identity goroutine, shared
maintenance loop) re-read the interval each tick and Reset the ticker
on change. A reloaded poll_interval takes effect on the next cycle.

Diff-and-warn so an operator never has to read the source to know
whether an edit took effect: changedNonReloadableFields compares the
old and new snapshots against an allowlist of re-read fields
(PollInterval, Repos, DiffCapLines, Budgets, Dispatch, and the
Containers sub-fields VolumeTTL + MaxConcurrency). Anything else that
changed logs Warn 'requires a restart'; a fully-applied reload logs
Info. The allowlist is deliberate: a field added to Config later
defaults to requires-restart, the safe direction.

NATS split-brain closed: containerEnv now hands containers the
endpoint the daemon's own client connected with at startup
(Daemon.ConnectedNATS, captured at construction) instead of the live
config, so a reloaded nats.url cannot point new containers at a
server the daemon is not publishing on. A nats change is therefore
classified requires-restart, and CLAUDE.md's 'hands its own nats.url
verbatim' note is updated to the meaning it now has (same today,
deliberately different after a reload).

Validation guard: poll_interval <= 0 is rejected in validate.go
beside the VolumeTTL guard. applyDefaults fills zero with 60s, so
this fires on an explicit negative -- which would panic
time.NewTicker/Reset at boot and, after the per-iteration reset,
on a SIGHUP mid-run.

Observability: config.ReloadStatus (last_error, last_error_at,
last_reload_at) is recorded by the controller and surfaced in
/api/config via Server.LastReload, so a stale running config is
visible in the dashboard, not just the logs. Rendering waits for the
UI step.

Recovery hatch: --no-config-overlay flag plus ARCHIE_SKIP_CONFIG_OVERLAY=1
(configuration.SkipOverlay), logged at startup. No overlay source
exists yet; the flag is the documented escape hatch the DB overlay
(step 3 of #348) will consult.

webui.Server.ConfigProvenance becomes atomic.Pointer so a reload can
swap it via SetProvenance independently of Cfg; the accepted skew
(new config alongside old provenance) is documented on the field.

Verified: task check clean; -race on daemon, webui, cmd/archied,
config, infrastructure/configuration.
2026-08-10 00:19:46 +10:00
sam b6f0aea930 refactor(daemon,webui): make Daemon.Cfg and webui.Server.Cfg reload-safe
Daemon.Cfg, webui.Server.Cfg, and telegramSetup.Cfg were all held by
value (or as a raw shared pointer in webui's case), so a future reload
path could not swap the running configuration -- the consumers held
frozen snapshots made at construction. This is the prerequisite for
issue #348.

Add config.Holder, a small RWMutex-backed snapshot holder:
  - Get returns a value copy under RLock; the copy shares its
    reference-type fields with the published snapshot, so a doc
    comment pins the immutability invariant (a published Config is
    immutable; reload constructs a fresh one).
  - Set replaces the snapshot under WLock.
  - Get is nil-safe; Set is not.

Convert every consumer to read through the accessor. The convention
'cfg := h.Get() once per logical operation' is documented on the
type so the next reader knows that inline d.Cfg.Get().Foo is a code
smell (a Set could land between two consecutive Gets).

Special cases:
  - webui.Server.Cfg: was *config.Config, a raw shared pointer read
    from handler goroutines. Convert to *config.Holder. The
    pre-existing 'cfg := s.Cfg' aliasing bug at api_config.go:295
    (which was *trying* to copy but only copied the pointer) now
    becomes 'cfg := s.Cfg.Get()', which is the value copy the
    read-only view always wanted.
  - daemon_test.go: one test mutated d.Cfg.DiffCapLines in place,
    which the Holder forbids. Replace with the documented
    read-modify-write pattern (prev := d.Cfg.Get(); prev.X = Y;
    d.Cfg.Set(prev)).
  - telegramSetup.Cfg: matches the daemon's API for consistency
    even though setup runs before the daemon is alive.

TaskContext.Cfg in internal/domain/workflow/workflow.go is
deliberately NOT converted -- a task that starts under one config
should finish under it, and a config changing between TDD and gate
stages is a bug, not a feature. Snapshotted at dispatch, as before.

Verified: go test ./... (61 packages), go vet, -race on
internal/daemon and internal/config.
2026-08-09 23:29:52 +10:00
sam 61b4b7b0cf refactor(store): replace legacy persistence with SQLite
Task and event persistence now uses a dedicated SQLite sidecar derived from the configured state prefix, matching the independently persisted conversation store. The configured path now means a state-path prefix rather than a database file so fresh SQLite state never attempts to interpret the previous storage format.

Remove the retired implementation and dependency completely, strengthen SQLite migrations and concurrency behavior, restore backend-neutral coverage, and align configuration, operational guidance, UI labels, and deployment documentation with the new storage and host-daemon topology.
2026-08-09 22:27:25 +10:00
sam e4385ef266 feat(webui,gateway): add /api/tasks/{id}/logs endpoint and task_logs chat tool
Adds two read surfaces for per-task persisted log history so an operator
can answer "why did task N park?" without needing to watch docker ps in
real time — the deployment model documented in CLAUDE.md.

/api/tasks/{id}/logs — built on logging.Tail the same way /api/logs
already is, defaults attempt to the tasks current Attempt from the store.

task_logs chat tool — lets Archie read task logs from Telegram or webui
chat. Identity enforcement matches chatTaskListerAdapter: a model bound
to one identity cannot read another identitys logs, and forge tasks
(empty identity) are denied.

internal/logging: add TaskRegistry.Path() for read-side access without
threading baseDir separately through every caller.

Fixes from adversarial review: strict identity check (no empty-identity
bypass), non-null Entries when disabled, component doc accuracy, and
adapter tests covering the security boundary.

Ref: #455
2026-08-09 19:14:35 +10:00
sam 951ddbc9f4 feat(daemon): consume agent system logs into per-task sinks and the dashboard feed
Closes the loop #451-#453 opened. archie-agent (#453) now publishes its
own logs to agentexec.SubjectForSystem(taskID) over NATS, and a task
log sink (#452) exists to persist them, but nothing on the daemon side
subscribed, demuxed, or wrote anything -- the messages had nowhere to
land.

- agentexec: SubjectSystemWildcard + TaskIDFromSystemSubject, the
  inverse of SubjectForSystem, for a daemon subscribing to every
  task's system subject with one wildcard subscription.
- logging: TaskRegistry manages one open TaskSink per running task
  attempt, mirroring every write onto the live dashboard Feed the same
  way a locally-written log would appear. Nil-receiver-safe, matching
  every other optional composition-wired Daemon field.
- daemon: Daemon.process opens a task's log sink for its own duration
  (covering both the sandboxed-container and in-process dispatch
  paths uniformly) and closes it on every return path.
- cmd/archied: constructs the registry, subscribes core NATS to
  SubjectSystemWildcard, and wires TaskLogs into both Daemon and
  webui.Server.
- webui: archiving a task now removes its log directory too, so
  archived tasks' logs don't accumulate forever. Documented as a
  known, accepted race: a task can reach terminal status (satisfying
  ArchiveTask's precondition) before process() itself returns and
  closes the sink, so a very late system-log write can race an
  archive's Remove. Not corrupting -- the write lands on a
  soon-to-be-unlinked inode and is discarded -- and not worth fixing
  here: it would mean tying Close to the store transition instead of
  to process()'s return, a larger change than this issue's scope.

Adversarial review (fresh subagent) found and fixed two real issues in
the first draft: a doc comment accidentally spliced between two
functions during an extraction meant to fix a lint regression, and a
genuine test gap on openTaskLog (added, using TaskRegistry.Write's own
true/false contract as the observable -- no Store/Forge/Trees fakes
needed, since openTaskLog only touches TaskLogs and Log).
2026-08-09 18:42:54 +10:00
sam fab8a66486 Merge pull request #457 from samcharles93/fix/452-task-log-sink
feat(logging): add a per-task log sink
2026-08-09 18:17:24 +10:00
sam 53da52ecd1 feat(agentexec): publish agent-side logs to the task's system subject
Part of #453 (task-observability epic #450). archie-agent logged to
os.Stderr only, and its container is created with AutoRemove: true
(internal/container/pool.go), so all of it disappeared the moment a
task's container exited -- there was no way to see why, say, a gate
command failed inside one.

Adds agentexec.SystemLogHandler: a slog.Handler that tees to whatever
it wraps (kept for an operator attached to a live container) and
additionally publishes each record, JSON-encoded as a logging.Entry,
to SubjectForSystem(taskID) -- a subject that already existed,
documented for exactly this ("log dumps, health... observability"),
with zero publishers anywhere in the codebase.

cmd/archie-agent/main.go wires it in only when this container was
spawned for one specific task (bootTaskID / .git/task.json present);
shared queue-group mode serves many unrelated tasks per process, so
there's no single task subject to ship to and keeps the stderr-only
logger.

Building this surfaced a real pre-existing bug: FeedHandler's attr
flattening applied the group chain active at Handle-time to *all*
accumulated attrs, including ones added via WithAttrs before a later
WithGroup call -- nesting them under a group they were never
actually inside, contrary to slog's own documented Handler contract.
It went unnoticed because nothing in the codebase called WithGroup
until this change needed to. Extracting the shared flattening logic
into exported FlattenAttrs/PrefixAttrs (so SystemLogHandler doesn't
duplicate it) is what surfaced it; fixed in the same change since I
was the one making it public API.

Docs on the wire format's own trap for the daemon-side consumer
(#454, not yet written): the published bytes are a marshaled
logging.Entry, meant for json.Unmarshal, not logging.decode() --
decode() parses raw slog JSON *log lines* off disk, a different and
incompatible shape. See the doc comment on SystemLogHandler.
2026-08-09 16:49:39 +10:00
sam a7f33fcb03 feat(logging): add a per-task log sink
Part of #452 (task-observability epic #450). A task's real work
happens in an ephemeral archie-agent container; currently nothing
persists what it logs once the container exits (AutoRemove: true).

Adds TaskSink: a per-task, per-attempt JSONL log destination sharing
the exact on-disk format the daemon-wide logger already writes (see
Entry/Tail/Query/Components in reader.go), so a task's history is
readable with the same reader rather than a second format to keep in
sync. Backed by the existing rotatingFile, so it gets the same
size-based rotation and retention as the daemon-wide log for free.

Deliberately does not fall back to stderr on open failure the way the
daemon-wide New does: that fallback exists because losing the daemon
over a broken log path is worse than losing the durable copy, and
there is no equivalent daemon-level fallback for one task's log.

This lands only the sink primitive and its own tests. Wiring an
agent-side publisher and a daemon-side NATS consumer into it is
tracked separately in #453 and #454.
2026-08-09 16:32:52 +10:00
sam 86d3071f39 fix(workflow): carry gate output into the baseline park error
StageBaselineGate captured a failing gate command's full output but
discarded it: a 200-char clip went into a Warn log, and the terminal
error carried only res.Status. A park reason like "go build ./...
fails ... (status: failed)" gave no way to see the actual compiler
error, and the container that had it is already gone (AutoRemove).

Now the park error carries a bounded tail of the real output (tail,
not head, since a failing test's explanation is at the end after
pages of "ok" lines), rune-safe so it can't emit invalid UTF-8 into
a persisted park reason. The three existing byte bounds in this
stage (log line, agent mission, park error) are named and documented
as independently sized rather than left as bare numbers.

Also adds docs/architecture/observability.md, the design doc for the
larger task-observability epic (#450) this fix is the first step of.
2026-08-09 16:23:24 +10:00
sam 4131f05036 feat(webui,gateway): wire config, channel, log-feed, and work-intake seams
Composition root work tying the config resolver, channel manager, log
feed, and executable workflow registry into the running daemon and
dashboard server: web.Channels/LogFeed/Workflows/ConfigProvenance,
channel-reload wiring, TaskStopper for the Stop action, and
SpawnRequest.Body/WorkRequests threading dashboard "Start work" requests
through the same admission path chat uses. Touches all three features at
once because this is where they're composed, not owned by any single one.

Refs #53, #54, #55
2026-08-09 01:26:58 +10:00
sam 92280f4039 fix(store,gateway): taskstate constants and mobile jump-to search
Follow-up pieces of the task-controls work: store.InsertEvent factors out
an execer-generic insertEvent so SQLite gets the same zero-time defaulting
ArchiveTask's audit path relies on; gateway tests assert against
taskstate constants instead of magic status strings; the navigation
command bar gets an accessible mobile search toggle.

Refs #53
2026-08-09 01:26:40 +10:00
sam a27d430215 feat(logging,workflow): daemon log feed and executable workflow registry
Add internal/logging.Feed, a logging-owned bounded replay buffer separate
from the persisted task-lifecycle event stream, with a live SSE endpoint.
Expose the executable workflow registry (skillbuild.Catalog) including
zero-run definitions and their origin (builtin vs. installed skill), not
just historical run statistics. Route() now returns an explicit failure
workflow for an unknown pre-assigned workflow name instead of silently
falling through to label-based routing.

Fixes #54
2026-08-09 01:26:10 +10:00
sam 472d8d717c feat(config,channels): format-neutral config resolution and truthful channel state
Loader.Resolve() selects TOML/YAML files or a config directory as the
production entry point, replacing the TOML-only Overlay() as the default
path. Add internal/channels.Manager, reporting each adapter's actual
runtime lifecycle (configured/starting/running/degraded/failed) instead of
inferring health from configuration presence alone. Wire channel reload
through a per-adapter seam.

Fixes #55
2026-08-09 01:25:58 +10:00
sam 0d500a4ca9 feat(webui): lifecycle-safe task controls with atomic retry and archive
Add cancel/stop/approve/reject/retry/abandon/archive to the dashboard task
action matrix, each guarded by an optimistic status check against the
store. Retry accounting (RetryCount, max_retries) is now atomic with the
status transition instead of a separate write. Archive removes a terminal
task's local record with a fail-safe audit event first. Add always-labelled
responsive navigation, mobile Jump-to search, and versioned Setup-complete
dismissal.

Fixes #53
2026-08-09 01:25:51 +10:00
sam b876fc80cc feat(gateway): make chat turns durable and idempotent 2026-08-09 01:15:41 +10:00
sam 80f9d92e51 fix(webui): close open-redirect in dashboard token-exchange handler
clean.RequestURI() echoed the incoming request's raw path/query straight
into http.Redirect's Location header. A path starting with "//" is treated
by browsers as protocol-relative, so a crafted token-exchange link could
redirect off-host after a legitimate token exchange. sameOriginPath()
collapses any leading "//" before the value is ever used.

Fixes #440
2026-08-09 01:13:20 +10:00
sam 6aa3772ce9 feat(gateway): tool-approval gate, session tools for chat, and /delete command
Adds a human-consent gate for tools marked RequiresApproval, exposes
session management as LLM-invokable chat tools, and gives the operator
/delete to retire conversations permanently.

**Tool-approval gate (internal/tools, internal/agentexec)**

- ApprovalRequester contract lives in internal/tools (not gateway) so
  agentexec can import it without pulling in the chat-adapter stack.
- ToolSetOptions.Approval wires the approver through to toolExecute.
- BuildToolSetFrom omits RequiresApproval tools when no approver is
  configured; dispatch paths (invokeTool, tool_call bridge) refuse
  them too, so the invariant holds at every indirection.
- The gate fails closed: only Approved/PermanentlyApproved pass;
  an unknown decision value from a faulty approver does not execute.
- Tool-level timeout (2 min) returns a plain error, not a context
  cancellation, so the model can report "approval timed out" instead
  of the turn aborting.
- BuildApprovalDescription callback lets a tool surface the specific
  resource being acted on ("Delete session \"abc123\"") rather than
  showing a static tool description.

**Session tools (internal/gateware/session_tools.go)**

- session_list (idempotent), session_resume (mutating), session_title
  (mutating), session_delete (mutating + RequiresApproval).
- Scoped to the current platform + channel: GetByChannel replaces
  List() so a model in chat A cannot list or manipulate chat B.
- session_title validates the session belongs to this channel before
  writing; session_resume/session_delete resolve refs against the
  channel-scoped list.
- Shared limit helper extracted to tools.ListLimit.

**Telegram approval adapter (internal/channels/telegram/approval.go)**

- telegramApprover implements ApprovalRequester via the three-button
  inline keyboard (Approve/Permanent/Deny). 2-min token expiry.
- Own map + mutex + "approval:" callback prefix — separate from the
  dangerous-command system.
- Permanent approval uses exact-match (not prefix) lookup keyed by
  action + resource description, so permanently approving "delete
  session abc123" only covers that one session, not every call to
  session_delete for 24h.

**Session lifecycle**

- /delete <session-id>: removes a session and its history. Refuses
  prefixes shorter than 4 chars even when unique. Exact ID wins over
  an ambiguous prefix. sessionTracker.forget sweeps every channel+
  thread pointing at the deleted session.
- Extracted resolveSessionRef and shortSessionID for /resume and
  /delete to share.

**Wiring**

- submitTurn builds the approver onto ctx before the LLM turn.
- chatGenerateOptions reads approver from ctx, passes it through
  ToolSetOptions.
- SessionTools built per turn on the current message, combined with
  taskTools into a single toolset.
2026-08-08 16:42:59 +10:00
sam 1081a6c3ed feat(daemon): identity-scoped isolation + gateway session titles and /delete
Multi-identity daemon isolation:
- Split runIdentities into per-identity poll loops plus one shared
  maintenance-and-drain loop, so reconcilePRs and storage cleanup run in
  multi-identity mode and the global max_concurrency / per-repo
  serialization holds across identities.
- Make the poll path identity-aware: each identity polls with its own
  forge client, bot user, label, trigger, and ack reaction.
- Make allowConcurrentFor task-based and identity-aware via repoFor.
- Roll back storage mounts when container acquisition fails; log
  Store.Update failures after branch set.

Container-mode RPC identity routing:
- forgerpc/worktreerpc gain identity-scoped subjects (SubjectFor,
  RegisterFor) and identity-carrying clients.
- registerTaskRPCServers registers one server pair per identity (plus
  root), so a container-mode task owned by a non-root identity is served
  by its own forge client and worktree manager.

Gateway/Telegram:
- LLM-proposed session titles generated in the background after turns
  (TitleGenerator + chatTitleGenerator), persisted by the gateway.
- /delete session command with prefix guard and tracker forget.
- Shared session display helpers (shortSessionID, sessionDisplayTitle,
  resolveSessionRef).
2026-08-08 01:32:35 +10:00