dbb6550ab3088baaaa70203fbed2c50d77641908
15
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
aba035a396 |
feat(webui): payload field mapping -- bind JSON paths to named fields, preview before saving
Adds t2db.3: schema-by-example field mapping for the no-code playbook epic. An operator picks a captured event, clicks through its payload to bind named fields to JSON paths, and previews the resolution against a real event before saving -- a missing, null, or type-changed path fails loudly (named by field and event) rather than handing an agent a blank value. - internal/domain/mapping: Mapping/Field types, path parsing/walking, Resolve (per-field failure reporting), Validate (incl. duplicate names) - internal/store/mappings.go: SQLite persistence, mirrors captures.go - internal/webui/api_mapping.go: CRUD + preview HTTP API - ui/src/mappings/: dashboard editor (clickable payload tree, live preview) Design decisions recorded in docs/prds/payload-field-mapping.md. t2db.4 (binding a mapping + matcher to a workflow) is out of scope here. |
||
|
|
a8f9f1b339 |
feat(webui): event inspector -- see captured payloads in the dashboard
Adds GET /api/captures (token-gated, matching the memory/skills enabled-false-not-an-error convention when Server.Captures is nil) and a new dashboard page at /captures listing recent inbound captures, newest first, with per-row expand to explore the redacted payload and headers pretty-printed. Live updates reuse the existing operator-activity event pipeline (Server.emit -> Events.Publish -> the /events SSE stream every other page already subscribes to) rather than polling or inventing new push plumbing: handleCapture now emits a Kind "capture" event with the same field shape the list endpoint returns, so a captured event appears in the inspector without a manual refresh. Frontend follows the tasks/task-row.js precedent: row rendering is split into ui/src/captures/capture-row.js so it can be unit tested without the page's fetch/SSE wiring. "Unbound" is shown for every row today since no binding data model exists yet (t2db.4) -- the column is wired now so that bead only needs to change what it renders, not add it. |
||
|
|
37bf8ef05a |
feat(capture): store unbound webhook events, retention-bounded
Adds POST /webhooks/capture/{source}: an unauthenticated endpoint (mounted
alongside /healthz, bypassing dashboard token auth by design) that persists
any inbound webhook archie has no binding for yet, so it can be inspected
and mapped later (t2db.2/t2db.3).
docs/prds/event-capture-storage.md decides SQLite over NATS JetStream for
storage: NATS is not unconditionally available until 7d5u.2 ships embedded
support, and capture's single-writer/single-reader access pattern doesn't
need JetStream's fan-out/replay machinery anyway.
internal/store/captures.go adds a captured_events table with prune-on-write
(retention age + row-count cap, both configurable). The HTTP handler applies
webhookguard redaction before persisting, and enforces two more independent
disk-bound mechanics before that: a per-request body-size cap and a
per-remote-address rate limit. New [capture] config section with documented
defaults; all fields optional, capture is on by default.
Adversarial review (manual, after the usual fresh-subagent pass hit a
session API limit mid-run) caught and fixed three real defects before this
landed, each with a regression test confirmed to fail against the pre-fix
code:
- the body-size cap silently no-op'd when CaptureMaxBodyBytes was left
unset, the one bound that limits a single payload's size rather than row
count or write rate
- the rate limiter was keyed by the request's own source path segment --
unregistered and attacker-chosen by design for this endpoint, violating
webhookguard.RateLimiter's documented bounded-key-space assumption and
letting a sender bypass the limit by rotating the segment; rekeyed to
remote address
- the retention prune compared time.RFC3339Nano strings, whose trimmed
variable-width fractional seconds break lexicographic ordering within the
same second; switched to fixed-width RFC3339
Closes archie-core-t2db.1.
|
||
|
|
ddd532b824 |
feat(update): stream live progress and roll back failed updates automatically
The /update command went silent between "starting" and one final message, gave no version summary, and required manual intervention on failure. CommandInstaller now streams the installer script's stdout live instead of buffering it, and parses a structured result (previous/installed versions, per-component status) from a sentinel line. Because the reference install script hands the daemon restart off to a detached systemd unit specifically so restarting archied.service doesn't kill the process that requested the update, the outcome is reported in two phases: the synchronous build/install result comes back through Install() itself, and the post-restart health check/rollback outcome is written by a new watchdog script and relayed by whichever archied process boots next, via a small JSON handoff file cleared once relayed. Also fixes three defects an adversarial review pass found in the first draft: the caller's request/handler context was passed straight into the install child process, so a concurrent /restart or a browser disconnect could SIGKILL it mid binary-copy; Telegram and the web dashboard each tracked "install in progress" with their own private flag despite sharing one Service, allowing two installs to race the same .prev backups; and the watchdog reported rolled_back:true even when no backup existed to restore, which would have told an operator to stand down while the daemon was still broken. internal/webui gains an unauthenticated /healthz the watchdog polls after restarting archied. |
||
|
|
171e1b2686 |
feat(config): surface overlay state, shadowed keys, and per-row reset
Three gaps closed, all in the 'your change didn't take effect and nothing told you' family: 1. MaxRetries joins the reloadable allowlist, and the criterion is fixed on reloadableFields: a field is reloadable when EVERY consumer re-reads it. MaxRetries is the worked example -- its only consumer is the webui maxRetriesFor handler, which re-reads per request (api_tasks.go:380,383); nothing in the daemon reads it, so warning 'requires restart' was spurious. Chat.Telegram.TokenEnv is documented as the counter-example (webui reads it per request but a startup-built telegram bot consumes it, so it is not reloadable). Pinned by TestChangedNonReloadableFields alongside the comment. 2. Boot-time overlay degradation is now surfaced. config.ReloadStatus gains OverlayUnavailable; main.go records the reason when the overlay store cannot be opened or its values fail validation, and /api/config carries it (merged into LastReload). Cleared by a successful dashboard write or reset. The settings page renders a warning banner, so a daemon running on file config alone says so where the operator is looking, not only in logs. 3. Overlay-shadowed file edits are visible and recoverable. ConfigView.Overridden lists the dotted keys the runtime overlay currently sets, so the UI marks those rows (their file value is shadowed until reset). POST /api/config/reset exposes the store's Delete as a per-row reset: it deletes the row and republishes file + remaining overlay, which is also the answer to 'my file edit isn't taking effect'. The dashboard can now remove every override it can create. Verified: task check clean; -race clean on cmd/archied, webui, infrastructure/configuration; new handler tests cover the reset 200/400/503 paths and the overridden list. |
||
|
|
da2d93a830 |
feat(config): runtime config overlay store and PATCH /api/config
The second half of #348's storage layer: dashboard-edited overrides live in their own SQLite file, separate from the task and conversation stores so each file owns its user_version and migrator with no contention (configDBPath -> cfg.DBPath + "-config.sqlite", matching the taskDBPath / conversationDBPath convention). Store (internal/infrastructure/configuration/overlay): - config_overlay(key TEXT PRIMARY KEY, value TEXT NOT NULL, updated_at, updated_by), own migration, WAL. - Set refuses DeniedKeys (db_path, work_dir) at write time as well as at the API layer; the map is exported so the dashboard can show the reason instead of silently omitting the edit affordance. - Snapshot JSON-decodes each value and nests dotted keys ("budgets.max_steps" -> map[b][m]) ready to decode into Config. Loader (ApplyOverlay / ApplyOverlayValues): - Decodes a nested overlay map with the same field-level precedence as a file overlay, records the runtime-overlay origin, then applies defaults and validates. Used at boot, on SIGHUP reload (ReloadController gained an optional WithOverlay source) and by the PATCH path. Boot and reload: - A broken overlay (unopenable store, unreadable snapshot, values that fail validation) degrades to file config alone -- it must never brick the daemon. --no-config-overlay / ARCHIE_SKIP_CONFIG_OVERLAY=1 skips the store entirely; the flag is now a real gate, not a log-only stub. PATCH /api/config (webui): - handleConfigUpdate decodes {"updates": {dotted.key: value}}, calls the composition-wired UpdateConfig seam, and classifies errors via webui sentinels: ErrConfigUpdateInvalid -> 400 (denylisted key, failed validation, unparseable value), ErrConfigUpdateUnavailable -> 503 (overlay skipped by recovery flag), anything else -> 500. - The seam (main.go) applies updates to a COPY of the published config, validates the materialised result, persists to the overlay, then publishes through the same path as reload -- it never calls Set directly from the handler, since web.Cfg is the daemon's own Holder. - ConfigView gains Locked (dotted key -> reason) so the UI disables denylisted rows explicitly. Verified: task check clean; new tests cover the store round-trip, denied keys, loader overlay layering/validation/provenance, reload with and without overlay, and the PATCH handler's 200/400/503 paths. |
||
|
|
31fdbddd61 |
fix(config): complete the reloadable allowlist and pin it against ForTask
The allowlist drifted before the chunk shipped. ForTask (config.go:457-465) is the authoritative per-dispatch snapshot and carries Models, Budgets, Dispatch, DiffCapLines, Notify and Forge.Host -- I allowlisted only three. And the poll path reads cfg.Label and cfg.BotUser live per cycle (daemon.go:359,390 and :368,381) off the same per-poll config as Dispatch.Trigger. Five fields would have reported 'requires a restart' while actually taking effect immediately: the safe direction, but exactly the warning-fatigue failure that kills the feature (an operator who is told to restart for a label change that already applied learns the warning is noise). The fix is enforcement, not a bigger list: TestReloadableFieldsCoverForTaskSnapshot reflects over TaskConfig and fails if any field ForTask carries is not in the allowlist (Forge maps to the Forge.Host sub-field entry). A field added to ForTask later now fails the test instead of silently producing a wrong warning. Label and BotUser are not carried by ForTask -- they come off the poll path -- so they are allowlisted by hand with a comment, and the same test pins them explicitly. Also documents on webui.Server.Cfg that in production it is the daemon's OWN Holder (web.Cfg = d.Cfg), so Set publishes to the running daemon: a config-mutating handler must apply to a copy, validate the materialised config, persist, then Set -- never Set directly. task check clean. |
||
|
|
f544dd0978 |
feat(config): SIGHUP-triggered reload with requires-restart warnings
The daemon can now re-read its configuration on SIGHUP (or on a future in-process trigger) and republish it through the Holder, closing the first half of issue #348: 'config can change at runtime, safely, and can't brick the daemon.' Reload path: - cmd/archied/reload.go: ReloadController re-runs loader.Resolve on the same paths main.go passed at startup (Resolve is pure -- no reload state added to the loader). apply is called only with a Document that passed validation; a failed reload records the error and its timestamp and leaves the running config untouched. That Set-not-called-on-error property is the safety core and is pinned by TestReloadValidationFailureDoesNotApply. - SIGHUP listener in main.go runs reloadLoop(ctx, ch, controller, log); the loop is extracted as a testable seam (syscall.SIGHUP is Unix-only, fine given the Linux target). PollInterval is genuinely reloadable now, not just re-read: the three ticker sites (Run, per-identity goroutine, shared maintenance loop) re-read the interval each tick and Reset the ticker on change. A reloaded poll_interval takes effect on the next cycle. Diff-and-warn so an operator never has to read the source to know whether an edit took effect: changedNonReloadableFields compares the old and new snapshots against an allowlist of re-read fields (PollInterval, Repos, DiffCapLines, Budgets, Dispatch, and the Containers sub-fields VolumeTTL + MaxConcurrency). Anything else that changed logs Warn 'requires a restart'; a fully-applied reload logs Info. The allowlist is deliberate: a field added to Config later defaults to requires-restart, the safe direction. NATS split-brain closed: containerEnv now hands containers the endpoint the daemon's own client connected with at startup (Daemon.ConnectedNATS, captured at construction) instead of the live config, so a reloaded nats.url cannot point new containers at a server the daemon is not publishing on. A nats change is therefore classified requires-restart, and CLAUDE.md's 'hands its own nats.url verbatim' note is updated to the meaning it now has (same today, deliberately different after a reload). Validation guard: poll_interval <= 0 is rejected in validate.go beside the VolumeTTL guard. applyDefaults fills zero with 60s, so this fires on an explicit negative -- which would panic time.NewTicker/Reset at boot and, after the per-iteration reset, on a SIGHUP mid-run. Observability: config.ReloadStatus (last_error, last_error_at, last_reload_at) is recorded by the controller and surfaced in /api/config via Server.LastReload, so a stale running config is visible in the dashboard, not just the logs. Rendering waits for the UI step. Recovery hatch: --no-config-overlay flag plus ARCHIE_SKIP_CONFIG_OVERLAY=1 (configuration.SkipOverlay), logged at startup. No overlay source exists yet; the flag is the documented escape hatch the DB overlay (step 3 of #348) will consult. webui.Server.ConfigProvenance becomes atomic.Pointer so a reload can swap it via SetProvenance independently of Cfg; the accepted skew (new config alongside old provenance) is documented on the field. Verified: task check clean; -race on daemon, webui, cmd/archied, config, infrastructure/configuration. |
||
|
|
b6f0aea930 |
refactor(daemon,webui): make Daemon.Cfg and webui.Server.Cfg reload-safe
Daemon.Cfg, webui.Server.Cfg, and telegramSetup.Cfg were all held by value (or as a raw shared pointer in webui's case), so a future reload path could not swap the running configuration -- the consumers held frozen snapshots made at construction. This is the prerequisite for issue #348. Add config.Holder, a small RWMutex-backed snapshot holder: - Get returns a value copy under RLock; the copy shares its reference-type fields with the published snapshot, so a doc comment pins the immutability invariant (a published Config is immutable; reload constructs a fresh one). - Set replaces the snapshot under WLock. - Get is nil-safe; Set is not. Convert every consumer to read through the accessor. The convention 'cfg := h.Get() once per logical operation' is documented on the type so the next reader knows that inline d.Cfg.Get().Foo is a code smell (a Set could land between two consecutive Gets). Special cases: - webui.Server.Cfg: was *config.Config, a raw shared pointer read from handler goroutines. Convert to *config.Holder. The pre-existing 'cfg := s.Cfg' aliasing bug at api_config.go:295 (which was *trying* to copy but only copied the pointer) now becomes 'cfg := s.Cfg.Get()', which is the value copy the read-only view always wanted. - daemon_test.go: one test mutated d.Cfg.DiffCapLines in place, which the Holder forbids. Replace with the documented read-modify-write pattern (prev := d.Cfg.Get(); prev.X = Y; d.Cfg.Set(prev)). - telegramSetup.Cfg: matches the daemon's API for consistency even though setup runs before the daemon is alive. TaskContext.Cfg in internal/domain/workflow/workflow.go is deliberately NOT converted -- a task that starts under one config should finish under it, and a config changing between TDD and gate stages is a bug, not a feature. Snapshotted at dispatch, as before. Verified: go test ./... (61 packages), go vet, -race on internal/daemon and internal/config. |
||
|
|
e4385ef266 |
feat(webui,gateway): add /api/tasks/{id}/logs endpoint and task_logs chat tool
Adds two read surfaces for per-task persisted log history so an operator
can answer "why did task N park?" without needing to watch docker ps in
real time — the deployment model documented in CLAUDE.md.
/api/tasks/{id}/logs — built on logging.Tail the same way /api/logs
already is, defaults attempt to the tasks current Attempt from the store.
task_logs chat tool — lets Archie read task logs from Telegram or webui
chat. Identity enforcement matches chatTaskListerAdapter: a model bound
to one identity cannot read another identitys logs, and forge tasks
(empty identity) are denied.
internal/logging: add TaskRegistry.Path() for read-side access without
threading baseDir separately through every caller.
Fixes from adversarial review: strict identity check (no empty-identity
bypass), non-null Entries when disabled, component doc accuracy, and
adapter tests covering the security boundary.
Ref: #455
|
||
|
|
951ddbc9f4 |
feat(daemon): consume agent system logs into per-task sinks and the dashboard feed
Closes the loop #451-#453 opened. archie-agent (#453) now publishes its own logs to agentexec.SubjectForSystem(taskID) over NATS, and a task log sink (#452) exists to persist them, but nothing on the daemon side subscribed, demuxed, or wrote anything -- the messages had nowhere to land. - agentexec: SubjectSystemWildcard + TaskIDFromSystemSubject, the inverse of SubjectForSystem, for a daemon subscribing to every task's system subject with one wildcard subscription. - logging: TaskRegistry manages one open TaskSink per running task attempt, mirroring every write onto the live dashboard Feed the same way a locally-written log would appear. Nil-receiver-safe, matching every other optional composition-wired Daemon field. - daemon: Daemon.process opens a task's log sink for its own duration (covering both the sandboxed-container and in-process dispatch paths uniformly) and closes it on every return path. - cmd/archied: constructs the registry, subscribes core NATS to SubjectSystemWildcard, and wires TaskLogs into both Daemon and webui.Server. - webui: archiving a task now removes its log directory too, so archived tasks' logs don't accumulate forever. Documented as a known, accepted race: a task can reach terminal status (satisfying ArchiveTask's precondition) before process() itself returns and closes the sink, so a very late system-log write can race an archive's Remove. Not corrupting -- the write lands on a soon-to-be-unlinked inode and is discarded -- and not worth fixing here: it would mean tying Close to the store transition instead of to process()'s return, a larger change than this issue's scope. Adversarial review (fresh subagent) found and fixed two real issues in the first draft: a doc comment accidentally spliced between two functions during an extraction meant to fix a lint regression, and a genuine test gap on openTaskLog (added, using TaskRegistry.Write's own true/false contract as the observable -- no Store/Forge/Trees fakes needed, since openTaskLog only touches TaskLogs and Log). |
||
|
|
4131f05036 |
feat(webui,gateway): wire config, channel, log-feed, and work-intake seams
Composition root work tying the config resolver, channel manager, log feed, and executable workflow registry into the running daemon and dashboard server: web.Channels/LogFeed/Workflows/ConfigProvenance, channel-reload wiring, TaskStopper for the Stop action, and SpawnRequest.Body/WorkRequests threading dashboard "Start work" requests through the same admission path chat uses. Touches all three features at once because this is where they're composed, not owned by any single one. Refs #53, #54, #55 |
||
|
|
68ba3456ee |
feat(webui): chat from the dashboard
Brings the chat surface to the Web UI: /api/chat with its own handlers and tests, a self-contained ui/src/chat feature folder, and the shared plumbing to reach the same router Telegram uses. Includes a dangerous-command approval path (internal/gateway/dangerous.go, internal/webui/dangerous.go, and the Telegram counterpart) so /rollback and /stop are gated the same way on both surfaces, persona handling for web sessions, and release-update formatting split out of the Telegram service so both can render it. web_parity_test.go pins that the two surfaces expose the same commands. Authored in a parallel session; committed here so it lands with the rest rather than sitting uncommitted while other work builds on top of it. task check passes with it in place. Screenshots used while building are gitignored (*.png) rather than committed, matching the call made on the earlier design reference. |
||
|
|
170fe0250b |
fix: close the duplicate-work loop and the dead paths around it
Applies the four-reviewer pass over the forge/dashboard-actions work. The through-line is that several capabilities were wired but inert, and one of them could cost real work. DUPLICATE WORK. "Closes #N" was dropped from PR bodies in favour of LinkBranch, which is sidebar linkage on Gitea and a no-op stub on GitHub -- neither closes anything, and nothing else did either. So a merged task left its issue open, labelled and assigned; the next poll re-enqueued it, and once the dashboard's Clear removed the task row that became a second implementation and a second PR for work already merged. reconcilePRs now closes the issue on merge, and a rejected task closes its issue too. A PR closed without merging deliberately leaves the issue open: that usually means "try again". MAX RETRIES. Configured, defaulted and reported by /api/config, and enforced nowhere -- markDead was unreachable and EffectiveMaxRetries had no callers, so the dashboard's retry incremented retry_count forever. The retry action now retires a task to dead at the cap. DEAD ON ARRIVAL. Tasks() never selected plan or source, so the "Decision required" panel could not render and chat tasks -- whose issue number is synthetic -- linked to forge issues that do not exist. COSMETIC FAILURE, FATAL RESULT. OpenPR failed the whole stage when LinkBranch errored, so the pull request was never opened and the task parked; a retry re-links the same branch, which Gitea answers with a conflict, so every retry parked again. It is best-effort now. INVISIBLE ACTIONS. Approve/retry/reject wrote store rows and nothing else, so the timeline and activity stream never showed that a human had intervened. They emit events now. Also: store write failures answer 500 rather than 409; the dashboard surfaces the server's actual reason instead of "409 Conflict"; action buttons stop propagation (acting on a task no longer also expands it), disable in flight, keep per-task errors instead of one shared string, and confirm before rejecting. Removed: the maybeRetryParked shim and markDead with their now-vacuous tests, WaitingTasks (no consumers; its tests were rewritten to assert the same routing through TaskByID), write-only Daemon.Runtime, and dir/dirs response bloat from /api/skills. skillSource checked workDir before sharedDir, mislabelling every shared skill when the two are equal. DefaultRoots now dedupes roots differing only by a trailing separator. The air build rule claiming to keep ui/dist current never ran -- it required one file to satisfy include_dir and include_file together, and ui/ is excluded from the walk. Deleted with the comment that promised it. task test:ui joins task check and CI; it ran nowhere before, so the dashboard suite could rot silently. Docs corrected where they described behaviour that no longer exists: parks no longer comment on the issue, state labels no longer drive retries, and forgerpc's doc claimed four proxied methods while listing three of five. |
||
|
|
4d44ef76f5 |
feat(webui): rebuild the dashboard
The old UI was a single 200-line Go file serving one hand-written HTML page of task rows. It showed that archied was running and almost nothing about what it was doing -- skills, workflows, memory, channels, logs and configuration were all invisible unless you read the source or the config file. The dashboard now covers each of those as its own section, built as a Vite app under ui/ at the repo root: it is a separate artefact from the Go daemon, not an internal package of it. dist/ is committed so building archied never needs a Node toolchain, and the !no_ui build tag keeps an embed-free build available. Both sides are split by concern rather than by layer -- one Go file per API area, one folder per UI section with its own CSS -- so a change to logs cannot disturb tasks. Pages report consequences, not just state: an empty memory page says Archie starts every conversation with no recollection of earlier ones, and the setup checklist names what is missing in plain language instead of showing a grid of zeroes. Nothing greets by a hardcoded name; the operator comes from [chat], which is deployment data. Auth is exposure control, not capability control -- the agent already holds this user's authority through its own tools. So a loopback bind needs no token, and only a reachable bind mints one and logs a URL, which means an instance cannot be exposed merely by omitting config. Container startup now degrades instead of aborting, for the reason resolveForge already does: a missing agent image should not deny the operator chat and the dashboard, and under Restart=on-failure it turns a recoverable problem into a crash loop with the real error scrolling past. `task dev` runs the daemon under air and the Vite server together in one terminal. |