Commit Graph
377 Commits
Author SHA1 Message Date
sam f76645bc2c Merge pull request #559 from samcharles93/worktree-reflective-cuddling-salamander
fix(gateway): replay completed-duplicate turns with their tool activity
2026-08-22 20:47:12 +10:00
sam 068a1f493a fix(gateway): replay completed-duplicate turns with their tool activity
A NATS redelivery or restart RecoverTurns redelivery hit the
TurnClaimCompleted branch in TurnRunner.Run and replayed only
turn.ResponseText, never the tool calls that produced it -- neither the
ledger nor the session store persisted them. The transcript showed the
original answer beside a tool-less duplicate, disagreeing about what ran.

TurnRecord now carries ToolCalls, saved in the same call as ResponseText via
a toolCallRecorder that tees Generate's TurnStream. The completed-duplicate
replay walks turn.ToolCalls into stream.ToolCall before replaying the text,
so a redelivered turn narrates the same activity as the original.

The other replay path (prior-reply, matched by conversation history rather
than turn ID) stays deliberately text-only: it exists for the crash race
where the assistant message was saved but the turn record was never
completed, so there is no persisted tool record for it to replay either.
Documented on Run's doc comment rather than left implicit.

Closes archie-core-ippu.
2026-08-22 20:43:29 +10:00
sam f4e79071c1 Merge pull request #558 from samcharles93/feat/code-fence-aware-truncation
fix(telegram): preserve code fences across message splits
2026-08-22 20:35:52 +10:00
sam 96572656f4 fix(telegram): preserve code fences across message splits
Oversized replies used to split at any newline, which could leave a Telegram message with an open Markdown fence. Fence-aware splitting now moves complete blocks intact or closes and reopens long blocks while keeping every part bounded.
2026-08-22 20:27:16 +10:00
sam 8a799cd8cd feat(forge): observable webhook receiver + dedup regression proof; decide adversarial review design
archie-core-7d5u.5 (idempotency and poll/webhook parity): the dedup
mechanism (TaskEnvelope.IdempotencyKey + PublishUnique + JetStream
Nats-Msg-Id dedup) and the shared dispatch predicate were already fully
wired by 7d5u.4. What was missing: proof the dedup actually holds, and
observability for a wedged receiver.

- internal/infrastructure/eventbus/nats/dedup_test.go: regression test
  proving PublishUnique suppresses a duplicate publish on the same
  WorkQueuePolicy stream ARCHIE_TASKS uses in production.
- internal/forge/webhook/receiver.go: Receiver.Status() tracks
  authenticated deliveries separately from successful publishes; a GET
  to the receiver's listen address (GitHub only ever POSTs) now returns
  it as JSON, so a wedged/non-delivering receiver is observable instead
  of indistinguishable from an idle repo.

Poll/webhook parity (disabling webhook still polls) needed no code
change: Cycle()/Run() never reference cfg.Forge.Intake anywhere in
internal/daemon, only setupForgeWebhook gates the webhook HTTP server.

archie-core-h019.1 (design: adversarial self-review stage): one-page
decision at docs/prds/adversarial-self-review.md. Reviewer runs in a
second workspace pointed at a .git-stripped snapshot of the reviewed
commit, never the implementer's worktree, so isolation is a property of
what's mounted rather than a prompt instruction. Findings contract is
workflow.ReviewFinding, owned by internal/domain/workflow. A surviving
error-level finding sets StatusParked, not WaitingHuman -- the epic's
informal wording named two different states in the actual machine; this
resolves which.

Both changes reviewed by a fresh subagent with no memory of the
implementation, per CLAUDE.md's adversarial-verification requirement;
zero blocking findings, two minor coverage gaps closed.
2026-08-22 19:45:31 +10:00
sam 875b7a50d5 feat(forge): GitHub webhook receiver for immediate issue dispatch
Add [forge] intake = "poll" (default) | "webhook" | "both", and a forge
webhook receiver (internal/forge/webhook) that verifies a GitHub webhook's
HMAC signature, decodes the "issues" event, and publishes a matched issue
through the same path the poller uses -- so a labelled or assigned issue
starts a task the moment it happens instead of waiting up to poll_interval
(bead archie-core-7d5u.4).

The dispatch predicate (workintake.MatchesDispatch/RequiresLabel) is now a
single shared definition consumed by both the poller and the webhook
receiver, so the two cannot drift -- including the empty-label guard from
e19ff3a (an empty label must never match every open issue). Assignee
matching is case-insensitive to agree with AssignedIssues' server-side
GitHub filter; label matching stays case-sensitive, since GitHub labels
are case-distinct.

daemon.publishTask is exported as PublishTask, the single enqueue path
now shared by the poller and the webhook intake.

Two failure modes fixed during adversarial self-review before this landed:
a webhook secret that fails to resolve at runtime now degrades (logs and
skips the receiver) instead of aborting the whole daemon -- under
intake="both" the poll exists specifically as a backstop for this exact
misconfiguration, so crashing boot over it defeated its own purpose;
multi-identity deployments ([[identities]]) are explicitly unsupported for
now (webhook intake refuses to start, logs why, and leaves each identity's
own poll loop unaffected) rather than silently matching issues against the
wrong identity's dispatch rules.
2026-08-22 19:21:09 +10:00
sam 3ade200ca7 feat(webui): payload field mapping -- bind JSON paths to named fields, preview before saving
Adds t2db.3: schema-by-example field mapping for the no-code playbook epic.
An operator picks a captured event, clicks through its payload to bind named
fields to JSON paths, and previews the resolution against a real event
before saving -- a missing, null, or type-changed path fails loudly (named
by field and event) rather than handing an agent a blank value.

- internal/domain/mapping: Mapping/Field types, path parsing/walking,
  Resolve (per-field failure reporting), Validate (incl. duplicate names)
- internal/store/mappings.go: SQLite persistence, mirrors captures.go
- internal/webui/api_mapping.go: CRUD + preview HTTP API
- ui/src/mappings/: dashboard editor (clickable payload tree, live preview)

Design decisions recorded in docs/prds/payload-field-mapping.md. t2db.4
(binding a mapping + matcher to a workflow) is out of scope here.
2026-08-22 19:18:15 +10:00
sam 51bb80f6a1 feat(nats): run an embedded server when no external NATS is configured
Wire github.com/nats-io/nats-server/v2 into production startup so a
single-process deployment gets reaction delivery and task distribution
over an in-process server rather than a separate one
(docs/prds/embedded-nats.md).

Meaning change: [nats].url empty used to mean "no NATS, use the SQLite
ClaimNext flow"; it now means "start an embedded NATS server". The old
behaviour is the explicit opt-out [nats].mode = "off". mode resolves from
url when unset (url set -> external, url empty -> embedded), and
agent.mode="nats" / [containers] now require mode="external" (an embedded
loopback server cannot be reached by agent workers or containers).

The daemon owns the embedded server's lifecycle: StartEmbedded before
dialing the client, Shutdown after the client closes (LIFO cleanups). The
embedded server binds loopback on a random port, enables JetStream in a
store dir under the data directory, and leaves its logger nil so a startup
failure surfaces as a Connect error rather than an os.Exit.

Retention becomes a nats.Config field (default WorkQueuePolicy) so a future
reaction stream can fan out under LimitsPolicy instead of the hardcoded
competing-consumer policy that starves a second consumer on overlapping
subjects.
2026-08-22 05:18:31 +10:00
sam 52b37333f4 fix(webui): stop the capture live-event notification duplicating payloads into the unbounded events table
Adversarial review of the just-shipped event inspector (3771492) found that
handleCapture's s.emit() call embedded the full (up to CaptureMaxBodyBytes,
default 256KiB) redacted body and headers into the events table's Data
column on every capture. That table has no retention or row-count prune
anywhere in the codebase -- unlike captured_events, whose three disk-bound
mechanics (size cap, rate limit, prune-on-write) this feature exists to
provide. An unauthenticated, rate-limited-only sender could grow the events
table without bound even though captured_events itself stayed correctly
capped, silently defeating docs/prds/event-capture-storage.md's stated
"max_events * max_body_bytes" worst-case guarantee.

Fix: the live "capture" SSE event now carries only {id, source} -- enough
for the dashboard to know something arrived -- and the event inspector
treats it purely as an invalidation signal, refetching GET /api/captures
for the authoritative row data instead of merging partial event fields into
its local list. This also fixes two smaller issues the same review found:
the previous approach's frontend dedup-by-id logic depended on a field
(Data["id"]) no test actually asserted, and received_at differed in
precision between the live-pushed and REST-listed copies of the same row --
both are moot now that the frontend has one source of truth for row data.
2026-08-22 05:16:49 +10:00
sam 37714928fc feat(webui): event inspector -- see captured payloads in the dashboard
Adds GET /api/captures (token-gated, matching the memory/skills
enabled-false-not-an-error convention when Server.Captures is nil) and a
new dashboard page at /captures listing recent inbound captures, newest
first, with per-row expand to explore the redacted payload and headers
pretty-printed.

Live updates reuse the existing operator-activity event pipeline
(Server.emit -> Events.Publish -> the /events SSE stream every other page
already subscribes to) rather than polling or inventing new push
plumbing: handleCapture now emits a Kind "capture" event with the same
field shape the list endpoint returns, so a captured event appears in the
inspector without a manual refresh.

Frontend follows the tasks/task-row.js precedent: row rendering is split
into ui/src/captures/capture-row.js so it can be unit tested without the
page's fetch/SSE wiring.

"Unbound" is shown for every row today since no binding data model exists
yet (t2db.4) -- the column is wired now so that bead only needs to change
what it renders, not add it.
2026-08-22 05:10:27 +10:00
sam 101c133239 feat(capture): store unbound webhook events, retention-bounded
Adds POST /webhooks/capture/{source}: an unauthenticated endpoint (mounted
alongside /healthz, bypassing dashboard token auth by design) that persists
any inbound webhook archie has no binding for yet, so it can be inspected
and mapped later (t2db.2/t2db.3).

docs/prds/event-capture-storage.md decides SQLite over NATS JetStream for
storage: NATS is not unconditionally available until 7d5u.2 ships embedded
support, and capture's single-writer/single-reader access pattern doesn't
need JetStream's fan-out/replay machinery anyway.

internal/store/captures.go adds a captured_events table with prune-on-write
(retention age + row-count cap, both configurable). The HTTP handler applies
webhookguard redaction before persisting, and enforces two more independent
disk-bound mechanics before that: a per-request body-size cap and a
per-remote-address rate limit. New [capture] config section with documented
defaults; all fields optional, capture is on by default.

Adversarial review (manual, after the usual fresh-subagent pass hit a
session API limit mid-run) caught and fixed three real defects before this
landed, each with a regression test confirmed to fail against the pre-fix
code:
- the body-size cap silently no-op'd when CaptureMaxBodyBytes was left
  unset, the one bound that limits a single payload's size rather than row
  count or write rate
- the rate limiter was keyed by the request's own source path segment --
  unregistered and attacker-chosen by design for this endpoint, violating
  webhookguard.RateLimiter's documented bounded-key-space assumption and
  letting a sender bypass the limit by rotating the segment; rekeyed to
  remote address
- the retention prune compared time.RFC3339Nano strings, whose trimmed
  variable-width fractional seconds break lexicographic ordering within the
  same second; switched to fixed-width RFC3339

Closes archie-core-t2db.1.
2026-08-22 04:51:25 +10:00
sam 311f5f71f9 fix(gateway): stop ClaimTurn's insert-race branch losing its own retry read
Found while investigating an `ineffassign` finding, not a stylistic report:
the branch handling "another connection inserted the same turn between our
read and insert" was discarding its own result.

`result, err := tx.ExecContext(ctx, ...)` for the INSERT sits inside
`if errors.Is(err, sql.ErrNoRows) { ... }`, a new block scope, with `result`
new to that scope -- so Go's `:=` declares a fresh, block-scoped `err`
shadowing the function's own `err` for the rest of the block, including the
retry read a few lines later (`err = tx.QueryRowContext(...).Scan(...)`).
That assignment landed in the shadow, which goes out of scope at the block's
closing brace. The `if err != nil` check afterward was therefore still
looking at the *original* err -- sql.ErrNoRows from the very first read --
regardless of whether the retry read succeeded. The loser of a genuine
insert race would always return `sessionstore: read turn: %w` wrapping
`sql.ErrNoRows`, contradicting the branch's own comment: "apply the normal
claim state machine instead of surfacing a uniqueness error."

Fix: the insert-path errors are now insertErr, not err, so the retry read's
plain assignment reaches the outer err the later check actually inspects.
Confirmed via careful trace of Go's block-scoping rules that this closes the
gap for the affected!=1 (loser) path without touching the affected==1
(winner) path, which returns from its own separately-scoped err before ever
reaching this one.

Commit the honest limitation along with the fix: a genuine two-connection
concurrent regression test could not be made to hit the exact branch. A
temporary, reverted-before-commit diagnostic (real concurrent race,
artificially widened with a sleep) showed the losing connection instead
fails its own INSERT outright with SQLITE_BUSY_SNAPSHOT under WAL mode with
this driver, before ON CONFLICT DO NOTHING can ever report 0 rows affected
-- meaning the specific branch this bug lived in may be effectively
unreachable via this driver/journal-mode combination today. The shipped
test verifies the weaker, still real property that matters operationally:
many genuinely concurrent same-turn-ID claims across separate connections
never produce the bug's error signature ("sessionstore: read turn:"),
distinct from unrelated SQLITE_BUSY/SQLITE_BUSY_SNAPSHOT lock contention,
which is retried rather than treated as a failure. The fix stands
regardless of provable live reachability -- the shadowing was unambiguous
and wrong on its own terms.

Adversarially reviewed by a fresh subagent, which independently reverted
just the fix (keeping the test) and confirmed empirically -- 20 runs, plus
1000 trials with a debug probe on the affected!=1 branch -- that the branch
is never reached under this driver, corroborating the test's own stated
limitation rather than contradicting it. gocognit/nestif findings on
ClaimTurn are pre-existing on the prior committed baseline, unchanged by
this fix (only variable names moved).
2026-08-22 02:03:27 +10:00
sam a0d569da09 fix: address simple findings from a repo-wide golangci-lint sweep
Six mechanical fixes from `golangci-lint run ./...` (30 findings total, 23
deliberately left -- see the next commit's message for the one that grew
into something bigger, and the following note for the rest):

- apply.go: %v -> %w for the wrapped yaml error, matching the identical
  double-%w pattern already used in decode.go/discovery.go/loader.go in the
  same package (errorlint).
- contract_test.go: filepath.Join("provider.go") with a single argument is a
  no-op join; replaced with the literal (gocritic).
- inprocess_test.go, yaegi_test.go: two goroutines whose whole purpose is
  timing (does Execute return promptly on cancellation / does Run block
  forever), where the call's result is irrelevant -- explicitly discarded
  with `_, _ =` and a comment saying why, rather than silently ignored
  (errcheck).
- find_test.go: an unchecked os.MkdirAll now matches the file's own
  createTestFile helper, which already checks and t.Fatals on the same
  operation (errcheck).

Not touched, and why: 18 cyclop/gocognit/gocyclo/funlen/nestif findings
require extracting helpers to reduce complexity -- the exact shape of two
regressions already recorded in CLAUDE.md's Lint-driven changes section
(one from --fix silently dropping a predicate, one from a nestif extraction
that doubled data), so a blind sweep across 9 unfamiliar files was declined
rather than risk a third. 2 godox findings (storage.go, worktree.go) mark
real missing work; deleting the comment would hide debt, not fix it. 3
recvcheck findings (Duration, TaskEnvelope, Provenance) each mix a
mandatory-pointer-receiver method (UnmarshalText/UnmarshalJSON's interface
contract, or genuine mutation) with correctly-value-receiver read-only
methods -- forcing consistency would either break interface conformance or
force pointer receivers onto call sites where the value isn't addressable.

Adversarially reviewed by a fresh subagent: all six confirmed
behavior-neutral or behavior-improving, all touched packages pass -race,
golangci-lint confirms exactly 23 issues remain (not more, not fewer), and
none are new.
2026-08-22 02:03:11 +10:00
sam 7c1d34ac85 feat(webhookguard): shared HMAC, rate-limit, and redaction mechanics
archie-core-7d5u.3 -- storage-agnostic building blocks for vetting inbound
webhook-shaped requests, extracted rather than duplicated:

- VerifyHMAC: behavior-preserving extraction of internal/channels/webhook's
  private validateHMAC. internal/channels/webhook now calls it instead of
  keeping its own copy; all 10 of that package's existing tests still pass
  unmodified, and the one now-redundant unit test was removed with a
  comment pointing to where its coverage relocated.
- RateLimiter: per-source token bucket. Source IDs come from a registered,
  operator-controlled set, not unbounded attacker input, so the bucket map
  is not itself bounded -- documented explicitly rather than left implicit.
- RedactPayload: key-name-based redaction, matching gateway/stream.go's
  sensitiveParameterKey marker list. Deliberately does NOT reuse that
  function's actual behavior: it replaces every string value regardless of
  key, correct for a bounded chat summary and wrong here -- a captured
  payload exists to be inspected for field mapping (t2db.3), so only values
  under a sensitive-looking key are touched.

Narrowed from 7d5u.3's original scope, which bundled capture storage and an
approval state machine into this package. Both moved to the beads that own
that state (t2db.1's capture store depends on 7d5u.2's still-open NATS
storage decision; t2db.4's binding is the only consumer of approval state
today) rather than building shared abstractions ahead of a second real
consumer.

Adversarially reviewed by a fresh subagent with zero surviving findings:
token-bucket math hand-traced on two scenarios and race-clean under -race,
the redaction/gateway-stream.go behavioral divergence verified by reading
both functions side by side, both the redaction and rate-limit guards
mutation-tested by disabling each and confirming the suite catches it.

task check exit 0, golangci-lint 0 issues, gofumpt clean.
2026-08-22 01:28:58 +10:00
sam 2873cbba82 fix(update): verify an update took effect instead of trusting the report (#530)
A pending update report used to be relayed verbatim. It is entirely
self-reported: scripts/archie-update-watchdog decides "passed" from
`curl -sf $HEALTH_URL/healthz`, which only proves that *a* daemon is
answering, then copies the version the installer *intended*
(ARCHIE_UPDATE_INSTALLED_GATEWAY) into the report. If the new binary
never made it into service, the previous daemon answers the probe just
as happily, so the message read "Update complete: daemon 1.9.8 -> 1.9.0.
Health check passed - archie-core is healthy on the new version" about a
version that was not running. That shipped: /update reported success and
the next task hit the exact bug the update was meant to fix (#468).

Report.Verify now checks the claims against the versions components
report for *themselves* -- archied's compiled-in gatewayVersion, which no
installer can talk it out of -- and partitions every claimed component
into Confirmed, Drift or Unverified.

What changed in MEANING:

- A passed health check used to mean "the new version is running". It
  now means only "something answered"; the version claim is a separate,
  separately-checked assertion.
- "Update complete ... healthy on the new version" used to be the only
  success wording. There are now three outcomes, because collapsing them
  is how the false success happened: confirmed, unconfirmed ("I could
  not confirm which version is actually running"), and contradicted
  ("Update did NOT take effect").
- An unverifiable component is NOT a failing one. Absent, empty,
  "unknown" (the watchdog's own default) and "dev" (an unstamped build)
  all mean "we could not check", never "it is wrong" -- treating them as
  drift would turn healthy updates whose bookkeeping went missing into
  reported failures.

Verification covers the daemon only, deliberately. archied's compiled-in
runtimeVersion records what its own pipeline stamped, not what the agent
container runs, so vouching for the agent would be the same kind of
guess this change exists to remove. The agent is therefore Unverified on
every successful update, and the success message names it as unchecked
rather than letting the confirmed daemon silently vouch for it.

Drift detail is attached per component: one component can sit on the
version the update meant to replace while another is on some third
version, and a single explanatory sentence would state a falsehood about
one of them.

Regression tests were confirmed to fail against the pre-fix code,
reproducing the exact false-success string, and each new guard was
mutation-tested. The drift-ordering test's component count is
load-bearing and documented: Go rotates small maps rather than permuting
them, so 8 components let an unsorted implementation pass ~12% of the
time while 9 fails it outright.

Refs #522, #468
2026-08-21 21:42:48 +10:00
sam e19ff3a1de fix(daemon): refuse to poll on an empty label-trigger match (#529)
GitHub's issues-list API treats an empty label filter as no filter at
all, so dispatch.trigger = "label"/"either" with cfg.Label unset
silently returned every open issue in the repo. A live incident
(GH#445) queued 124 unrelated issues in one poll cycle after
[dispatch.labels] -- a different field mapping task states to their
own labels -- was configured while the actual trigger-match label was
left blank.

pollIssuesWithConfig's "label" branch now refuses to poll and logs an
error when cfg.Label is empty; pollEitherWithConfig skips only the
label-side poll in that case, still returning assignee matches.
validateDispatch rejects the same misconfiguration at config-load
time. A per-identity gap remains: identities have no Label field of
their own and share the root's, so an identity overriding
dispatch.trigger to "label" isn't caught by validation -- only by
the runtime guard, which is why the runtime guard matters even with
validation in place.

Fixes archie-core-1786637501009-424-5b8332dd / GH#445.
2026-08-20 23:19:53 +10:00
sam 7a40cc8636 fix(gateway): stop persona from asserting unverified claims as fact (#528)
The chat agent labelled unchecked figures "Verified details" and
fabricated an entire "Hermes gateway" subsystem with invented file
paths, stating both as confirmed fact until directly challenged.

Reword core rule 1 to separate static tool-capability claims (fixed
by the tool list) from dynamic file/memory/action/result claims,
which now must be verified this turn rather than merely "in this
conversation" -- the looser wording let a stale claim be treated as
still verified. Add rules 8 and 9: never label something
verified/confirmed without a this-turn tool check, and never state a
specific system detail (path, subsystem, count, config value) with
confidence unless just retrieved.

Fixes archie-core-3jag / GH#524.
2026-08-20 23:19:34 +10:00
sam 8792c17538 fix: restore provider gate and dashboard token accounting
Restores the provider test gate and durable dashboard token accounting.
2026-08-20 21:16:15 +10:00
sam 7c409aadb5 feat(update): stream live progress and roll back failed updates automatically
The /update command went silent between "starting" and one final message,
gave no version summary, and required manual intervention on failure.

CommandInstaller now streams the installer script's stdout live instead of
buffering it, and parses a structured result (previous/installed versions,
per-component status) from a sentinel line. Because the reference install
script hands the daemon restart off to a detached systemd unit specifically
so restarting archied.service doesn't kill the process that requested the
update, the outcome is reported in two phases: the synchronous build/install
result comes back through Install() itself, and the post-restart health
check/rollback outcome is written by a new watchdog script and relayed by
whichever archied process boots next, via a small JSON handoff file cleared
once relayed.

Also fixes three defects an adversarial review pass found in the first
draft: the caller's request/handler context was passed straight into the
install child process, so a concurrent /restart or a browser disconnect
could SIGKILL it mid binary-copy; Telegram and the web dashboard each
tracked "install in progress" with their own private flag despite sharing
one Service, allowing two installs to race the same .prev backups; and the
watchdog reported rolled_back:true even when no backup existed to restore,
which would have told an operator to stand down while the daemon was still
broken.

internal/webui gains an unauthenticated /healthz the watchdog polls after
restarting archied.
2026-08-19 01:47:29 +10:00
sam 39101dba33 fix(telegram): drain live renderer in concurrency test 2026-08-18 09:58:42 +10:00
sam eca391b9e8 fix(agentworker,daemon): reconcile worktree ownership before push
Agent containers run as root (no USER in the Dockerfile, no
userns-remap), so a commit the agent writes to the bind-mounted
worktree lands owned by UID 0 on the host. The daemon then reads
those same loose objects to push, running as its own non-root host
user, and fails with "openat objects/../..: permission denied".

containerEnv() now passes the daemon's own UID/GID as
WORKTREE_UID/WORKTREE_GID. hybridTrees.Push -- the single point
where a commit's objects are handed back for the actual push --
chowns the worktree to that owner first, via os.Lchown so a
legitimately-tracked-but-dangling symlink doesn't abort a push that
would otherwise have succeeded.

Closes archie-core-1786749700606-6-bf6aa32d.
2026-08-18 09:05:31 +10:00
sam 0b5c808400 fix(container): stop a cancelled task's container immediately
Release() always slept the configured GracePeriod and always used
Docker's default ~10s graceful SIGTERM-then-SIGKILL timeout, even
when the release was happening because /stop had just cancelled the
task's context. A cancelled task's container -- and the in-flight
generation/tool calls inside it -- could sit doing nothing for up to
GracePeriod+~10s before actually dying, so /stop reported success
while work visibly continued for tens of seconds to a couple minutes.

releaseDecision(ctx, grace) now skips the grace sleep and requests an
immediate SIGKILL (Timeout: 0) when ctx is already cancelled; a live
ctx (normal completion) keeps the original behaviour byte for byte.

Closes archie-core-1786749700580-5-e3878224.
2026-08-18 09:05:21 +10:00
sam a5a878c2d0 feat(secret): add sops, vault/openbao and age engine plugins
CLI-based, env-configured, cache-less engines resolving SecretRef values
through the existing Yaegi plugin surface. Fresh resolve per call, so
values updated in the backend are picked up without daemon restart —
closing the gap that motivated the remote secret-input epic (#504),
whose value-entry web form duplicated the Bitwarden app for bws.

- examples/secret-engines/: sops (SOPS_FILE), vault (VAULT_ADDR/TOKEN,
  key "path/field" or VAULT_SECRET_PATH prefix), age (AGE_FILE/
  AGE_IDENTITY, key=value lines). //go:build ignore keeps them out of
  the Go toolchain; yaegi treats the tag as a comment and loads them.
- enginehost: host-side Run helper registered as a Yaegi symbol. yaegi's
  interpreted os/exec cannot pass env to child processes, so plugins
  shell out through host code; bounded by a 30s timeout like the bws
  engine.
- secret.LoadDir seeds the interpreter with the host environment: yaegi's
  interpreted os package keeps a private env otherwise, and plugin
  os.Getenv would silently see nothing.
- cmd/archied wires enginehost + unrestricted symbols into the engine
  plugin loader, matching skillscript's interpretation posture.
- tests load the real shipped plugin files with fake CLIs on PATH;
  added CONTEXT.md glossary for the remote secret-input vocabulary.
2026-08-18 00:32:57 +10:00
sam 8239e8e935 style: apply gofumpt formatting 2026-08-17 00:02:31 +10:00
sam 62adab69af fix(update): stamp install type and bound retry backoff 2026-08-16 23:59:48 +10:00
sam edf38eb3e4 refactor: extract daemon bootstrap into phased wiring 2026-08-16 03:12:18 +10:00
sam 4374bfd57c refactor(builtin): split read executor to lower complexity 2026-08-16 01:37:04 +10:00
sam 81a8b9b5e1 style: remove trailing whitespace 2026-08-16 01:28:25 +10:00
sam 2c6a4f9adc fix(tests): check type assertions in overlay and gateway tests 2026-08-16 01:24:31 +10:00
sam 63cff4fe53 refactor: flatten nested config and tool fallback logic 2026-08-16 01:20:48 +10:00
sam 79a4db5362 fix(tools): flatten write overwrite checks 2026-08-16 01:16:22 +10:00
sam fa826fb486 fix(email): use context-aware test networking 2026-08-16 01:10:48 +10:00
sam 4e03cd7c17 test: use t.Context() for helper subprocess 2026-08-16 01:08:47 +10:00
sam 282f67b6b6 fix(tools): document intentional fallback skips 2026-08-15 23:55:21 +10:00
sam f5f57b3097 fix(tests): add contexts to HTTP test requests 2026-08-15 23:43:55 +10:00
sam f112daa1cd fix(tools): avoid predeclared max names 2026-08-15 23:41:44 +10:00
sam cb8c99b978 fix(indexing): remove unused code search test runner 2026-08-15 23:40:23 +10:00
sam b5064e570f fix(telegram): preserve URL boundaries in responses
Keep bare GitHub issue and pull-request links separate from following prose across one-shot sends, live edits, and long-message splitting. Preserve Markdown delimiters and newline boundaries while retaining plain-text fallbacks.
2026-08-15 23:09:28 +10:00
sam 0dfdd72f83 feat(gateway): polish /status command response format
Format /status response with clear heading, visual task status icons, canonical lifecycle ordering, and separated runtime details with graceful fallbacks.
2026-08-15 22:43:41 +10:00
sam db9544dc53 feat(ratelimit): add window-based stale-entry eviction
EvictStale sweeps the per-key hit map and removes entries whose hits
have all aged out of the sliding window, so a long-lived limiter
driven on a ticker doesn't accumulate one map entry per identity for
the life of the process. The package doc now states the map is
unbounded until something actually calls EvictStale, rather than
claiming it's bounded by a mechanism nothing yet wires up.
2026-08-15 22:25:07 +10:00
sam 39d8b067ec feat(agentexec,workflow): surface tool calls on the task timeline
archie-core-467 wired tool-call visibility into the interactive chat
gateway, but a workflow task's own transcript never showed what its
tools actually did -- only stage/agent-finish events existed, so
there was no way to confirm an agent's summary matched what it really
ran.

Adds a ToolCallReporter, invoked once per completed call from
toolExecute (the single choke point every tool call passes through
before reaching a model), reported via a new ToolSetOptions.OnToolCall
hook. Runner.Run grows a report parameter to carry it in: InProcessRunner
wires it through, since it executes in the same process as its tools;
SubprocessRunner and NATSRunner accept and ignore it, since their tool
calls happen in a different process with nothing here to observe them.

The workflow engine turns each report into a tool_call event on the
task's own Bus (agent.go, implement.go), riding the same NATS
event-forwarding archie-core-518 already built to get container
events onto the daemon's timeline.

Scoped to the central tool registry (read/write/edit/grep/shell/...)
built via BuildToolSet; capture tools (structured finish output) and
run_go_script build their ToolSet entries directly and are not yet
covered.
2026-08-15 12:53:41 +10:00
sam 8f73c1febb fix(agentworker): back off before refetching a nak'd stage message
On a stage-handle failure, the main loop logged the error, called
Nak (which requests redelivery), and looped straight back into
FetchStage with no delay -- unlike the fetch-error branch three lines
above, which already backs off for a second. A deterministic failure
(a broken gate, a malformed request) therefore refetched and failed
again immediately, flooding the log with one "handle failed" line per
loop iteration instead of per genuine retry -- hundreds of identical
lines within the same second in production.

Adds the same time.Second backoff after Nak.
2026-08-15 12:53:31 +10:00
sam a37f126f79 fix(logging): resolve error attrs to their message string, not {}
SystemLogHandler and FeedHandler both json.Marshal Entry.Fields
directly rather than through slog's own JSON handler, so a logged
`error` value went straight into a map[string]any as the raw
interface. encoding/json has no Error()-aware special case, and the
concrete type behind almost every Go error (errors.errorString,
fmt.wrapError, ...) holds only unexported fields, so it marshalled to
`{}` and silently discarded the message everywhere it was logged --
including on every container/system-log line shipped to the daemon
and shown in the webui, which is where this was actually noticed: a
flood of `"err":{}}` with no way to see what had actually failed.

attrValue now resolves an error attr to its .Error() string before it
reaches the map. It also guards against the classic Go footgun where
a non-nil error interface wraps a nil concrete pointer (var e *T; var
err error = e) -- a naive err != nil check would call Error() on a
nil receiver and crash the one thing that must never crash: logging
itself.
2026-08-15 12:53:24 +10:00
sam 37792c9ec0 fix(worktree): force checkout so retry recovers from a dirty worktree
refresh checked out the task branch without Force before hard-resetting
to base. A prior attempt's leftover uncommitted changes -- a killed
container, or a stage that edits files before ever reaching a commit --
made go-git's default MergeReset checkout refuse to reconcile them, even
with HEAD already on the correct branch and the target commit unmoved.
The fallback then made it worse: any checkout failure was assumed to
mean the branch didn't exist yet, so it retried with Create: true, which
failed with a much more confusing error since the branch did exist --
"a branch named ... already exists" -- masking the real cause and
parking the retry with the same unhelpful message every time.

Reproduced directly against a live deployment's actual broken worktree
(modified tracked file, untracked file, branch already checked out) and
confirmed retry then succeeds and correctly restores to base.

Force is safe unconditionally here: refresh hard-resets to base a few
lines later regardless of what checkout leaves behind.

Closes #523
2026-08-15 11:22:22 +10:00
sam 54be74ae89 feat(releaseupdate): stamp install type at build time, fail closed on unknown
/update could report success at the correct target version while the
archie-agent container running it never actually changed: the update
mechanism trusted the configured installer's exit code and reported
whatever version number the release claimed, with nothing verifying
the two agreed.

The first piece of the fix: a new internal/installtype package
reports how a binary was built for distribution (Type(), defaulting
to "unknown"), stamped once per release artifact via
"-ldflags -X .../installtype.buildType=<value>" rather than guessed
at runtime by probing the environment -- restic and Tailscale both
treat "which install shape is this" as a build-time decision for the
same reason: a probed signal is guessable, a compiled-in one is not.

releaseupdate.Service.Install now refuses to run the configured
Installer at all when InstallType is unset or installtype.Unknown, so
an unstamped or misconfigured build can no longer silently attempt an
update it has no way to know is even the right kind for how this
instance was deployed. Wired through every real archied build path so
the gate doesn't just exist in principle: Taskfile.yml's build task
(binary), Dockerfile.archied (container), and install.sh (binary) all
now stamp it; cmd/archied/telegram_setup.go wires
Service.InstallType from installtype.Type().

Configuration-page display of install type/version and post-update
verification against the actually-running reference (not just the
installer's reported version) remain open -- this lands the
correctness gate, not the full observability surface.

Refs #522
2026-08-15 10:34:46 +10:00
sam a253d3043b fix(webui): filter turn_completed noise from the Live Activity SSE feed
turn_completed fires on every chat turn so input-driven curators can
wake on it, but it carries no task correlation (TaskID always 0,
Detail a raw session ID) -- on the dashboard's "Live activity" panel
this drowned real task-lifecycle events: 27 of the last 30 rows in a
live sample were content-free turn_completed noise.

Excluded at the SSE boundary (sseVisible), not at publish, so
curators reading the Bus directly are unaffected. since still
advances past a filtered event exactly as it would for a sent one --
EventsSince only ever returns events with id > since, so leaving
since behind a filtered event would make the next catchUp refetch it
forever.

Closes #521
2026-08-15 10:34:33 +10:00
sam 9d6a11f491 fix(agentworker): forward workflow events across the NATS worker boundary
A workflow run through the container/NATS path never wired
workflow.TaskContext.Bus, so every tc.Emit call (stage_start,
stage_finish, outcome, parked) was a silent no-op. archied's own
in-process path sets Bus and always worked; anything dispatched to an
archie-agent worker did not, which meant the dashboard's per-task
timeline showed "No events yet" for every task regardless of status
or stage -- confirmed live across a Working task and two differently
parked tasks.

Bridges the gap the same way system logs already cross this boundary:
ForwardTaskEvents relays a per-task in-process *events.Bus to a new
NATS subject (SubjectForEvents), and a new daemon-side subscriber
(subscribeAgentEvents) republishes decoded events on the daemon's own
Bus, landing them in the same persist-and-broadcast pipeline an
in-process workflow already used.

Closes #518
2026-08-15 10:34:25 +10:00
sam 8ae23164bb refactor(webui): decompose handleSSE to cut cognitive complexity
handleSSE mixed connection registration, backlog catch-up, and live
broadcast relay into one function with three nested closures --
gocognit flagged it at 52 against a limit of 30. Extracted each
concern into its own top-level piece: sseSince (resume-point
resolution), registerSSEConn (subscribe/unsubscribe), and a new
sseStream type carrying the per-connection state (send, catchUp,
sendPage, drain) that the closures used to capture implicitly.

Behavior is unchanged -- catchUp/sendPage's page-boundary and
target-reached logic is the same control flow the original closure
had, just named and independently testable. handleSSE itself is now
a straight-line sequence of calls.

Added direct unit tests against sendPage/catchUp for the boundary
cases a full HTTP round-trip test doesn't exercise: a target reached
mid-page vs. at a page boundary, a target of zero (drain everything),
a write failure stopping the page immediately, and backlog spanning
more than one EventsSince fetch.
2026-08-15 02:24:43 +10:00
sam 95bb4b5bb6 fix(telegram): use NewRequestWithContext to satisfy noctx
Pre-existing lint finding, unrelated to any behavior change.
2026-08-15 02:19:57 +10:00
sam c2b6823eba fix(telegram): guarantee no message is stranded on gateway stop/restart
A daemon shutdown or /restart cancels turnCtx without any guarantee
that the goroutine owning a liveReply ever runs again to notice and
clean up after itself -- a full process exit gives it no chance at
all. Gateway now tracks every liveReply mid-turn in a registry and
Stop drains it, marking each as interrupted so nothing is left ending
in the "typing" cursor forever.

finalize and abandon now run their bodies inside a per-reply
sync.Once (the terminal field): a turn finishing naturally and a
concurrent Stop draining the same reply were reaching it from two
goroutines at once, both sending their own edit for the same message
with the outcome decided nondeterministically by which HTTP response
Telegram processed last -- including a complete real answer getting
silently overwritten by the restart marker. Only one side's body now
ever runs; the other blocks on the winner and then does nothing.

The drain itself is bounded (liveDrainTimeout, default 5s): a single
stalled Telegram API call must not hang gateway, and therefore
daemon, shutdown indefinitely. Entries still in flight past the
deadline keep running in the background rather than blocking Stop.

A liveReply created in the narrow window between Stop claiming the
registry and its own registration call is abandoned immediately
instead of being registered into (and lost from) a registry nothing
will drain again.

Regression tests cover the mutual-exclusion race directly (20
iterations of finalize and abandonRestarted racing on the same
reply, asserting exactly one outcome ever wins), the bounded drain
against a deliberately hung API call, and the post-stop registration
gap.
2026-08-15 02:19:53 +10:00