Commit Graph
2877 Commits
Author SHA1 Message Date
sam 7bbc31be15 fix(gateway): restore media result after adapter removal
BasePlatformAdapter used to own SendResult even though no live channel implemented that adapter contract. The adapter is now gone, but launch-scoped MediaSender values still need the delivery result to classify Telegram failures. Keep SendResult with the media capability contract and make the tests describe senders instead of reviving the dead lifecycle abstraction.
2026-08-23 03:27:44 +10:00
sam 3f0de9f696 Merge pull request #562 from samcharles93/fix-agentworker-json-import
fix(agentworker): move JSONL decode into storage, unblocking CI
2026-08-23 03:00:32 +10:00
sam 27ea40fec5 fix(agentworker): move JSONL decode into storage, unblocking CI
TestApplicationOwnsNoTransportWireMechanics was failing on main since
d82b232: persistent_runner.go imported encoding/json to decode its
own project-memory JSONL log, which the architecture test treats as
a transport-wire-mechanics violation. That ban exists to keep NATS
message encoding out of the app layer -- this was local disk I/O, not
wire traffic, but the test can't tell the difference from an import
alone, and it shouldn't have to: the fix is to not decode JSON in
agentworker at all.

storage.ReadJSONLines is AppendJSONLine's read-side counterpart,
added to the package that already owns this format (see
organisation.md, "a package owns its own format end to end").
persistent_runner.go now calls it instead of scanning and decoding
itself, and no longer imports encoding/json or bufio.

This was blocking every deploy.yml run on main, independent of and
found while shipping the video-generation PR (#561).
2026-08-23 02:57:41 +10:00
sam 485b55b95b Merge pull request #561 from samcharles93/worktree-federated-wiggling-milner
feat: video generation via MiniMax, with a channel media-delivery pipeline
2026-08-23 02:54:19 +10:00
sam 61aec59c3e chore: beads interactions and issues 2026-08-23 02:51:45 +10:00
sam e04045907c feat(archied): wire generate_video end to end, gate it behind config
drainChatStream now detects a tools.MultimodalResult in a tool
result's Output and reports each URL via turn.Media(...), alongside
the existing turn.ToolCall(...) narration -- the missing link between
a tool producing media and a channel delivering it.

MinimaxConfig registers the tool only once an API key actually
resolves (registerMinimaxTool), matching web_fetch's "never advertise
a broken capability" pattern. Off by default, unlike web_fetch: this
spends real API credits per call.
2026-08-23 02:48:25 +10:00
sam 6834452870 feat(tools): add generate_video tool via MiniMax video generation
internal/tools/minimax wraps MiniMax's v2/video_generation (submit)
and v2/query/video_generation/{id} (poll) endpoints. Generation is
asynchronous on MiniMax's side but archie's tool subsystem has no
async-tool flow control (ToolEntry.IsAsync carries no flow control
today), so GenerateAndWait blocks the calling goroutine -- a tool
call, not the model's own generating goroutine -- submitting and
polling until the job reaches a terminal status or MaxWait elapses.

The poll loop tracks elapsed time purely from its injected Sleep
calls rather than a real time.Now() deadline, so a test that fakes
Sleep as an instant counter also makes the loop terminate instantly
instead of spinning in real wall-clock time.

Text-to-video only for now; image-to-video (image_url content items)
is a schema-and-routing change this can grow into. The tool returns a
tools.MultimodalResult carrying the video as a URLs entry -- the
shared envelope the gateway wiring downstream turns into a delivered
attachment.
2026-08-23 02:48:20 +10:00
sam da8ffc0908 feat(tools): add MultimodalResult.URLs for remote-hosted media
MultimodalResult only modeled local bytes written under SubdirHint
(Files). A tool whose media lives at a remote, already-hosted URL --
a generation API's result, for example -- had nowhere to put it.
MediaRef{Type, URL} and the new URLs field give it one, reusing the
existing envelope rather than forking a second convention for the
same concept.
2026-08-23 02:48:12 +10:00
sam f6d7d0dce2 feat(webui): render a media event as a link fallback
The dashboard has no inline media rendering path yet
(archie-core-1786748942243-6-f109697e). chatStreamSink.Media degrades
a MediaEvent to a link appended to the delta stream -- the same
fallback Telegram's liveReply uses when its own SendMedia call fails
-- rather than the event reaching nowhere.
2026-08-23 02:48:06 +10:00
sam d7308b9207 feat(telegram): deliver media attachments, capability-gated
telegramMediaSender routes a MediaAttachment to the matching Bot API
method (sendVideo/sendPhoto/sendAudio/sendDocument) rather than
sendMessage, which cannot carry a file. It is constructed per-launch
from the same *bot.Bot the update handler already has, matching
telegramApprover's lifetime rule: a /restart abandons an in-flight
send against the outgoing bot instance rather than silently
redirecting it through the new one.

liveReply.Media wires TurnStream into this: it checks
gateway.CapabilitiesOf before attempting delivery -- not just on
failure -- so a sender that already knows it cannot deliver a given
media kind never costs a doomed network round trip. Either path
(capability-limited or a failed send) falls back to rendering the
attachment's URL as a visible link, so a generated asset is never
silently dropped.

newMediaSender is an injectable field so tests can prove the
capability check actually short-circuits, since the real Telegram
sender always reports Media: true and can't exercise that path
itself.
2026-08-23 02:48:01 +10:00
sam 242ae53fdf feat(gateway): add MediaSender/CapabilityReporter and TurnStream.Media
Channels had no shared contract for delivering non-text tool output --
MessageEvent/MediaAttachment/MsgVideo existed but nothing produced or
consumed them. MediaSender and CapabilityReporter give a channel an
optional, capability-queryable way to deliver an attachment; TurnStream
gains Media(event) alongside Delta/ToolCall so a tool's media reaches
the channel in the same ordered pass as its text.

CapabilitiesOf takes any deliberately: it must not depend on the
soon-to-be-removed BasePlatformAdapter (archie-core-azbo), since the
live channels implement channels.Channel, not that interface.
2026-08-23 02:47:53 +10:00
sam 00c5e06152 refactor: remove dead BasePlatformAdapter code 2026-08-22 23:23:51 +10:00
sam 657fdaf3f0 chore: beads interactions and issues 2026-08-22 22:06:55 +10:00
sam 770cada41a fix(worktree): reject diffs without merge base 2026-08-22 22:06:54 +10:00
sam 6988f8b4a2 fix(worktree): reset retries to pristine base
Retry used to retain ignored and untracked output from an interrupted attempt, which CommitAll could publish later. Refresh now derives the tracked HEAD shape, removes every other worktree entry outside .git, and fails closed when the configured base cannot resolve.
2026-08-22 22:06:54 +10:00
sam 9e46e31759 fix(worktree): bind publication to dispatch grants
Worktree RPC used to accept repository coordinates from the sandbox. It now accepts only an opaque per-dispatch grant, and the daemon derives the repository path and branch from grant-owned state.

Repository worktree paths also move from an ambiguous flattened key to encoded owner/repo components. Origin-matched legacy clones migrate safely; mismatched legacy paths are preserved.
2026-08-22 22:06:54 +10:00
sam 3751f303d8 Merge pull request #560 from samcharles93/worktree-abstract-swimming-bentley
fix(container): persist npm cache for per-task MCP servers
2026-08-22 21:16:13 +10:00
sam 1c975b8348 fix(container): persist npm cache for per-task MCP servers
Ephemeral task containers re-resolve and re-download npx-launched MCP
server packages from the npm registry on every task, since each task
gets a fresh container with no persistent cache -- a worse amplifier
of a registry outage than the daemon-restart case d4bb914b already
fixed.

Extract the daemon's npm-cache env logic into mcp.NpmCacheEnv, shared
by both the daemon's own long-lived npx MCP servers and archie-agent's
per-task ones. Mount a new archie-cache-mcp-npm Docker volume at
/data/cache/mcp-npm on every task container regardless of ecosystem
(MCP servers are daemon-wide config, not tied to the task repo's
language), and point npx-launched per-task stdio transports at it.

Closes archie-core-qk8m.
2026-08-22 21:13:33 +10:00
sam 925719a962 chore: beads interactions and issues 2026-08-22 21:00:20 +10:00
sam d82b232d1b feat(storage): persist agent project state 2026-08-22 21:00:20 +10:00
sam f76645bc2c Merge pull request #559 from samcharles93/worktree-reflective-cuddling-salamander
fix(gateway): replay completed-duplicate turns with their tool activity
2026-08-22 20:47:12 +10:00
sam 068a1f493a fix(gateway): replay completed-duplicate turns with their tool activity
A NATS redelivery or restart RecoverTurns redelivery hit the
TurnClaimCompleted branch in TurnRunner.Run and replayed only
turn.ResponseText, never the tool calls that produced it -- neither the
ledger nor the session store persisted them. The transcript showed the
original answer beside a tool-less duplicate, disagreeing about what ran.

TurnRecord now carries ToolCalls, saved in the same call as ResponseText via
a toolCallRecorder that tees Generate's TurnStream. The completed-duplicate
replay walks turn.ToolCalls into stream.ToolCall before replaying the text,
so a redelivered turn narrates the same activity as the original.

The other replay path (prior-reply, matched by conversation history rather
than turn ID) stays deliberately text-only: it exists for the crash race
where the assistant message was saved but the turn record was never
completed, so there is no persisted tool record for it to replay either.
Documented on Run's doc comment rather than left implicit.

Closes archie-core-ippu.
2026-08-22 20:43:29 +10:00
sam f4e79071c1 Merge pull request #558 from samcharles93/feat/code-fence-aware-truncation
fix(telegram): preserve code fences across message splits
2026-08-22 20:35:52 +10:00
sam 96572656f4 fix(telegram): preserve code fences across message splits
Oversized replies used to split at any newline, which could leave a Telegram message with an open Markdown fence. Fence-aware splitting now moves complete blocks intact or closes and reopens long blocks while keeping every part bounded.
2026-08-22 20:27:16 +10:00
sam 8a799cd8cd feat(forge): observable webhook receiver + dedup regression proof; decide adversarial review design
archie-core-7d5u.5 (idempotency and poll/webhook parity): the dedup
mechanism (TaskEnvelope.IdempotencyKey + PublishUnique + JetStream
Nats-Msg-Id dedup) and the shared dispatch predicate were already fully
wired by 7d5u.4. What was missing: proof the dedup actually holds, and
observability for a wedged receiver.

- internal/infrastructure/eventbus/nats/dedup_test.go: regression test
  proving PublishUnique suppresses a duplicate publish on the same
  WorkQueuePolicy stream ARCHIE_TASKS uses in production.
- internal/forge/webhook/receiver.go: Receiver.Status() tracks
  authenticated deliveries separately from successful publishes; a GET
  to the receiver's listen address (GitHub only ever POSTs) now returns
  it as JSON, so a wedged/non-delivering receiver is observable instead
  of indistinguishable from an idle repo.

Poll/webhook parity (disabling webhook still polls) needed no code
change: Cycle()/Run() never reference cfg.Forge.Intake anywhere in
internal/daemon, only setupForgeWebhook gates the webhook HTTP server.

archie-core-h019.1 (design: adversarial self-review stage): one-page
decision at docs/prds/adversarial-self-review.md. Reviewer runs in a
second workspace pointed at a .git-stripped snapshot of the reviewed
commit, never the implementer's worktree, so isolation is a property of
what's mounted rather than a prompt instruction. Findings contract is
workflow.ReviewFinding, owned by internal/domain/workflow. A surviving
error-level finding sets StatusParked, not WaitingHuman -- the epic's
informal wording named two different states in the actual machine; this
resolves which.

Both changes reviewed by a fresh subagent with no memory of the
implementation, per CLAUDE.md's adversarial-verification requirement;
zero blocking findings, two minor coverage gaps closed.
2026-08-22 19:45:31 +10:00
sam ee650997c7 chore(release): archied v1.10.0 2026-08-22 19:28:07 +10:00
sam 08be2aedec chore(beads): record session interaction 2026-08-22 19:27:23 +10:00
sam b8f0e49de8 docs: remove commit-permission gating from CLAUDE.md, keep push gated
Every "do not commit without asking" line (the repository-hygiene bullet
and the Beads-managed Conservative/Minimal/Team-maintainer profile split)
required stopping mid-task for approval on finished, gate-clean work --
friction with no benefit on a solo project. Pushing and Dolt remote sync
still require explicit ask; only the commit gate is removed.
2026-08-22 19:23:16 +10:00
sam 875b7a50d5 feat(forge): GitHub webhook receiver for immediate issue dispatch
Add [forge] intake = "poll" (default) | "webhook" | "both", and a forge
webhook receiver (internal/forge/webhook) that verifies a GitHub webhook's
HMAC signature, decodes the "issues" event, and publishes a matched issue
through the same path the poller uses -- so a labelled or assigned issue
starts a task the moment it happens instead of waiting up to poll_interval
(bead archie-core-7d5u.4).

The dispatch predicate (workintake.MatchesDispatch/RequiresLabel) is now a
single shared definition consumed by both the poller and the webhook
receiver, so the two cannot drift -- including the empty-label guard from
e19ff3a (an empty label must never match every open issue). Assignee
matching is case-insensitive to agree with AssignedIssues' server-side
GitHub filter; label matching stays case-sensitive, since GitHub labels
are case-distinct.

daemon.publishTask is exported as PublishTask, the single enqueue path
now shared by the poller and the webhook intake.

Two failure modes fixed during adversarial self-review before this landed:
a webhook secret that fails to resolve at runtime now degrades (logs and
skips the receiver) instead of aborting the whole daemon -- under
intake="both" the poll exists specifically as a backstop for this exact
misconfiguration, so crashing boot over it defeated its own purpose;
multi-identity deployments ([[identities]]) are explicitly unsupported for
now (webhook intake refuses to start, logs why, and leaves each identity's
own poll loop unaffected) rather than silently matching issues against the
wrong identity's dispatch rules.
2026-08-22 19:21:09 +10:00
sam 3ade200ca7 feat(webui): payload field mapping -- bind JSON paths to named fields, preview before saving
Adds t2db.3: schema-by-example field mapping for the no-code playbook epic.
An operator picks a captured event, clicks through its payload to bind named
fields to JSON paths, and previews the resolution against a real event
before saving -- a missing, null, or type-changed path fails loudly (named
by field and event) rather than handing an agent a blank value.

- internal/domain/mapping: Mapping/Field types, path parsing/walking,
  Resolve (per-field failure reporting), Validate (incl. duplicate names)
- internal/store/mappings.go: SQLite persistence, mirrors captures.go
- internal/webui/api_mapping.go: CRUD + preview HTTP API
- ui/src/mappings/: dashboard editor (clickable payload tree, live preview)

Design decisions recorded in docs/prds/payload-field-mapping.md. t2db.4
(binding a mapping + matcher to a workflow) is out of scope here.
2026-08-22 19:18:15 +10:00
sam 51bb80f6a1 feat(nats): run an embedded server when no external NATS is configured
Wire github.com/nats-io/nats-server/v2 into production startup so a
single-process deployment gets reaction delivery and task distribution
over an in-process server rather than a separate one
(docs/prds/embedded-nats.md).

Meaning change: [nats].url empty used to mean "no NATS, use the SQLite
ClaimNext flow"; it now means "start an embedded NATS server". The old
behaviour is the explicit opt-out [nats].mode = "off". mode resolves from
url when unset (url set -> external, url empty -> embedded), and
agent.mode="nats" / [containers] now require mode="external" (an embedded
loopback server cannot be reached by agent workers or containers).

The daemon owns the embedded server's lifecycle: StartEmbedded before
dialing the client, Shutdown after the client closes (LIFO cleanups). The
embedded server binds loopback on a random port, enables JetStream in a
store dir under the data directory, and leaves its logger nil so a startup
failure surfaces as a Connect error rather than an os.Exit.

Retention becomes a nats.Config field (default WorkQueuePolicy) so a future
reaction stream can fan out under LimitsPolicy instead of the hardcoded
competing-consumer policy that starves a second consumer on overlapping
subjects.
2026-08-22 05:18:31 +10:00
sam 52b37333f4 fix(webui): stop the capture live-event notification duplicating payloads into the unbounded events table
Adversarial review of the just-shipped event inspector (3771492) found that
handleCapture's s.emit() call embedded the full (up to CaptureMaxBodyBytes,
default 256KiB) redacted body and headers into the events table's Data
column on every capture. That table has no retention or row-count prune
anywhere in the codebase -- unlike captured_events, whose three disk-bound
mechanics (size cap, rate limit, prune-on-write) this feature exists to
provide. An unauthenticated, rate-limited-only sender could grow the events
table without bound even though captured_events itself stayed correctly
capped, silently defeating docs/prds/event-capture-storage.md's stated
"max_events * max_body_bytes" worst-case guarantee.

Fix: the live "capture" SSE event now carries only {id, source} -- enough
for the dashboard to know something arrived -- and the event inspector
treats it purely as an invalidation signal, refetching GET /api/captures
for the authoritative row data instead of merging partial event fields into
its local list. This also fixes two smaller issues the same review found:
the previous approach's frontend dedup-by-id logic depended on a field
(Data["id"]) no test actually asserted, and received_at differed in
precision between the live-pushed and REST-listed copies of the same row --
both are moot now that the frontend has one source of truth for row data.
2026-08-22 05:16:49 +10:00
sam 37714928fc feat(webui): event inspector -- see captured payloads in the dashboard
Adds GET /api/captures (token-gated, matching the memory/skills
enabled-false-not-an-error convention when Server.Captures is nil) and a
new dashboard page at /captures listing recent inbound captures, newest
first, with per-row expand to explore the redacted payload and headers
pretty-printed.

Live updates reuse the existing operator-activity event pipeline
(Server.emit -> Events.Publish -> the /events SSE stream every other page
already subscribes to) rather than polling or inventing new push
plumbing: handleCapture now emits a Kind "capture" event with the same
field shape the list endpoint returns, so a captured event appears in the
inspector without a manual refresh.

Frontend follows the tasks/task-row.js precedent: row rendering is split
into ui/src/captures/capture-row.js so it can be unit tested without the
page's fetch/SSE wiring.

"Unbound" is shown for every row today since no binding data model exists
yet (t2db.4) -- the column is wired now so that bead only needs to change
what it renders, not add it.
2026-08-22 05:10:27 +10:00
sam 101c133239 feat(capture): store unbound webhook events, retention-bounded
Adds POST /webhooks/capture/{source}: an unauthenticated endpoint (mounted
alongside /healthz, bypassing dashboard token auth by design) that persists
any inbound webhook archie has no binding for yet, so it can be inspected
and mapped later (t2db.2/t2db.3).

docs/prds/event-capture-storage.md decides SQLite over NATS JetStream for
storage: NATS is not unconditionally available until 7d5u.2 ships embedded
support, and capture's single-writer/single-reader access pattern doesn't
need JetStream's fan-out/replay machinery anyway.

internal/store/captures.go adds a captured_events table with prune-on-write
(retention age + row-count cap, both configurable). The HTTP handler applies
webhookguard redaction before persisting, and enforces two more independent
disk-bound mechanics before that: a per-request body-size cap and a
per-remote-address rate limit. New [capture] config section with documented
defaults; all fields optional, capture is on by default.

Adversarial review (manual, after the usual fresh-subagent pass hit a
session API limit mid-run) caught and fixed three real defects before this
landed, each with a regression test confirmed to fail against the pre-fix
code:
- the body-size cap silently no-op'd when CaptureMaxBodyBytes was left
  unset, the one bound that limits a single payload's size rather than row
  count or write rate
- the rate limiter was keyed by the request's own source path segment --
  unregistered and attacker-chosen by design for this endpoint, violating
  webhookguard.RateLimiter's documented bounded-key-space assumption and
  letting a sender bypass the limit by rotating the segment; rekeyed to
  remote address
- the retention prune compared time.RFC3339Nano strings, whose trimmed
  variable-width fractional seconds break lexicographic ordering within the
  same second; switched to fixed-width RFC3339

Closes archie-core-t2db.1.
2026-08-22 04:51:25 +10:00
sam 3d74f0136e Merge pull request #535 from samcharles93/fix/lint-sweep-simple-findings
Fix/lint sweep simple findings
2026-08-22 02:11:41 +10:00
sam 311f5f71f9 fix(gateway): stop ClaimTurn's insert-race branch losing its own retry read
Found while investigating an `ineffassign` finding, not a stylistic report:
the branch handling "another connection inserted the same turn between our
read and insert" was discarding its own result.

`result, err := tx.ExecContext(ctx, ...)` for the INSERT sits inside
`if errors.Is(err, sql.ErrNoRows) { ... }`, a new block scope, with `result`
new to that scope -- so Go's `:=` declares a fresh, block-scoped `err`
shadowing the function's own `err` for the rest of the block, including the
retry read a few lines later (`err = tx.QueryRowContext(...).Scan(...)`).
That assignment landed in the shadow, which goes out of scope at the block's
closing brace. The `if err != nil` check afterward was therefore still
looking at the *original* err -- sql.ErrNoRows from the very first read --
regardless of whether the retry read succeeded. The loser of a genuine
insert race would always return `sessionstore: read turn: %w` wrapping
`sql.ErrNoRows`, contradicting the branch's own comment: "apply the normal
claim state machine instead of surfacing a uniqueness error."

Fix: the insert-path errors are now insertErr, not err, so the retry read's
plain assignment reaches the outer err the later check actually inspects.
Confirmed via careful trace of Go's block-scoping rules that this closes the
gap for the affected!=1 (loser) path without touching the affected==1
(winner) path, which returns from its own separately-scoped err before ever
reaching this one.

Commit the honest limitation along with the fix: a genuine two-connection
concurrent regression test could not be made to hit the exact branch. A
temporary, reverted-before-commit diagnostic (real concurrent race,
artificially widened with a sleep) showed the losing connection instead
fails its own INSERT outright with SQLITE_BUSY_SNAPSHOT under WAL mode with
this driver, before ON CONFLICT DO NOTHING can ever report 0 rows affected
-- meaning the specific branch this bug lived in may be effectively
unreachable via this driver/journal-mode combination today. The shipped
test verifies the weaker, still real property that matters operationally:
many genuinely concurrent same-turn-ID claims across separate connections
never produce the bug's error signature ("sessionstore: read turn:"),
distinct from unrelated SQLITE_BUSY/SQLITE_BUSY_SNAPSHOT lock contention,
which is retried rather than treated as a failure. The fix stands
regardless of provable live reachability -- the shadowing was unambiguous
and wrong on its own terms.

Adversarially reviewed by a fresh subagent, which independently reverted
just the fix (keeping the test) and confirmed empirically -- 20 runs, plus
1000 trials with a debug probe on the affected!=1 branch -- that the branch
is never reached under this driver, corroborating the test's own stated
limitation rather than contradicting it. gocognit/nestif findings on
ClaimTurn are pre-existing on the prior committed baseline, unchanged by
this fix (only variable names moved).
2026-08-22 02:03:27 +10:00
sam a0d569da09 fix: address simple findings from a repo-wide golangci-lint sweep
Six mechanical fixes from `golangci-lint run ./...` (30 findings total, 23
deliberately left -- see the next commit's message for the one that grew
into something bigger, and the following note for the rest):

- apply.go: %v -> %w for the wrapped yaml error, matching the identical
  double-%w pattern already used in decode.go/discovery.go/loader.go in the
  same package (errorlint).
- contract_test.go: filepath.Join("provider.go") with a single argument is a
  no-op join; replaced with the literal (gocritic).
- inprocess_test.go, yaegi_test.go: two goroutines whose whole purpose is
  timing (does Execute return promptly on cancellation / does Run block
  forever), where the call's result is irrelevant -- explicitly discarded
  with `_, _ =` and a comment saying why, rather than silently ignored
  (errcheck).
- find_test.go: an unchecked os.MkdirAll now matches the file's own
  createTestFile helper, which already checks and t.Fatals on the same
  operation (errcheck).

Not touched, and why: 18 cyclop/gocognit/gocyclo/funlen/nestif findings
require extracting helpers to reduce complexity -- the exact shape of two
regressions already recorded in CLAUDE.md's Lint-driven changes section
(one from --fix silently dropping a predicate, one from a nestif extraction
that doubled data), so a blind sweep across 9 unfamiliar files was declined
rather than risk a third. 2 godox findings (storage.go, worktree.go) mark
real missing work; deleting the comment would hide debt, not fix it. 3
recvcheck findings (Duration, TaskEnvelope, Provenance) each mix a
mandatory-pointer-receiver method (UnmarshalText/UnmarshalJSON's interface
contract, or genuine mutation) with correctly-value-receiver read-only
methods -- forcing consistency would either break interface conformance or
force pointer receivers onto call sites where the value isn't addressable.

Adversarially reviewed by a fresh subagent: all six confirmed
behavior-neutral or behavior-improving, all touched packages pass -race,
golangci-lint confirms exactly 23 issues remain (not more, not fewer), and
none are new.
2026-08-22 02:03:11 +10:00
sam 9dc009d267 chore(beads): record 7d5u.3 merge to main (9b62c89, PR #534) 2026-08-22 01:36:14 +10:00
sam 9b62c89773 Merge pull request #534 from samcharles93/feat/webhookguard-shared-mechanics
feat(webhookguard): event-source decisions + shared webhook mechanics
2026-08-22 01:34:48 +10:00
sam 7c1d34ac85 feat(webhookguard): shared HMAC, rate-limit, and redaction mechanics
archie-core-7d5u.3 -- storage-agnostic building blocks for vetting inbound
webhook-shaped requests, extracted rather than duplicated:

- VerifyHMAC: behavior-preserving extraction of internal/channels/webhook's
  private validateHMAC. internal/channels/webhook now calls it instead of
  keeping its own copy; all 10 of that package's existing tests still pass
  unmodified, and the one now-redundant unit test was removed with a
  comment pointing to where its coverage relocated.
- RateLimiter: per-source token bucket. Source IDs come from a registered,
  operator-controlled set, not unbounded attacker input, so the bucket map
  is not itself bounded -- documented explicitly rather than left implicit.
- RedactPayload: key-name-based redaction, matching gateway/stream.go's
  sensitiveParameterKey marker list. Deliberately does NOT reuse that
  function's actual behavior: it replaces every string value regardless of
  key, correct for a bounded chat summary and wrong here -- a captured
  payload exists to be inspected for field mapping (t2db.3), so only values
  under a sensitive-looking key are touched.

Narrowed from 7d5u.3's original scope, which bundled capture storage and an
approval state machine into this package. Both moved to the beads that own
that state (t2db.1's capture store depends on 7d5u.2's still-open NATS
storage decision; t2db.4's binding is the only consumer of approval state
today) rather than building shared abstractions ahead of a second real
consumer.

Adversarially reviewed by a fresh subagent with zero surviving findings:
token-bucket math hand-traced on two scenarios and race-clean under -race,
the redaction/gateway-stream.go behavioral divergence verified by reading
both functions side by side, both the redaction and rate-limit guards
mutation-tested by disabling each and confirming the suite catches it.

task check exit 0, golangci-lint 0 issues, gofumpt clean.
2026-08-22 01:28:58 +10:00
sam 75a9838d8b docs(prds): decide event sources and reaction delivery
archie-core-7d5u.1 -- closes with the decision recorded. Investigated three
mechanisms already in the tree rather than designing from scratch:

- No out-of-process extension boundary exists or is needed; in-process typed
  families throughout, and plugins-and-extensions.md does not need amending.
- Reaction delivery reuses internal/eventbus's Publisher/Consumer contract,
  but NOT the existing nats.Client unmodified: it hardcodes
  jetstream.WorkQueuePolicy (client.go:65), the competing-consumer policy
  correct for ARCHIE_TASKS and wrong for fan-out reactions -- two reactions on
  overlapping subjects would hit the exact silent-starvation trap CLAUDE.md
  already documents. Retention needs to become a Config field; reactions get
  a second stream.
- Reactions in these two epics are producer-only (curator's WakeOnPrimaryInput
  for cheap wake, or the existing pollNATS->publishTask shape for real work).
  No veto/mutate-in-flight capability is being built here.
- No single generic Source interface. workintake.TaskEnvelope already is
  forge intake's typed contract and needs no new type -- its IdempotencyKey is
  keyed on owner/repo/number, not delivery source, so a webhook-sourced
  envelope dedups against a poll-discovered one for free. What's genuinely
  shared is a small cross-cutting package for HMAC/rate-limit/capture/approval
  mechanics, owned by neither domain.

Retitled and rescoped archie-core-7d5u.3 to match: it was still asking for
'one registry for every kind of inbound caller,' which this decision
explicitly rejects. Removed a false blocking dependency of t2db.1 on 7d5u.4
(GitHub-specific receiver) -- they are siblings sharing the new cross-cutting
package, not a chain -- and linked them instead.
2026-08-22 01:01:52 +10:00
sam d046ad6859 docs(prds): decide webhook intake security ahead of the code it constrains
archie-core-t2db.5 -- decides the five non-negotiables its description named
before capture (t2db.1) or the intake family (7d5u.3) exist to build against
it: per-source HMAC auth reusing internal/channels/webhook's existing scheme,
a draft/pending_approval/armed binding state machine modeled on telegram's
dangerous-command approval flow, per-source rate limiting (precedent:
pairing.MaxPendingPerPlatform) that rejects rather than silently drops,
blast-radius bounding via the same gate/diff-cap/sandbox every task already
gets rather than a new relaxation, and 7-day retention with best-effort
redaction reusing gateway/stream.go's sensitiveParameterKey heuristic.

The bead stays open: its code-level acceptance criteria constrain beads that
haven't landed yet and cannot be satisfied by a document alone.
2026-08-22 00:54:00 +10:00
sam e8b053b95a chore(beads): correct NATS-optional mistake in event-sources epic
Two prior updates to archie-core-7d5u/.1/.2 characterized 'no external NATS
configured' as a permanent no-NATS fallback and framed reaction delivery as
a choice between NATS and a second in-process bus. Sam corrected both: the
intended second deployment shape was always NATS EMBEDDED, not NATS-optional.
Verified github.com/nats-io/nats-server/v2 is already a direct dependency and
already used to run a real embedded server in this codebase's own tests
(internal/infrastructure/agenttransport/nats/transport_test.go and others) --
it was simply never wired into production daemon startup. 7d5u.2 retitled and
rescoped accordingly.
2026-08-22 00:28:46 +10:00
sam 2c3bfb767e chore(beads): drop dead global-database references from metadata
global_dolt_database/global_project_id pointed at beads_global on the shared
Dolt server, which no longer backs this project after the move to embedded
mode. They were inert rather than harmful: `bd --global` is gated on
shared-server mode, so it fails identically with or without these keys, and
beads_global held zero issues. Removing them leaves metadata.json matching
tau, crew and nell-engine, which have always run embedded and never had them.

Restoring `bd --global` would mean running the shared server again, not
restoring these keys.
2026-08-22 00:07:20 +10:00
sam dfc33b99cd chore(beads): switch to embedded Dolt
dolt_mode in .beads/metadata.json is what selects the storage engine --
'embedded' runs in-process out of .beads/embeddeddolt with no server, 'server'
runs a project-local sql-server out of .beads/dolt. The dolt.shared-server key
in config.yaml only chooses between a shared and a project-local server; it
does not reach embedded mode, which is why the earlier switch landed on
server mode instead.

Data moved from .beads/dolt/archie_core to .beads/embeddeddolt/archie_core;
508 issues intact and the Dolt remote push verified from the new location.
Both paths were already gitignored.
2026-08-21 23:59:55 +10:00
sam 175b8560dd chore(beads): recover from v1.2.1 schema skew; move to a project-local Dolt
The accidental, untested beads v1.2.1 release migrated the schema from v53 to
v65 while the supported v1.2.2 binary expects v53, so bd refused to run. All
three databases on the shared server were affected, not just this one --
archie_core, ai_sdk and beads_global were each at v65.

Recovery per docs/RECOVERY-1.2.1.md (beads v1.2.2): rolled the schema cursor
back to v53 in each database and re-tracked the events table to restore
audit-event versioning. Issue data was not touched; 508 issues before and
after. Data the 1.2.1 release wrote into 1.2.x-only structures (work-lease
state, events-journal rows, provenance, storage_class markers) remains in the
database and is simply unused by the 1.1 line.

shared-server flips to false, so this project now runs its own Dolt server out
of .beads/dolt instead of sharing /home/sam/.beads/shared-server with ai_sdk
and beads_global. Note bd 1.2.2 has no "embeddeddolt" mode -- that layout
belongs to the older embedded builds; project-local server is the equivalent
today, and .beads/dolt is already gitignored.

Hooks were reinstalled: the installed set was still 1.2.1.

The rest of the config diff is bd reflowing the YAML indentation on write.
2026-08-21 23:54:42 +10:00
sam 1f581de428 chore(beads): add no-code playbooks epic; correct plugin-doc misquote
Epic archie-core-t2db: capture unrecognised inbound events, inspect real
payloads, map fields by example, bind matcher+mapping to an agentic workflow
at runtime. Security is P0 within the epic -- the intake surface is public and
its output is an agent with tool access.

Also corrects archie-core-7d5u and 7d5u.1, which misquoted
plugins-and-extensions.md as banning 'untyped hooks'. The rule is scoped to
Workflow semantics and does not block an event system.
2026-08-21 23:46:14 +10:00
sam f0bd5ccefd chore(beads): add event sources and typed reactions epic
Epic archie-core-7d5u with six children: the gating design decision, reaction
delivery, the typed intake family, a GitHub forge webhook receiver behind a
feature flag, idempotency/poll parity, and the docs reconciliation. Linked to
the existing WebhookAdapter beads so intake work is not duplicated.

Only the design bead is ready; the rest are gated behind it.
2026-08-21 23:27:31 +10:00
sam 4f838a6cd1 chore(beads): add adversarial self-review epic
Epic archie-core-h019 with seven children covering the design decision,
findings contract, reviewer isolation, workflow wiring, calibration, PR and
dashboard surfacing, and the docs promotion. Only the design bead is ready;
the rest are gated behind it per the scope-discipline rule in #533.
2026-08-21 23:03:53 +10:00
sam 879c0ff74d chore(beads): track the issues.jsonl export
ceee96b ("cleanup of noisey beads files", 2026-08-15) untracked
.beads/issues.jsonl on the rationale that JSONL exports are local
interchange artifacts. That reverses here: the export is now tracked so
issue state is readable and recoverable from a plain git clone, without
standing up a Dolt server.

What has NOT changed is which store is authoritative. Dolt still is, synced
via refs/dolt/data, with GitHub Issues as the external surface. The tracked
file is a mirror, not a source: it is written by bd's auto-export and must
never be hand-edited, because the next export overwrites it.

export.git-add flips to true in the same change, and the two settings only
make sense together. The exporter runs on a 60s interval and on the
pre-commit hook, so a tracked file that is not auto-staged would sit
permanently dirty in git status and block rebases and checkouts until
stashed -- the same friction .beads/interactions.jsonl already causes.
Staging it with the commit that regenerates it is what makes tracking
tolerable.

The cost accepted: most commits now carry issue churn, and parallel branches
can conflict on this file. Resolve such a conflict by taking either side and
re-running `bd export` -- the content is derived, so neither side is more
correct than the other.

Removing the ignore line is what tracks the file. Do not "fix" this with a
!issues.jsonl negation: .beads/.gitignore warns that negations override the
fork protection in .git/info/exclude.
2026-08-21 22:39:27 +10:00