Commit Graph
10 Commits
Author SHA1 Message Date
sam aba035a396 feat(webui): payload field mapping -- bind JSON paths to named fields, preview before saving
Adds t2db.3: schema-by-example field mapping for the no-code playbook epic.
An operator picks a captured event, clicks through its payload to bind named
fields to JSON paths, and previews the resolution against a real event
before saving -- a missing, null, or type-changed path fails loudly (named
by field and event) rather than handing an agent a blank value.

- internal/domain/mapping: Mapping/Field types, path parsing/walking,
  Resolve (per-field failure reporting), Validate (incl. duplicate names)
- internal/store/mappings.go: SQLite persistence, mirrors captures.go
- internal/webui/api_mapping.go: CRUD + preview HTTP API
- ui/src/mappings/: dashboard editor (clickable payload tree, live preview)

Design decisions recorded in docs/prds/payload-field-mapping.md. t2db.4
(binding a mapping + matcher to a workflow) is out of scope here.
2026-08-22 19:18:15 +10:00
sam 37bf8ef05a feat(capture): store unbound webhook events, retention-bounded
Adds POST /webhooks/capture/{source}: an unauthenticated endpoint (mounted
alongside /healthz, bypassing dashboard token auth by design) that persists
any inbound webhook archie has no binding for yet, so it can be inspected
and mapped later (t2db.2/t2db.3).

docs/prds/event-capture-storage.md decides SQLite over NATS JetStream for
storage: NATS is not unconditionally available until 7d5u.2 ships embedded
support, and capture's single-writer/single-reader access pattern doesn't
need JetStream's fan-out/replay machinery anyway.

internal/store/captures.go adds a captured_events table with prune-on-write
(retention age + row-count cap, both configurable). The HTTP handler applies
webhookguard redaction before persisting, and enforces two more independent
disk-bound mechanics before that: a per-request body-size cap and a
per-remote-address rate limit. New [capture] config section with documented
defaults; all fields optional, capture is on by default.

Adversarial review (manual, after the usual fresh-subagent pass hit a
session API limit mid-run) caught and fixed three real defects before this
landed, each with a regression test confirmed to fail against the pre-fix
code:
- the body-size cap silently no-op'd when CaptureMaxBodyBytes was left
  unset, the one bound that limits a single payload's size rather than row
  count or write rate
- the rate limiter was keyed by the request's own source path segment --
  unregistered and attacker-chosen by design for this endpoint, violating
  webhookguard.RateLimiter's documented bounded-key-space assumption and
  letting a sender bypass the limit by rotating the segment; rekeyed to
  remote address
- the retention prune compared time.RFC3339Nano strings, whose trimmed
  variable-width fractional seconds break lexicographic ordering within the
  same second; switched to fixed-width RFC3339

Closes archie-core-t2db.1.
2026-08-22 04:51:25 +10:00
sam 61b4b7b0cf refactor(store): replace legacy persistence with SQLite
Task and event persistence now uses a dedicated SQLite sidecar derived from the configured state prefix, matching the independently persisted conversation store. The configured path now means a state-path prefix rather than a database file so fresh SQLite state never attempts to interpret the previous storage format.

Remove the retired implementation and dependency completely, strengthen SQLite migrations and concurrency behavior, restore backend-neutral coverage, and align configuration, operational guidance, UI labels, and deployment documentation with the new storage and host-daemon topology.
2026-08-09 22:27:25 +10:00
sam 0d500a4ca9 feat(webui): lifecycle-safe task controls with atomic retry and archive
Add cancel/stop/approve/reject/retry/abandon/archive to the dashboard task
action matrix, each guarded by an optimistic status check against the
store. Retry accounting (RetryCount, max_retries) is now atomic with the
status transition instead of a separate write. Archive removes a terminal
task's local record with a fail-safe audit event first. Add always-labelled
responsive navigation, mobile Jump-to search, and versioned Setup-complete
dismissal.

Fixes #53
2026-08-09 01:25:51 +10:00
sam 308666b410 chore: remove legacy terminology and dead code
- Reword docs and comments to drop vague "legacy" qualifiers where they no longer describe deprecation status
- Remove unused code: RecentEvents, MessageEvent.ToLegacy, index setState, stderr redaction tool, telegramOnlyCommands
- Delete or comment out tests covering removed or superseded behavior
2026-08-06 22:55:26 +10:00
sam 170fe0250b fix: close the duplicate-work loop and the dead paths around it
Applies the four-reviewer pass over the forge/dashboard-actions work.
The through-line is that several capabilities were wired but inert, and
one of them could cost real work.

DUPLICATE WORK. "Closes #N" was dropped from PR bodies in favour of
LinkBranch, which is sidebar linkage on Gitea and a no-op stub on
GitHub -- neither closes anything, and nothing else did either. So a
merged task left its issue open, labelled and assigned; the next poll
re-enqueued it, and once the dashboard's Clear removed the task row that
became a second implementation and a second PR for work already merged.
reconcilePRs now closes the issue on merge, and a rejected task closes
its issue too. A PR closed without merging deliberately leaves the issue
open: that usually means "try again".

MAX RETRIES. Configured, defaulted and reported by /api/config, and
enforced nowhere -- markDead was unreachable and EffectiveMaxRetries had
no callers, so the dashboard's retry incremented retry_count forever.
The retry action now retires a task to dead at the cap.

DEAD ON ARRIVAL. Tasks() never selected plan or source, so the
"Decision required" panel could not render and chat tasks -- whose issue
number is synthetic -- linked to forge issues that do not exist.

COSMETIC FAILURE, FATAL RESULT. OpenPR failed the whole stage when
LinkBranch errored, so the pull request was never opened and the task
parked; a retry re-links the same branch, which Gitea answers with a
conflict, so every retry parked again. It is best-effort now.

INVISIBLE ACTIONS. Approve/retry/reject wrote store rows and nothing
else, so the timeline and activity stream never showed that a human had
intervened. They emit events now.

Also: store write failures answer 500 rather than 409; the dashboard
surfaces the server's actual reason instead of "409 Conflict"; action
buttons stop propagation (acting on a task no longer also expands it),
disable in flight, keep per-task errors instead of one shared string,
and confirm before rejecting.

Removed: the maybeRetryParked shim and markDead with their now-vacuous
tests, WaitingTasks (no consumers; its tests were rewritten to assert
the same routing through TaskByID), write-only Daemon.Runtime, and
dir/dirs response bloat from /api/skills. skillSource checked workDir
before sharedDir, mislabelling every shared skill when the two are equal.
DefaultRoots now dedupes roots differing only by a trailing separator.

The air build rule claiming to keep ui/dist current never ran -- it
required one file to satisfy include_dir and include_file together, and
ui/ is excluded from the walk. Deleted with the comment that promised it.
task test:ui joins task check and CI; it ran nowhere before, so the
dashboard suite could rot silently.

Docs corrected where they described behaviour that no longer exists:
parks no longer comment on the issue, state labels no longer drive
retries, and forgerpc's doc claimed four proxied methods while listing
three of five.
2026-08-05 03:54:25 +10:00
sam 712388dc6c fix: Phase 4 structural refactor — reduce lint findings from 28 to 5
cyclop (5→0): extract subcommand/case bodies in codesearch,
subprocess, Telegram, indexing
funlen (5→3): extract memory action handlers, tool provider startup
gocognit (7→2): extract config finalize/loadDirInto helpers, release
announcer, capture tool validation, Telegram gateway (200→108),
archie-agent main loop (35→0)
nestif (8→0): extract identity validation, container validation,
conf.d scanning, email multipart parsing, Telegram model/update
parsing, build-no-changes closure
interfacebloat (3→0): split Forge/IssueForge/PRForge/RepoForge,
SessionStore/SessionLifecycle/MessageHistory,
TaskStore/TaskLifecycle/TaskQueries/TaskEvents

Remaining: daemon process, workflow Feasibility/TDD/AgentStage,
archied main.go run (108→30 remaining). These require deeper
refactoring best done per-function in follow-up beads.

Updated contract tests and archied wiring tests to check both
main.go and the new telegram_setup.go.
2026-07-29 10:15:22 +10:00
sam f73b2d1c8c feat: memory manager sync/prefetch/scan/shutdown + broad in-flight epic work
Checkpoint commit covering concurrent work across several archie-core-abg
epics to avoid losing in-progress state:

- Memory manager (18.7-18.16): background sync pipeline, prefetch with
  timeout, memory-write notification, threat scanning, shutdown drain
  with abandoned-write tracking, frozen snapshot pattern. Wired
  ScanContent and NotifyMemoryToolWrite into HandleToolCall so scanning
  and notification actually apply to writes, not just exist as unused
  primitives.
- Fixed a literal unresolved merge-conflict that had been committed to
  internal/memory/scrubber/scrubber.go.
- Tool provider/registry, MCP client/transport (stdio, HTTP, SSE),
  multimodal results, disclosure.
- Pairing device auth store, rate limiting.
- Gateway compression, session store, tasks.
- Various workflow, plugin host, and agentexec updates.
2026-07-27 03:34:46 +10:00
sam f6d51416f0 chore: replace 607 em-dashes with double hyphens project-wide
Includes MCP crash-after-init test fix (crash moved from handle() to serve()
entry point so transport sees immediate subprocess death).
2026-07-26 09:53:33 +10:00
sam 4c1cc151a5 feat: NellDB storage backend replacing SQLite (archie-core-abg.30)
Extracted store.TaskStore interface (20 methods) covering task
lifecycle, queries, and observability. Separate WorkflowStore for
the agent RPC proxy.

New internal/nell/ package:
  - OpenStore(path, nodeID) — persistent logstore.LogStore backend
  - OpenMemory(nodeID) — in-memory store for tests
  - Full TaskStore implementation via sdk.DocDB with MVCC

All consumers updated to interface:
  - daemon.Daemon.Store, webui.Server.Store, storerpc.Server.Store
  - workflow.TaskContext.Store (WorkflowStore)
  - cmd/archied uses nell.OpenStore(cfg.DBPath, cfg.BotUser)

7 integration tests. Build + 32 packages pass.

Closes: archie-core-abg.30
2026-07-26 01:06:05 +10:00