Commit Graph
39 Commits
Author SHA1 Message Date
sam cc53e5258d fix(telegram): compact tool output and drop turn caps
Render completed ToolCallEvents as Telegram-safe streaming blocks
with compact name/status and a short cleaned preview. Summarize
file contents and listings instead of dumping source or quoted
JSON. Collapse repeated legacy turn-budget failures.

Remove max_tokens turn gating and the 200000 turn_budget_chars
default so absent or commented config cannot restore an aggregate
cap. Keep per-result max_result_chars and spill.

Confine find/grep discovery to the active workspace/worktree.

Refs #604
2026-08-23 16:40:16 +00:00
sam eaa436b09b docs+scripts: add archie-update-check, document missing config sections
The check-side of the reference update adapter never existed:
scripts/ had archie-update-install and archie-update-watchdog but no
archie-update-check, so any deployment that configured
chat.telegram.update_check_command pointing at the documented
"/usr/local/bin/archie-update-check" path had nothing there to run --
/update would report "not configured" regardless of what was actually
set. Add it: read-only, reports the daemon/agent components via git
describe against the local checkout and origin/main (no fetch-and-pull
the way install does), using the same "daemon"/"agent" component IDs
archie-update-watchdog's report and releaseupdate.ComponentDaemon/
ComponentAgent already use, so Service.Enrich and Report.Verify (this
session's earlier work) recognise it automatically.

Also fix archie-update-install's archie-agent build: it stamped
archied's ldflags but built archie-agent with none at all, so even a
native (non-container) archie-agent install reported an unstamped
"dev" build with installtype.Unknown.

config.example.toml gained every section that existed in the Go schema
but wasn't documented anywhere, not even commented: skills_dir,
plugin_dir, [chat.email], [chat].webhook_addr, [log], [memory] (noted
as currently unwired -- RegisterExternal is declared but never called
from cmd/archied, so the section has no effect yet), [[identities]]
(pointer to docs/architecture/identity.md rather than duplicating it),
and [[repos]].max_retries. tools.tool_policy.parallel_execution was
deliberately left undocumented: it's declared on ToolPolicy but never
read anywhere, genuinely dead rather than just underdocumented.
2026-08-24 00:32:37 +10:00
sam e04045907c feat(archied): wire generate_video end to end, gate it behind config
drainChatStream now detects a tools.MultimodalResult in a tool
result's Output and reports each URL via turn.Media(...), alongside
the existing turn.ToolCall(...) narration -- the missing link between
a tool producing media and a channel delivering it.

MinimaxConfig registers the tool only once an API key actually
resolves (registerMinimaxTool), matching web_fetch's "never advertise
a broken capability" pattern. Off by default, unlike web_fetch: this
spends real API credits per call.
2026-08-23 02:48:25 +10:00
sam 875b7a50d5 feat(forge): GitHub webhook receiver for immediate issue dispatch
Add [forge] intake = "poll" (default) | "webhook" | "both", and a forge
webhook receiver (internal/forge/webhook) that verifies a GitHub webhook's
HMAC signature, decodes the "issues" event, and publishes a matched issue
through the same path the poller uses -- so a labelled or assigned issue
starts a task the moment it happens instead of waiting up to poll_interval
(bead archie-core-7d5u.4).

The dispatch predicate (workintake.MatchesDispatch/RequiresLabel) is now a
single shared definition consumed by both the poller and the webhook
receiver, so the two cannot drift -- including the empty-label guard from
e19ff3a (an empty label must never match every open issue). Assignee
matching is case-insensitive to agree with AssignedIssues' server-side
GitHub filter; label matching stays case-sensitive, since GitHub labels
are case-distinct.

daemon.publishTask is exported as PublishTask, the single enqueue path
now shared by the poller and the webhook intake.

Two failure modes fixed during adversarial self-review before this landed:
a webhook secret that fails to resolve at runtime now degrades (logs and
skips the receiver) instead of aborting the whole daemon -- under
intake="both" the poll exists specifically as a backstop for this exact
misconfiguration, so crashing boot over it defeated its own purpose;
multi-identity deployments ([[identities]]) are explicitly unsupported for
now (webhook intake refuses to start, logs why, and leaves each identity's
own poll loop unaffected) rather than silently matching issues against the
wrong identity's dispatch rules.
2026-08-22 19:21:09 +10:00
sam 51bb80f6a1 feat(nats): run an embedded server when no external NATS is configured
Wire github.com/nats-io/nats-server/v2 into production startup so a
single-process deployment gets reaction delivery and task distribution
over an in-process server rather than a separate one
(docs/prds/embedded-nats.md).

Meaning change: [nats].url empty used to mean "no NATS, use the SQLite
ClaimNext flow"; it now means "start an embedded NATS server". The old
behaviour is the explicit opt-out [nats].mode = "off". mode resolves from
url when unset (url set -> external, url empty -> embedded), and
agent.mode="nats" / [containers] now require mode="external" (an embedded
loopback server cannot be reached by agent workers or containers).

The daemon owns the embedded server's lifecycle: StartEmbedded before
dialing the client, Shutdown after the client closes (LIFO cleanups). The
embedded server binds loopback on a random port, enables JetStream in a
store dir under the data directory, and leaves its logger nil so a startup
failure surfaces as a Connect error rather than an os.Exit.

Retention becomes a nats.Config field (default WorkQueuePolicy) so a future
reaction stream can fan out under LimitsPolicy instead of the hardcoded
competing-consumer policy that starves a second consumer on overlapping
subjects.
2026-08-22 05:18:31 +10:00
sam 101c133239 feat(capture): store unbound webhook events, retention-bounded
Adds POST /webhooks/capture/{source}: an unauthenticated endpoint (mounted
alongside /healthz, bypassing dashboard token auth by design) that persists
any inbound webhook archie has no binding for yet, so it can be inspected
and mapped later (t2db.2/t2db.3).

docs/prds/event-capture-storage.md decides SQLite over NATS JetStream for
storage: NATS is not unconditionally available until 7d5u.2 ships embedded
support, and capture's single-writer/single-reader access pattern doesn't
need JetStream's fan-out/replay machinery anyway.

internal/store/captures.go adds a captured_events table with prune-on-write
(retention age + row-count cap, both configurable). The HTTP handler applies
webhookguard redaction before persisting, and enforces two more independent
disk-bound mechanics before that: a per-request body-size cap and a
per-remote-address rate limit. New [capture] config section with documented
defaults; all fields optional, capture is on by default.

Adversarial review (manual, after the usual fresh-subagent pass hit a
session API limit mid-run) caught and fixed three real defects before this
landed, each with a regression test confirmed to fail against the pre-fix
code:
- the body-size cap silently no-op'd when CaptureMaxBodyBytes was left
  unset, the one bound that limits a single payload's size rather than row
  count or write rate
- the rate limiter was keyed by the request's own source path segment --
  unregistered and attacker-chosen by design for this endpoint, violating
  webhookguard.RateLimiter's documented bounded-key-space assumption and
  letting a sender bypass the limit by rotating the segment; rekeyed to
  remote address
- the retention prune compared time.RFC3339Nano strings, whose trimmed
  variable-width fractional seconds break lexicographic ordering within the
  same second; switched to fixed-width RFC3339

Closes archie-core-t2db.1.
2026-08-22 04:51:25 +10:00
sam c32319968c fix(mcp): persist npx package cache across daemon restarts
config.docker.toml launches MCP servers with `npx -y <pkg>@<version>`,
and the container has no persistent npm cache, so every daemon restart
re-resolved and re-downloaded the package from the registry -- slow,
and it turns a registry outage or a yanked version into a daemon
outage. It directly amplified the crash loop observed on 2026-07-30,
since each restart repeated the whole download.

configuredMCPProvider now points npx-launched servers (matched by base
name, so an absolute-path npx still qualifies) at NPM_CONFIG_CACHE
under the daemon's own persistent work_dir, plus
NPM_CONFIG_PREFER_OFFLINE=true so a warm cache skips the registry
round trip entirely. Does not cover a package's own separate
downloads (e.g. Playwright/Puppeteer fetching a browser binary via
its postinstall script).

Closes archie-core-1786637490923-43-d4bb914b.

The identical, arguably worse instance in
internal/app/agentworker/mcp_providers.go (every per-task ephemeral
container, not just daemon restarts) needs a persistent Docker volume
that doesn't exist yet, so it's filed separately as
archie-core-qk8m rather than folded into this scoped fix.
2026-08-19 02:08:33 +10:00
sam a5a878c2d0 feat(secret): add sops, vault/openbao and age engine plugins
CLI-based, env-configured, cache-less engines resolving SecretRef values
through the existing Yaegi plugin surface. Fresh resolve per call, so
values updated in the backend are picked up without daemon restart —
closing the gap that motivated the remote secret-input epic (#504),
whose value-entry web form duplicated the Bitwarden app for bws.

- examples/secret-engines/: sops (SOPS_FILE), vault (VAULT_ADDR/TOKEN,
  key "path/field" or VAULT_SECRET_PATH prefix), age (AGE_FILE/
  AGE_IDENTITY, key=value lines). //go:build ignore keeps them out of
  the Go toolchain; yaegi treats the tag as a comment and loads them.
- enginehost: host-side Run helper registered as a Yaegi symbol. yaegi's
  interpreted os/exec cannot pass env to child processes, so plugins
  shell out through host code; bounded by a 30s timeout like the bws
  engine.
- secret.LoadDir seeds the interpreter with the host environment: yaegi's
  interpreted os package keeps a private env otherwise, and plugin
  os.Getenv would silently see nothing.
- cmd/archied wires enginehost + unrestricted symbols into the engine
  plugin loader, matching skillscript's interpretation posture.
- tests load the real shipped plugin files with fake CLIs on PATH;
  added CONTEXT.md glossary for the remote secret-input vocabulary.
2026-08-18 00:32:57 +10:00
sam ddfc8a7b91 fix(config,webui): make show_tool_calls one cross-channel setting
show_tool_calls gated Telegram's tool-activity narration only; the
web dashboard's SSE chat stream narrated every completed tool call
unconditionally with no way to turn it off, leaking tool names and
parameter shapes to anyone with dashboard access even when an
operator had already opted out for Telegram.

Moved ShowToolCalls from TelegramConfig to ChatConfig so it is one
setting for every channel, and wired the webui stream to it:
chatStreamSink now carries a showToolCalls flag (snapshotted once per
turn, mirroring how Telegram's liveReply snapshots it) and ToolCall
no-ops when it's off.

BREAKING: an existing [chat.telegram] show_tool_calls = true in
config.toml now parses as an unrecognized key under [chat.telegram]
and is silently ignored (TOML decoding is non-strict) rather than
erroring -- move it to the top-level [chat] table.
2026-08-15 01:57:06 +10:00
sam 0d73f2fa5a feat(chat): show tool calls inline and stream replies into one message
Chat returned only the final answer, so a wrong tool choice, a
hallucinated action, or a footgun like `systemctl stop` against
`restart` was invisible from the conversation. Only text deltas were
threaded through the turn, and tool calls executed inside Generate with
nowhere to report.

Replace the delta callback with gateway.TurnStream, which carries text
fragments and completed tool calls on one ordered path from the model
loop to the channel. The sequence is the point: a tool line that
overtakes the sentence it interrupted describes a turn that never
happened. Text-only callers use gateway.DeltaFunc and are unaffected.

A tool is reported on its result, never on its call  --  the call part
carries no outcome yet, so reporting both would list every tool twice,
once with nothing to say.

Telegram renders it behind [chat.telegram] show_tool_calls, off by
default: this is debugging visibility, and it narrates every internal
step to provide it. The dashboard always shows it, being the operator's
own observability surface; its tool lines are live-only and are not
persisted with the conversation.

The Telegram live reply changes mechanism, and with it what a frame
MEANS. It used to stream through sendMessageDraft: drafts are ephemeral
30-second previews, so a tool call longer than that outlived the
preview, and successive frames were observed concatenated with their
cursors intact. It now sends one real message and edits it. An update
used to be "the next preview of the answer"; it is now "the whole
answer as it currently stands, replacing what is there". The cursor is
appended at render time and never stored, so exactly one exists and
always at the end, and a frame too long for one message is cut to its
tail rather than rejected outright.

A stopped or failed turn now edits the cursor away instead of leaving
it blinking on a message nothing will finish. That cleanup runs on a
live context because turnCtx is already dead, and it runs on both
failure branches: ai-sdk raises an aborted stream as ErrAborted with
ctx.Err() formatted in rather than wrapped, so errors.Is cannot be
trusted to tell a /stop from a fault.

Closes #467
2026-08-14 02:36:12 +10:00
sam 61b4b7b0cf refactor(store): replace legacy persistence with SQLite
Task and event persistence now uses a dedicated SQLite sidecar derived from the configured state prefix, matching the independently persisted conversation store. The configured path now means a state-path prefix rather than a database file so fresh SQLite state never attempts to interpret the previous storage format.

Remove the retired implementation and dependency completely, strengthen SQLite migrations and concurrency behavior, restore backend-neutral coverage, and align configuration, operational guidance, UI labels, and deployment documentation with the new storage and host-daemon topology.
2026-08-09 22:27:25 +10:00
sam e10cbbc92c refactor: pre-cleanup. 2026-08-06 01:40:15 +10:00
sam 0ddc43081e feat(setup): implement the interactive setup steps (archie-core-cbk)
internal/infrastructure/configuration/setup.Run drives operator
identity, LLM provider, models, forge, and Telegram chat channel
prompts and returns TOML edits -- not a config.Config. Building a
parallel in-memory representation to validate against was rejected
during design review: it's a second source of truth about the exact
thing (text vs. struct disagreeing) this whole feature exists to stop.
Tests instead render the edits against the real embedded template and
load the result through a real configuration.Loader, proving the
actual defaults/decode/validate path accepts it.

Two boot-time landmines in config.example.toml surfaced by that
approach, both fixed here:

- [providers.openai] ships active with api_key pointed at the bws
  engine. cmd/archied/provider_secrets.go resolves every entry in
  cfg.Providers at startup, not just the ones [models] references, and
  refuses to start the daemon if any fails -- so picking any other
  provider (or picking openai but skipping the key) left a config that
  wouldn't boot on a machine without the bws CLI installed (the common
  case). stepProvider now neutralises providers.openai.api_key to the
  zero-value secret ref whenever openai wasn't the actual choice.
- [[repos]] shipped active with a placeholder owner/name that
  validates cleanly (owner/name are only checked non-empty) and would
  have archied silently polling a repo that doesn't exist. Commented
  out in the template with an explanation; setup does not write this
  section since tomlwrite cannot target array-of-tables.
  TestExampleConfigLoads updated to expect zero repos.

Also: forge and provider secret env-var names are each defined in one
table (forgeToken, cloudProviders) read by both the config edit and
the secret sink, so the two can't name a secret differently -- the
exact bug class this feature exists to close for the installer, tested
directly (TestRun_ForgeTokenKeyMatchesStoredSecretKey compares the
decoded key against the key the secret was actually stored under, not
a hardcoded literal on both sides). Telegram writes chat.telegram.
token_env, not a secret.SecretRef -- TelegramConfig has no such field.
Telegram also requires at least one allowed_user_id: AllowedUserIDs is
deny-by-default, so a bot token with none configured answers nobody,
which reads as broken rather than as the safe default it is.

SecretSink is a caller-supplied sink (Put buffers, Commit persists),
not a return value: Run never hands back plaintext secrets, and the
caller is expected to call Commit only after proving the generated
config loads -- so a validation failure never leaves a secret written
without the config that references it.

Refs archie-core-cbk
2026-08-05 12:58:53 +10:00
sam 170fe0250b fix: close the duplicate-work loop and the dead paths around it
Applies the four-reviewer pass over the forge/dashboard-actions work.
The through-line is that several capabilities were wired but inert, and
one of them could cost real work.

DUPLICATE WORK. "Closes #N" was dropped from PR bodies in favour of
LinkBranch, which is sidebar linkage on Gitea and a no-op stub on
GitHub -- neither closes anything, and nothing else did either. So a
merged task left its issue open, labelled and assigned; the next poll
re-enqueued it, and once the dashboard's Clear removed the task row that
became a second implementation and a second PR for work already merged.
reconcilePRs now closes the issue on merge, and a rejected task closes
its issue too. A PR closed without merging deliberately leaves the issue
open: that usually means "try again".

MAX RETRIES. Configured, defaulted and reported by /api/config, and
enforced nowhere -- markDead was unreachable and EffectiveMaxRetries had
no callers, so the dashboard's retry incremented retry_count forever.
The retry action now retires a task to dead at the cap.

DEAD ON ARRIVAL. Tasks() never selected plan or source, so the
"Decision required" panel could not render and chat tasks -- whose issue
number is synthetic -- linked to forge issues that do not exist.

COSMETIC FAILURE, FATAL RESULT. OpenPR failed the whole stage when
LinkBranch errored, so the pull request was never opened and the task
parked; a retry re-links the same branch, which Gitea answers with a
conflict, so every retry parked again. It is best-effort now.

INVISIBLE ACTIONS. Approve/retry/reject wrote store rows and nothing
else, so the timeline and activity stream never showed that a human had
intervened. They emit events now.

Also: store write failures answer 500 rather than 409; the dashboard
surfaces the server's actual reason instead of "409 Conflict"; action
buttons stop propagation (acting on a task no longer also expands it),
disable in flight, keep per-task errors instead of one shared string,
and confirm before rejecting.

Removed: the maybeRetryParked shim and markDead with their now-vacuous
tests, WaitingTasks (no consumers; its tests were rewritten to assert
the same routing through TaskByID), write-only Daemon.Runtime, and
dir/dirs response bloat from /api/skills. skillSource checked workDir
before sharedDir, mislabelling every shared skill when the two are equal.
DefaultRoots now dedupes roots differing only by a trailing separator.

The air build rule claiming to keep ui/dist current never ran -- it
required one file to satisfy include_dir and include_file together, and
ui/ is excluded from the walk. Deleted with the comment that promised it.
task test:ui joins task check and CI; it ran nowhere before, so the
dashboard suite could rot silently.

Docs corrected where they described behaviour that no longer exists:
parks no longer comment on the issue, state labels no longer drive
retries, and forgerpc's doc claimed four proxied methods while listing
three of five.
2026-08-05 03:54:25 +10:00
sam 3ae59c99c0 fix(secrets): wire provider credentials through engines 2026-07-31 08:45:58 +10:00
sam 6c32550345 chore: replace personal owner names in tests and config with generic values
Test fixtures used 'sam', 'samcharles93', and 'sam/<repo>' as owner
names. These are personal and machine-specific — the repo should use
generic placeholders like 'acme' (matching the existing convention in
worktree_test.go).

config.example.toml: samcharles93 -> your-github-username
Test owner: sam -> acme
Repo refs: sam/archie-core -> acme/archie-core, etc.
Example skills: sam -> acme
2026-07-31 02:15:41 +10:00
sam 6be754787d polish: add install.sh, XDG setup, deployments/ examples, and github-token docs\n\n- Add install.sh following XDG Base Directory specification (~/.config/archie, ~/.local/share/archie, ~/.local/bin)\n- Add interactive setup for code forge (GitHub / Gitea / Standalone) and LLM provider selection (Ollama model picker / Cloud keys)\n- Implement NoopForge in internal/forge/noop.go for running Archie in standalone mode without a forge\n- Enable loginctl user linger and systemd user unit setup in install.sh\n- Add docs/github-token.md with step-by-step guidance for Classic PAT generation\n- Add top-level deployments/ directory with scenario templates (single-forge GitHub, multi-forge GitHub+Gitea, local Ollama standalone, systemd runbook) 2026-07-30 01:25:46 +10:00
sam 561e56b70a feat: make /stop cancel the running chat turn
/stop could not reach a running turn, and the reason was worse than a
missing cancel. go-telegram/bot runs one update worker that calls
handlers synchronously (defaultWorkers = 1, never overridden), and the
chat turn ran on it. While a turn streamed -- or sat in a shell tool --
the worker was blocked and /stop sat unread in the update channel until
the turn it was meant to kill had already finished.

Add gateway.Turns: a per-session serial lane that turns are submitted to
and that the event loop never waits on. The worker is free again in
microseconds, so /stop is serviced immediately; ordering comes from the
lane being serial rather than from the loop being blocked.

- Bare /stop cancels the running turn and clears the backlog behind it,
  with no approval step. Approval guards destructive commands, and
  gating the brake behind a confirmation defeats it. /stop <name> keeps
  the existing process-termination behaviour and its approval.
- Cancellation propagates through the turn context, so it reaches the
  model stream and any tool underneath -- a shell command built with
  exec.CommandContext dies with the turn.
- Queues are unbounded and never shed. A dropped message is a silent
  failure that surfaces later as an agent that ignored something.
- Turns are recovered per-lane: the channel's panic middleware wraps the
  event-loop goroutine, which no longer runs the turn.

Also raise the chat step budget. It was hardcoded at 8 -- roughly one
read, one grep and two edits -- so turns hit the cap and stopped
mid-task with no indication they had been cut off. It is now
chat.max_steps, defaulting to 100. Interruption is /stop's job.
2026-07-29 16:41:55 +10:00
sam e03cef17c9 feat(tools): lift tau's file and shell tools into the tool registry
The chat agent had two tools -- memory_edit and skill_activate -- so it
could not read a file, search a tree or run a command. ai-sdk's runtime
already drives a tool loop for it (MaxSteps 8, toolExecute wired as the
executor), so the loop was never the missing piece: the registry was empty
of anything useful.

Lifted from tau internal/agent/tools at commit f5289ea, per the recorded
copy-and-mutate strategy: every file carries a provenance header naming its
tau source path, the commit, and the mutations made, so refreshing from
upstream is a diff rather than archaeology.

  internal/tools/builtin/          the lifted tools, ~3.1k lines
  internal/tools/provider/builtin/ the adapter to tools.ToolEntry

Six tools: read, write, edit, shell, grep, find. Tau's own tests came with
them and pass unmodified, which is the evidence that the lift is faithful.

Deliberately not lifted:
  - agent.go (1214 lines): subagent spawning, and the only real coupling in
    the package -- 10 tau packages including chat, sessions, store and
    registry. Nothing the chat agent needs to do work.
  - skill.go: depends on tau's skills package; archie has internal/skill and
    already exposes skill_activate.
  - docs.go: serves tau's own embedded documentation via tau/docs.

Not lifting the coordinator either. It is tau's agent loop (6.4k lines over
ten packages) and archie already has one in the ai-sdk runtime; porting it
would leave two loops competing. Its cancellation and tool events are worth
having, but they are a few hundred lines built against the existing path
rather than six thousand lifted onto a second one.

The adapter bridges two shape differences: archie hands handlers decoded
arguments where tau's executor takes raw JSON, and tau reports tool-level
failure in Result.IsError rather than as an error -- surfaced as a real
error so the runtime and the guardrail engine both see it. Tools are
classified idempotent or mutating for guardrails and budgets; that is
categorisation, not a jail.

chat.workspace roots the tools and defaults to empty, which registers none
of them. They read, write and execute, so the directory has to be a
deliberate choice rather than whatever the daemon started in -- in the
container that is /.

Verified: go build, go vet, gofmt clean; full suite passes.
2026-07-29 15:46:21 +10:00
sam bf5a7618e6 fix(chat): operator name from config, session wiring, prompt rules
Three defects found from a real Telegram transcript.

1. The operator's name was compiled into the prompt.

internal/gateway/persona.go and prompt.go both hardcoded "Sam's" into the
default persona and the fallback prompt. The persona renders first into
every system prompt, so a personal name was sent to the model provider on
every turn, and any other deployment got an assistant that believed it
worked for someone else. There was nowhere to put it: Repo.Owner is a repo
owner and config.Config had no operator concept.

ChatConfig.Operator now carries it, rendered in the prompt's <env> block as
runtime metadata and omitted entirely when unset. config.example.toml
documents that setting it sends the name to the model provider.

2. /new claimed to clear history that it had not cleared.

The chat responder keyed history by channel id through a local sessionKey
helper, while the session commands operate on tracked session ids.
sessionTracker.resolve mints deterministic ids so the two agree -- until
/new, /branch or /resume assigns a generated id, after which they address
different histories permanently. /new reported "Conversation history has
been cleared" while the next turn still saw everything, and /undo,
/compress, /resume and /sessions acted on a namespace the conversation
never read.

gateway.Router.ResolveSessionKey already existed for this, and
session_commands.go:104 said outright that main.go should call it instead of
its inline helper. It never got wired. The responder now resolves through
the router and the dead helper is deleted so it cannot come back.

buildTelegramRouter constructs the router before the responder so the
dependency is visible rather than captured by reference.

3. Prompt rules for two behaviours the transcript showed.

The agent offered to accept shell access it has no tool to use, which is
both unhonourable and a request for the user's secrets. It also opened
mid-conversation replies with a greeting, and described its own manner
("no fluff") in stock phrasing. Added core rule 7 and three communication
rules covering greetings, self-description and register.

Note the memory claim in that transcript was NOT a fabrication: the
memory_edit tool's own description names MEMORY.md and USER.md, and tool
descriptions are part of the prompt.

Also added chat-turn logging (session id, channel, thread, history depth).
Chat turns were entirely unlogged, so a report of lost context had no
evidence to check: 76 log lines covered two days and none of them was a
conversation. A separate report of mid-conversation amnesia remains
undiagnosed -- single process, 43h uptime, compression untriggered at three
messages -- and this is what makes it diagnosable next time.

Regression tests: prompt renders/omits the operator and carries no personal
name; the responder resolves via ResolveSessionKey and defines no local
session key helper.

Verified: go build, go vet, gofmt clean; full suite passes.
2026-07-29 15:17:23 +10:00
sam 05cfc30da3 fix(indexing): surface index config properly and stop silent degradation
Follow-up to c924281, which introduced two defects of its own.

1. indexing.Config was a private config island: index paths existed nowhere
   in internal/config or config.example.toml, so they were undiscoverable
   and unsettable by anyone reading the real configuration.

   Now [indexing] in config.toml, defined in its own internal/config/
   indexer.go rather than appended to the 21-struct config.go, and its
   defaulting lives with it as IndexingConfig.withDefaults so finalize
   delegates instead of growing. Paths default off work_dir, keeping the
   two archied instances on one host from sharing an index. Documented in
   config.example.toml.

   internal/indexing keeps its own plain Config and does NOT import
   internal/config, so it stays a straight diff against its tau original.

2. A failed index build was invisible. refreshAsync wrote the reason to a
   database column with no reader and degraded to unindexed search in
   silence — the failure mode that costs hours precisely because nothing
   appears to be wrong. Every give-up path now logs, and a successful
   build logs once.

The logger is resolved at the point of use, not in the constructor: a
Manager can be built as a struct literal (the ported tests do), and
constructor-only defaulting panicked on the build goroutine, where a panic
takes the daemon with it. Regression test covers the zero and nil Manager.
2026-07-28 01:37:19 +10:00
sam bcca6f9c25 feat(telegram): add approved component updates 2026-07-27 13:02:11 +10:00
sam ec8534dfe2 feat(gateway): ship model controls, shell tools, and release tracking 2026-07-27 06:16:39 +10:00
sam 2b1045c0ad feat: secrets-engine registry with yaegi extension interface (archie-core-xec)
Adds internal/secret: a pluggable Engine interface (Name/Version/Resolve)
resolved through a Registry, so secrets backends (sops, Bitwarden Secrets
Manager, Vault/OpenBao) aren't hard-coded into the binary. Only the "env"
engine is compiled in; custom engines are Yaegi-interpreted .go plugins
loaded via Registry.LoadDir, mirroring internal/plugin's conventions
(secretextract provides the generated, nil-guarded symbol table for the
Yaegi/Go interface boundary).

config.Forge.TokenEnv is replaced end-to-end with config.Forge.Token
(secret.SecretRef{Engine, Key}) -- finalize() defaulting, forge token
resolution in cmd/archied/main.go, and per-identity validation that now
requires an explicit token instead of silently resolving to "". No
backwards-compat shim.

Provider.APIKeyEnv, NATSConfig.TokenEnv, and TelegramConfig.TokenEnv are
deliberately not migrated here -- they're forwarded by name to
ai-sdk/runtime and subprocess workers rather than resolved centrally,
which needs its own eager-resolve-then-inject design pass
(archie-core-xec.4). The three shipped engine plugins (sops, bws,
vault/openbao) are tracked separately as archie-core-xec.1-3.

Adversarially reviewed by a fresh agent with no implementation context;
fixed the two findings that survived (missing per-identity token
validation, underdocumented nil-wrapper Resolve edge case).
2026-07-26 14:49:32 +10:00
sam f6d51416f0 chore: replace 607 em-dashes with double hyphens project-wide
Includes MCP crash-after-init test fix (crash moved from handle() to serve()
entry point so transport sees immediate subprocess death).
2026-07-26 09:53:33 +10:00
sam e4368f70af feat: persistent volumes with TTL for container/worktree storage (archie-core-j7v)
Add [containers].volume_ttl (default 72h). Persistent-storage repos get a
per-repo bare Git cache (--reference-if-able --dissociate) so task
worktrees clone fast without repeated full network clones, while staying
fully independent of the cache's lifecycle — cache expiry can never
corrupt an active or parked task's worktree. Docker repo volumes are
labeled as Archie-owned and swept non-forced (a volume still attached to a
running container survives and retries next cycle); the repo cache
directory is swept by mtime. Both cleanups run at daemon startup and each
poll cycle.

Filed archie-core-756 (repo-wide golangci-lint baseline) and archie-core-340
(intermittent JetStream TempDir teardown flake) as separate follow-ups
rather than scope-creeping them into this change.
2026-07-24 10:22:11 +10:00
sam 4be28e0e48 feat: optional NATS token authentication (archie-core-l1q)
Add token_env to [nats] config, resolved and validated at daemon startup
(fails early if configured but the named env var is unset). Container
workers get NATS_URL/NATS_TOKEN; archie-agent honors both while CLI-supplied
URL still takes precedence. Anonymous auth remains the default for
single-node deployments.

Unblocks archie-core-5zh (promote archie-agent to per-task process).
2026-07-24 07:51:06 +10:00
sam b6b23bbb59 feat: per-repo allow_concurrent override for same-repo dispatch (archie-core-g1l)
Add config.Repo.AllowConcurrent (allow_concurrent toml key). Repos opted in
skip the daemon's default FIFO same-repo task serialization while still
respecting [containers].max_concurrency globally. Worktrees are already keyed
per issue number, so opted-in repos don't collide on disk.
2026-07-24 07:47:10 +10:00
sam f8df051b19 feat: bounded concurrent task dispatch in daemon (archie-core-u64)
Daemon previously processed one task at a time. Add bounded concurrent
dispatch for SQLite and NATS agent modes, reusing [containers].max_concurrency
(0 = unlimited). Same-repository tasks stay FIFO-serialized to avoid worktree
conflicts; different repositories now run concurrently.

Unblocks archie-core-l1q (NATS transport) and archie-core-g1l
(per-repo allow_concurrent override).
2026-07-24 07:42:13 +10:00
archie d2b2f29096 Implement max_retries cap on maybeRetryParked to prevent infinite parked-task loops (archie)
Implements #61
2026-07-23 18:49:38 +10:00
sam de57f72cce feat: add Docker container sandbox for archie-agent
Dockerfile: multi-stage fat image (golang:1.26 builder → ubuntu:24.04)
with Go 1.26, Python 3.12/3.13 via deadsnakes, UV, Node 22, pnpm,
TypeScript, esbuild, git, gh, golangci-lint, gofumpt.

Container pool (internal/container): manages Docker container lifecycle
via moby/moby client. Per-task containers with bind-mounted workspaces.

- Acquire: create, start, mount workspace, inject NATS + API key env
- Release: stop, remove (AutoRemove as safety net)
- Orphan recovery on daemon startup (label: archie-daemon=true)
- MaxConcurrency enforcement, MaxUptime config (unused in MVP)
- Image pull with "missing"/"always" policy

Config: [containers] block (enabled, image, max_concurrency, max_uptime,
pull_policy). Validation: requires agent.mode = "nats" and [nats] url.

Daemon: acquires container before workflow.Run, releases after.
Container environment auto-populated from provider API keys.
2026-07-22 02:39:17 +10:00
sam 8c4bb1ef1d feat: promote archie-agent to NATS-connected per-task worker
Replaces stdin/stdout subprocess transport with NATS JetStream
request-reply pattern. The Runner interface and workflow orchestration
are unchanged.

New NATSRunner (implements Runner):
- Publishes AgentRequestMessage to JetStream with X-Archie-Reply header
- Blocks on core NATS inbox subscription for response
- Same Run(ctx, workspace, req) (Result, error) contract

Rewritten archie-agent (cmd/archie-agent):
- Long-running NATS-connected process (was 28-line stdin/stdout)
- Pull-consumes from ARCHIE_TASKS with archie.agent.> filter
- Builds InProcessRunner, executes agent loop, replies to inbox
- Graceful shutdown via signal handling

Extended internal/nats:
- SubjectAgentWildcard, SubjectForAgentRequest, ReplyHeader
- Conn() accessor, NewReplyInbox(), PublishMsg() for JetStream
- Stream subjects extended to ["archie.task.>", "archie.agent.>"]

Config: "nats" added as valid agent.mode alongside inprocess/subprocess
Tests: NATSRunner round-trip with embedded NATS server + handler
2026-07-22 02:13:01 +10:00
sam bb2193e5e4 feat: add optional NATS JetStream task distribution
Adds optional NATS-based task distribution alongside the existing
SQLite flow. When [nats] is absent from config, zero behaviour change.

New internal/nats package:
- Subject constants matching PRD §2 (archie.task.{bug,feature,default,bootstrap})
- Client: Connect (stream + durable pull consumer), PublishTask
  (Nats-Msg-Id dedup), Fetch, DecodeTask
- Tests with embedded NATS server (connect/publish, dedup, subject routing)

Config: NATSConfig{URL, TokenEnv} on Config, commented-out [nats] block
in config.example.toml

Store: ClaimByIssue atomic claim by owner/repo/number for NATS consumer

Daemon: Nats client field, poll() branches to pollNATS/pollSQLite,
Cycle() branches to drainNATS/drainSQLite (NATS fetch then SQLite
ClaimNext fallback for requeued tasks), processNATSTask
(decode → EnqueueIssue → ClaimByIssue → process → Ack)

Scratch: explorer and plan agent outputs from parallel-explore phase
2026-07-22 02:02:09 +10:00
sam 86339b4d0d feat: add subprocess agent runner 2026-07-21 23:21:24 +10:00
sam 251cc7022c feat: add configurable forge and workflow behavior 2026-07-21 15:03:41 +10:00
sam 748d9aa9ad feat: per-repo protected suffixes, requeue flag, templ-aware gate config
The first feature run parked exactly as designed but for an avoidable
reason: todo is a templ project and the agent edited generated
_templ.go files, leaving literal template placeholders in rendered
HTML. Systemic fixes, all environmental:

- repos gain protect = ["_templ.go"]: suffixes agents can never write
  (agentloop.ProtectPaths), forcing edits to the sources; the gate's
  new leading ["templ","generate","./components/"] command
  regenerates. AgentStage.ProtectPaths now composes with the repo list
  and takes TaskContext like every other stage hook.
- archied -requeue <task-id>: the retry path for parked/waiting tasks
  (keeps the task's workflow; Requeue with empty workflow preserves it,
  clears the park reason).

Retry of the parked feature task drove the full loop home: PRD-guided
plan -> builder edited components.templ (generated file write-blocked),
gate regenerated + tested -> PR #8 with 109 lines of tests.
2026-07-21 04:34:53 +10:00
sam 239b8eda42 feat: feasibility workflow — assess, PRD, human go/no-go loop
Routed via the feature label. Stages: assess (read-only agent with a
structured decide tool — verdict as a tool call, not parsed prose;
not-a-fit closes the issue as won't-do with reasons) -> PRD (read-only
agent; problem, solution, surface, acceptance criteria, non-goals) ->
deliver (PRD posted as the watched issue comment, optional [notify]
webhook POST for n8n->email, task -> waiting_human).

The daemon's cycle now watches waiting_human tasks: replies after the
watch comment (excluding the bot) are judged by an LLM (triage role,
APPROVE/REJECT/UNCLEAR + reason — never keyword matching). Approvals
requeue under implement (whose planner now treats an existing PRD as
prior context); rejections close with the reasoning acknowledged;
unclear replies keep waiting.

Supporting changes: forge.Comment returns the comment id;
forge.RepliesAfter; store watch_comment_id (additive migration),
WaitingTasks, Requeue; Route honours a pre-assigned workflow;
AgentStage.Extra for stage-specific tools.

Live: todo#7 assessed fit -> PRD delivered -> waiting_human.
2026-07-21 04:27:17 +10:00
sam bb9c6f89a2 docs: document [web] dashboard section in example config 2026-07-21 04:21:02 +10:00
sam c2c8e92304 feat: archied walking skeleton — poll, store, worktree, workflow engine, bootstrap PR
Repurposes archie-core as the archie orchestrator daemon (the old
Pocketbase PM-backend plan is archived under .references/).

- cmd/archied: -config, -once (single cycle for timers/testing)
- internal/config: TOML; per-repo gate command lists; role->model refs
- internal/store: SQLite (modernc, WAL) task state machine with
  transitions audit, idempotent enqueue, crash recovery (running ->
  queued, attempt++)
- internal/forge: go-github v78 — labelled-issue polling, invitation
  auto-accept, push verification, PR create/state, comments, close
- internal/worktree: fresh clone per task, bot identity, askpass token
  handling, diff-size stats, cleanup on merge/reject
- internal/workflow: extensible engine (stages over TaskContext, shared
  step library, label routing bug->tdd feature->feasibility default->
  implement); parks always comment on the issue. Bootstrap workflow
  (deterministic marker-file PR) proves the plumbing without LLM spend.

Store and worktree round-trips covered by tests (local bare remote).
2026-07-21 03:46:06 +10:00