Commit Graph
170 Commits
Author SHA1 Message Date
sam 472d8d717c feat(config,channels): format-neutral config resolution and truthful channel state
Loader.Resolve() selects TOML/YAML files or a config directory as the
production entry point, replacing the TOML-only Overlay() as the default
path. Add internal/channels.Manager, reporting each adapter's actual
runtime lifecycle (configured/starting/running/degraded/failed) instead of
inferring health from configuration presence alone. Wire channel reload
through a per-adapter seam.

Fixes #55
2026-08-09 01:25:58 +10:00
sam 308666b410 chore: remove legacy terminology and dead code
- Reword docs and comments to drop vague "legacy" qualifiers where they no longer describe deprecation status
- Remove unused code: RecentEvents, MessageEvent.ToLegacy, index setState, stderr redaction tool, telegramOnlyCommands
- Delete or comment out tests covering removed or superseded behavior
2026-08-06 22:55:26 +10:00
sam e10cbbc92c refactor: pre-cleanup. 2026-08-06 01:40:15 +10:00
sam 0ddc43081e feat(setup): implement the interactive setup steps (archie-core-cbk)
internal/infrastructure/configuration/setup.Run drives operator
identity, LLM provider, models, forge, and Telegram chat channel
prompts and returns TOML edits -- not a config.Config. Building a
parallel in-memory representation to validate against was rejected
during design review: it's a second source of truth about the exact
thing (text vs. struct disagreeing) this whole feature exists to stop.
Tests instead render the edits against the real embedded template and
load the result through a real configuration.Loader, proving the
actual defaults/decode/validate path accepts it.

Two boot-time landmines in config.example.toml surfaced by that
approach, both fixed here:

- [providers.openai] ships active with api_key pointed at the bws
  engine. cmd/archied/provider_secrets.go resolves every entry in
  cfg.Providers at startup, not just the ones [models] references, and
  refuses to start the daemon if any fails -- so picking any other
  provider (or picking openai but skipping the key) left a config that
  wouldn't boot on a machine without the bws CLI installed (the common
  case). stepProvider now neutralises providers.openai.api_key to the
  zero-value secret ref whenever openai wasn't the actual choice.
- [[repos]] shipped active with a placeholder owner/name that
  validates cleanly (owner/name are only checked non-empty) and would
  have archied silently polling a repo that doesn't exist. Commented
  out in the template with an explanation; setup does not write this
  section since tomlwrite cannot target array-of-tables.
  TestExampleConfigLoads updated to expect zero repos.

Also: forge and provider secret env-var names are each defined in one
table (forgeToken, cloudProviders) read by both the config edit and
the secret sink, so the two can't name a secret differently -- the
exact bug class this feature exists to close for the installer, tested
directly (TestRun_ForgeTokenKeyMatchesStoredSecretKey compares the
decoded key against the key the secret was actually stored under, not
a hardcoded literal on both sides). Telegram writes chat.telegram.
token_env, not a secret.SecretRef -- TelegramConfig has no such field.
Telegram also requires at least one allowed_user_id: AllowedUserIDs is
deny-by-default, so a bot token with none configured answers nobody,
which reads as broken rather than as the safe default it is.

SecretSink is a caller-supplied sink (Put buffers, Commit persists),
not a return value: Run never hands back plaintext secrets, and the
caller is expected to call Commit only after proving the generated
config loads -- so a validation failure never leaves a secret written
without the config that references it.

Refs archie-core-cbk
2026-08-05 12:58:53 +10:00
sam 6ee86f2593 feat(setup): Prompter interface and a TTY-gated terminal implementation
archie-core-via. internal/infrastructure/configuration/setup defines
Prompter -- Select, ReadLine, ReadSecret, Confirm, all context-aware --
which is all the setup step logic (to follow) needs from a prompt
surface. No terminal dependency, so steps are testable without a TTY.

internal/infrastructure/terminalprompt.Terminal is the real
implementation, using golang.org/x/term. New refuses construction when
its input isn't a real terminal, with a message naming the problem --
enforcing "a non-TTY invocation fails with a clear message rather than
writing a partial config" at the earliest possible point, rather than
falling back to a plain read that would silently accept piped/scripted
input (ssh, `archied setup < answers.txt`, CI). Terminal satisfies
Prompter structurally; this package does not import that one, so
there's no infrastructure -> infrastructure edge for what Go's type
system already gives for free.

Each blocking read runs in a goroutine raced against ctx.Done(), so
Ctrl+C returns promptly instead of leaving the operator stuck; the
losing goroutine is abandoned on cancellation; term.ReadPassword's own
raw-mode restoration is not interruptible, so an abandoned secret read
can leave the terminal in raw mode until it unblocks on its own --
documented on Terminal.readSecret rather than worked around, since a
cancelled setup command is expected to exit shortly after via its own
signal handling anyway.

Tested by constructing a Terminal directly (bypassing New, same
package) over a plain pipe with an injected readSecret seam -- New's
TTY gate itself is exercised separately against a real os.Pipe(). No
weakening of the production constructor for testability.

Refs archie-core-via
2026-08-05 12:47:20 +10:00
sam 1579b027b8 feat(configuration): export Validate for archied setup to call
Loader.finalize's decode-then-validate sequence called an unexported
validate(cfg *config.Config), so nothing outside the package -- e.g.
archied setup, which builds a config.Config in memory before writing
it to disk -- could run the same checks. Validate wraps it; Loader now
calls the exported form internally (single call site, no behavior
change).

Documented the sharp edge this alone doesn't solve: Validate doesn't
apply defaults (agent.mode, dispatch.trigger, forge.type all only pass
once defaulted), and defaulting is still an unexported method on
*Loader. A cfg setup builds by hand needs those fields filled in
itself, or -- safer, since it can't drift from the real load path --
setup should render to TOML and load it back through a Loader rather
than call Validate on the in-memory value directly.

Refs archie-core-ewk.1
2026-08-05 12:44:42 +10:00
sam 32bd705286 fix(tomlwrite): recognise table headers carrying a trailing comment
activeHeaderRe/commentedHeaderRe were anchored with no allowance for a
trailing "# ..." comment on the header line itself. config.example.toml
has four such headers -- "[models] # role -> ...", [indexing], [web],
[notify] -- so Apply never recognised them as table headers at all. An
edit targeting "models" fell through every matching stage and got
appended as a second, duplicate [models] table at EOF, which
BurntSushi/toml then refuses to decode ("Key 'models' has already been
defined").

This is a real blocker for archied setup (archie-core-cbk): the models
step targets exactly this table. Found via plan review before writing
setup itself, rather than by setup's own tests discovering it later --
would have made the failure look like a setup bug rather than the
tomlwrite one it actually is.

Refs archie-core-ewk
2026-08-05 12:43:29 +10:00
sam 0d4d045050 fix(memory): fall back to the default directory instead of exiting
d615de5 made an unusable memory directory fatal. That was the wrong
call and contradicts the forge and container paths: memory is one
capability among many, and refusing to start over a path denies the
operator chat, the dashboard and everything else -- then crash-loops
under Restart=on-failure with the real error scrolling past unread.

builtin.New now reports an unusable directory through IsAvailable and
Err rather than failing, so the caller decides. The daemon warns and
falls back to the location an unset work_dir would have produced, which
is where someone who copied a config from another host will look. If
that is unusable too it says what the operator loses -- no recollection
between conversations, and failing writes -- and carries on.

The provider stays readable while degraded, so a bad path reads as an
empty memory rather than breaking mid-conversation, and the dashboard's
memory page already renders IsAvailable so this is visible there.

Also fixes a defect the fallback test caught: an empty work_dir made
filepath.Join produce the relative path "memory", which MkdirAll
creates wherever the daemon was started from -- succeeding, and putting
memory somewhere nobody would look.

Refs archie-core-rml
2026-08-04 02:13:05 +10:00
sam 76f6299f22 fix(catalog): match SDK provider fallback 2026-07-31 08:48:38 +10:00
sam 3ae59c99c0 fix(secrets): wire provider credentials through engines 2026-07-31 08:45:58 +10:00
sam 2307c8c81a feat: unify provider and model selection 2026-07-31 03:40:03 +10:00
sam 6c32550345 chore: replace personal owner names in tests and config with generic values
Test fixtures used 'sam', 'samcharles93', and 'sam/<repo>' as owner
names. These are personal and machine-specific — the repo should use
generic placeholders like 'acme' (matching the existing convention in
worktree_test.go).

config.example.toml: samcharles93 -> your-github-username
Test owner: sam -> acme
Repo refs: sam/archie-core -> acme/archie-core, etc.
Example skills: sam -> acme
2026-07-31 02:15:41 +10:00
sam bc5d8dcdc9 test: drop tests that grep source text instead of behaviour
These asserted on file contents rather than what the code does:
Dockerfile strings, deploy.yml YAML, CHANGELOG headings, and greps of
main.go for identifiers. Two also referenced config.docker.toml and
docker-compose.yml, which were never committed, so they could not pass
at all.

The worst compared strings.Index byte offsets in main.go to assert
startup ordering -- it would pass on code that never runs and fail on
a correct refactor that moved a line.
2026-07-31 01:13:16 +10:00
sam 14be7c5567 feat: embed config.example.toml and add comment-preserving TOML writer
Groundwork for archied setup owning config generation (archie-core-ewk),
so install.sh stops emitting a second copy of the config that drifts from
the example operators actually read.

configtemplate lives at the module root because go:embed cannot reach
outside its own directory and config.example.toml is kept at the root
where it is easy to hand-edit. tomlwrite applies setup answers to that
template in place, preserving the comments that make the generated file
readable - BurntSushi/toml round-trips values but not comments, so
re-encoding a decoded document would strip them.
2026-07-30 02:24:20 +10:00
sam 6be754787d polish: add install.sh, XDG setup, deployments/ examples, and github-token docs\n\n- Add install.sh following XDG Base Directory specification (~/.config/archie, ~/.local/share/archie, ~/.local/bin)\n- Add interactive setup for code forge (GitHub / Gitea / Standalone) and LLM provider selection (Ollama model picker / Cloud keys)\n- Implement NoopForge in internal/forge/noop.go for running Archie in standalone mode without a forge\n- Enable loginctl user linger and systemd user unit setup in install.sh\n- Add docs/github-token.md with step-by-step guidance for Classic PAT generation\n- Add top-level deployments/ directory with scenario templates (single-forge GitHub, multi-forge GitHub+Gitea, local Ollama standalone, systemd runbook) 2026-07-30 01:25:46 +10:00
sam 6108fc5548 fix: use errors.Is/As for context errors and cleanup 2026-07-30 00:06:10 +10:00
sam 9d5040fd65 refactor: add internal/eventbus as the broker-neutral messaging contract
dependencies-and-contracts.md rule 1 says domains must not import
internal/infrastructure. internal/daemon and internal/agentexec both did,
and for exactly the thing the doc carves out: "internal/eventbus is an
approved shared application contract ... it MUST remain broker-neutral."

The Publisher/Consumer/Requester interfaces existed but lived inside the
NATS package, so a caller could depend on the behaviour yet still had to
import the implementation to name it. They now live in internal/eventbus,
and nats.Client asserts against eventbus.Bus.

The larger violation was ownership of meaning. The doc says "message schemas
stay with their owning domains; the event bus transports messages, it does
not own their meaning." The bus exported TaskEnvelope and a PublishTask
method, so the transport knew what a unit of work was, and it exported the
archie.agent.* subject helpers, so it also owned the agent protocol.

  - TaskEnvelope, its routing Kind, the label vocabulary and the task
    subjects move to internal/domain/workintake, which encodes and decodes
    its own bytes. This starts internal/domain/ per organisation.md.
  - The agent request/response/system subjects move to internal/agentexec,
    which defines what those messages mean.
  - internal/workflow keeps only the kind-to-registry mapping; the label
    table stays single and lives with the domain that reads forge issues.
  - The bus now carries opaque payloads on plain-string subjects.

Consumers declare the smallest interface they need rather than taking the
whole Bus: daemon.TaskBus is publish-unique/fetch/request, agentexec.
RequestBus is publish-request/reply-inbox, and archie-agent's stageBus is
fetch/respond. None can reach what it does not use.

Two defects this surfaced:

  - archie-agent declared its own JetStream stream with a second copy of the
    name, subjects, retention, dedup window, MaxDeliver and TTLs that
    nats.Connect already sets -- two definitions of one stream, free to
    diverge. Config now takes Subjects and FilterSubject from composition,
    which is what let the agent use the shared client and delete its
    hand-rolled setup.
  - Replying to a request needs core NATS, not the JetStream publish path:
    a reply inbox is ephemeral and belongs to no stream, so a durable
    publish to it fails. The old code hid this by using the raw connection.
    The contract now distinguishes Respond from Publish. The agentexec
    round-trip test caught it after I had made the same mistake in
    archie-agent.

Also translated the SDK's ErrNoResponders into eventbus.ErrNoResponders, so
the daemon's taskrun retry no longer imports the NATS SDK to name a
transient condition. The SDK is now confined to the nats package, the *rpc
packages (themselves NATS infrastructure), and cmd/*.

The agentexec test fake previously hand-rolled a JetStream consumer; it now
drains the client's own consumer, which is both closer to production and
avoids conflicting consumers on a work-queue stream.

Verified: go build, go vet, gofmt clean; full test suite passes.
2026-07-29 11:40:24 +10:00
sam c66c0a2a38 refactor(config): move loading into infrastructure/configuration
internal/config both defined the types every package imports and performed
the file I/O to produce them. Importing a settings struct therefore pulled
in a TOML decoder and a YAML decoder, and the shared model owned discovery,
decoding, overlays, defaulting and validation all at once. See
docs/architecture/configuration.md, which requires loading to move out and
internal/config to be dissolved.

internal/config is now types only -- no os, no filepath, no toml. The new
package owns loading, one concern per file:

  features.go    the closed set of recognised feature files
  discovery.go   directory scan and classification
  decode.go      TOML/YAML decoding, format dispatch
  defaults.go    filling absent values, no errors
  validate.go    judging supplied values, no mutation
  provenance.go  which file contributed what
  loader.go      Loader wiring the above; File/Overlay/Dir

Beyond relocating, this fixes what the old code conflated:

  - Defaulting and validation were interleaved. applyAgentDefaults and
    applyDispatchDefaults returned validation errors; validateContainers and
    validateSingleIdentity quietly mutated the config. There was no point at
    which you could see what the operator actually supplied. They are now
    separate passes: applyDefaults never fails, validate never writes.
  - Errors were fmt.Errorf strings prefixed "config: ", so a caller could not
    distinguish operator-fixable input from an unreadable file without
    matching message text. Five sentinels now carry that distinction.
  - dataHome() read XDG_DATA_HOME directly and discarded the error from
    os.UserHomeDir, so defaulting could not be tested without the
    environment. It is injectable on Loader and no longer swallows the error.
  - Nothing recorded where a value came from. With overlays, feature files
    and conf.d/ all able to set the same field, "which file set this?" was
    answerable only by re-deriving precedence by hand. Provenance records it.
  - Feature files were applied in map iteration order, making load order and
    any resulting error arbitrary. Discovery now returns them sorted.
  - loadMainConfig reported "no config.yaml or config.toml found in %s" with
    an empty string: the directory was lost when it was extracted from
    loadDirInto, so the error named no directory.
  - Unreadable or malformed conf.d/ extras were skipped silently, which
    presents as a setting the operator wrote simply not taking effect.
  - Decode errors from the main config were returned bare, without naming
    the file that failed to parse.

The 40 loader tests moved with the code and their assertions are unchanged,
so the defaulting and validation rewrite is behaviour-preserving.

Document.Config is the temporary seam: still internal/config.Config, named
as a field so the compromise is visible at each use site, with its deletion
condition recorded in doc.go. Building typed per-domain sections now, while
every consumer still reads config.Config, would mean two shapes kept in sync
by hand with no consumer for the new one; that belongs with the dissolution
work, not ahead of it.

No consumer outside cmd/archied changed.

Verified: go build, go vet, gofmt clean; go test ./internal/config/... and
./internal/infrastructure/configuration/... pass. Pre-existing gateway and
telegram failures are unchanged.
2026-07-29 11:07:41 +10:00
sam a8efbf5046 refactor: route tasks on a typed kind, not forge labels
The event bus was inspecting forge issue labels ("bug", "feature",
"bootstrap") to pick a subject. That put a forge vocabulary inside
infrastructure and duplicated the label table already owned by
workflow.Route -- adding a label meant remembering both places, and
nothing would have caught it if you forgot.

The label table now lives once, in internal/workflow, which already owns
label semantics:

  - workflow.Kind names the routing category, with labelKinds as the
    single label vocabulary and kindWorkflows mapping kind to registry
    entry. Route consumes the ordered kind list so a "bug,feature" task
    still falls through to feasibility when no tdd workflow is
    registered, exactly as the switch did.
  - nats.TaskKind is a closed, validated set of subject tokens.
    PublishTask rejects an unknown kind rather than silently delivering
    it to the default queue; the zero kind means default, so messages
    queued before this field decode unchanged.
  - The daemon classifies at the publish site and sets TaskEnvelope.Kind.

Also fixes two defects this exposed:

  - drainNATS never reached its SQLite fallback. Replacing the old
    (nil, nil) no-message signal with ErrNoMessage in 492ca20 made that
    case continue at the top of the loop, but no-message is precisely
    when the fallback must run -- requeued tasks (waiting_human approval,
    retry-parked) never go through a NATS publish, so nothing would ever
    arrive to trigger them. go vet caught it as unreachable code.
  - config.classifyFeatureFile silently overwrote duplicate feature
    files, so config.gateway.yaml and conf.d/gateway.yaml together
    applied one at random. Its own doc comment recorded the bug
    ("or "" on success/duplicate"). The known-feature list in the error
    message is now derived from knownFeatureNames instead of being
    spelled out a second time beside it.

A test also called t.Fatalf from a non-test goroutine, where it does not
stop the test.

Verified: go build, go vet, gofmt clean. Pre-existing gateway (18) and
telegram (1) failures are unchanged in count -- they are lost routing
wiring from the parallel telegram merges, tracked separately.
2026-07-29 10:43:52 +10:00
sam 492ca209f9 refactor(nats): rebuild as infrastructure/eventbus/nats with real boundaries
Relocating internal/nats as-is would have cemented its problems into the
approved layout, so this is a rewrite rather than a move.

What was wrong:
- Conn() *nats.Conn, PublishMsg(*nats.Msg) and Fetch() jetstream.Msg pushed
  SDK types into all four consumers, so the broker was not replaceable --
  the opposite of what dependencies-and-contracts.md requires.
- PublishTask took seven positional parameters, six of them strings, where a
  transposed argument was a silent bug.
- Labels travelled as a comma-separated string and were re-split on every
  publish.
- Fetch returned (nil, nil) for 'no message', which every caller had to
  remember to special-case.
- The logger field was stored and never used.
- Stream name, consumer name and every timeout were hardcoded constants.
- Errors were inconsistently wrapped; several paths returned bare errors.
- Connect repeated nc.Close() on each failure branch.

What replaces it:
- One concern per file: config, errors, subjects, message, client,
  publisher, consumer.
- Publisher/Consumer/Requester interfaces with compile-time assertions, so
  callers depend on behaviour and can be faked without a broker.
- A Message wrapper keeps jetstream types behind the boundary; consumers no
  longer import the NATS SDK at all.
- TaskEnvelope with []string labels and JSON methods preserving the
  comma-separated wire form for already-queued messages.
- Sentinel errors (ErrNoMessage, ErrNotConnected, ErrNoReplyAddress,
  ErrInvalidConfig) and %w wrapping with operation context throughout.
- Config with validation and defaults; a single deferred cleanup in Connect.
- CoreConn() as the one documented escape hatch, for the *rpc packages that
  are themselves NATS infrastructure.

Consumers migrated: cmd/archied, cmd/archie-agent, internal/daemon,
internal/agentexec. Behaviour, subjects and wire format unchanged.
2026-07-29 10:19:09 +10:00