Files
archie/deployments/systemd-user-service.md
T
sam 9d8f364af1
Quality / check (push) Successful in 5m53s
deploy / gate (push) Canceled after 0s
deploy / archied-image (push) Canceled after 0s
deploy / agent-image (push) Canceled after 0s
deploy / dist-zip (push) Canceled after 0s
feat(releaseupdate): signed OCI distribution sources for releases and the catalogue (#1715)
2026-10-31 18:18:43 +11:00

10 KiB

Managing archied with Systemd User Services

This runbook describes how to manage archied as a background user service on Linux servers.


1. Unit Files (written by install.sh)

install.sh writes all five units -- archied.service, archie-state-store.service, archie-gateway.service, archie-ui.service and archie-messaging.service -- with the contents it ships. This page describes what runs and why; the installer owns the contents, so inspect a unit with systemctl --user cat <unit> rather than copying it from here, where it would drift from what is written.

The Gateway unit is archie-gateway.service, written by install.sh beside archied.service.

The State Store unit is archie-state-store.service. It owns the task data in PostgreSQL and serves the StateStore gRPC contract. BOTH archied and archie-gateway dial it via [services.state].target (see the archied config below) and never open the task tables themselves. A host-only agent uses the loopback bind below; a container-mode agent needs the Docker bridge gateway address, and a non-loopback bind needs [tls] (see config.example.toml):

install.sh writes archie-state-store.service with that loopback bind.


Messaging Service Unit (~/.config/systemd/user/archie-messaging.service)

The Telegram, email and webhook channels are served by archie-messaging, not by archied, since the v1.30.0 extraction. archied starts no gateway for them and logs nothing when one is absent, so a host that updates across v1.30.0 without this unit runs on with dead channels and a daemon that looks perfectly healthy. install.sh writes this unit on a fresh install; archied update install refuses an update when a host is missing a unit and names what is missing, which now fires for a host that drifted or predates this release rather than the normal path.

It dials the Gateway for the chat contract and the State Store for the stored channel settings, and reads the same config.toml, including [tls].

install.sh writes archie-messaging.service. Its After= and Wants= lines only tidy the start sequence: nothing depends on that order, because each service waits for the peers it needs.

2. UI Service Unit (~/.config/systemd/user/archie-ui.service)

The dashboard is served by the standalone UI Service, not by archied. It reads the same config.toml, dials the Gateway and State Store targets already in it, and binds [web].listen. Open the dashboard to configure the organisation, workspace and first administrator account. Complete setup before exposing it publicly; remote access requires an HTTPS reverse proxy.

Auth note: archie-ui authenticates to the Gateway, State Store and harness with its [tls] certificate (archie-ui.pem) from config.toml.

For supervision, GET /healthz on [web].listen is the process's liveness probe (token-free by design); the authenticated readiness surface lives at the dashboard's health endpoint behind account authentication. A watchdog can curl /healthz without credentials.

install.sh writes archie-ui.service. Browser accounts are configured through the dashboard.

After= orders the start, it does not wait for the store to bind. The UI process closes that gap itself: its event pump retries priming with backoff until the State Store answers, so the activity feed goes live once the store is up rather than staying history-only until the next restart.

For remote access, configure a TLS reverse proxy and the instance public URL in Access → Authentication. Enrol passkeys from Your account on HTTPS or localhost.


3. Enabling User Linger (loginctl)

By default, systemd terminates user services when you log out of SSH. Enabling linger allows your archied service to run continuously 24/7 across reboots and logouts:

loginctl enable-linger "$USER"

Verify linger status:

loginctl show-user "$USER" | grep Linger
# Output: Linger=yes

4. Service Commands

Reload systemd daemon files:

systemctl --user daemon-reload

Enable and start the service immediately:

systemctl --user enable --now archie-state-store archie-gateway archie-messaging archie-ui archied

Check service status:

systemctl --user status archied

Tail live daemon logs:

journalctl --user -u archied -f

Restart or stop the service:

systemctl --user restart archied
systemctl --user stop archied

5. Default: Native Daemon, Embedded NATS, Managed Containers

Embedded NATS is the default. The systemd service runs archied natively, so interactive chat keeps the host access the operator configured, while every autonomous repository workflow runs in a task-scoped Docker container. No Compose service or host archie-agent binary is required.

Docker must be installed and reachable by the service user. With no explicit broker or container settings, Archie defaults containers.image to git.catlow.cloud/sam/archie-agent:latest. single-forge-github.toml and docker-nats-stack.toml override that default with a pinned reference (standalone.toml and dev.toml leave it) -- see Pinning the archie-agent image below. An explicit equivalent, pinned the same way, is:

[nats]
mode = "embedded"

[containers]
image = "git.catlow.cloud/sam/archie-agent@sha256:<digest>"
pull_policy = "missing"

At startup Archie resolves the Docker bridge used by its task containers, binds the authenticated embedded broker only to that bridge's host gateway, and passes the resulting endpoint to each worker. There is no port to publish and no broker URL to maintain.

The State Store is a separate process and, with a container-mode agent, must be reachable from the container, so bind it to the same bridge gateway with a token and point [services.state].target at it (start archie-state-store with -listen <bridge>:9090 -token <token>); without a token a non-loopback bind fails closed. For a host-only agent, 127.0.0.1:9090 with no token is correct.

If Docker or the image is unavailable, archied still serves chat and the dashboard. Autonomous tasks park with an explicit capability error; they never fall back to a host model loop.


6. Pinning the archie-agent Image

archie-agent:latest is a moving target: every release re-publishes it, so it can change under a running deployment with no warning and no way to reproduce a prior build. Two paths publish it, to different references:

  • .github/workflows/deploy.yml publishes git.catlow.cloud/sam/archie-agent as :latest and :sha-<commit> when the commit carries a release tag; an ordinary push to main publishes nothing for the agent image.
  • tools/publish.sh, run by hand for a release, publishes git.catlow.cloud/sam/archie-agent as :latest and :sha-<commit>, which is the registry the shipped default above names.

A distribution zip pins whichever one its own release published, by digest, in release.json; archied update install then pulls exactly that.

As of this writing, archie-agent has no published semantic-version tag -- both paths publish only :latest and an immutable :sha-<commit> (archied is versioned independently per RELEASING.md, but that versioning is not yet reflected as a registry tag for the agent image). Until one publishes it, the only reproducible reference is a content digest, resolved once and pinned explicitly:

[containers]
image = "git.catlow.cloud/sam/archie-agent@sha256:<digest>"

This is the digest latest resolved to on 2026-09-05. Digest pinning is the strongest option for production: it is immutable and independently verifiable (docker manifest inspect git.catlow.cloud/sam/archie-agent@sha256:...), at the cost of being opaque in a config file a human is meant to edit -- resolve a fresh digest deliberately (docker pull ...:latest followed by docker inspect --format '{{index .RepoDigests 0}}') rather than editing this one in place expecting it to track new releases.

If/when CI starts publishing a semantic-version tag for archie-agent (e.g. git.catlow.cloud/sam/archie-agent:v1.21.0), prefer that tag for day-to-day config readability, and reserve digest pinning for deployments that need byte-for-byte reproducibility guarantees a tag alone cannot give (tags are mutable pointers; a compromised or accidentally-overwritten tag still resolves, a digest cannot).

A reference on a registry that needs a login needs the credential in [containers].registry_auth -- the release's own git.catlow.cloud/sam/archie-agent digest is one, since an anonymous pull cannot reach it:

[containers]
registry_auth = {engine = "env", key = "ARCHIE_REGISTRY_AUTH"}

The secret resolves to a Docker AuthConfig JSON document: the object docker login stores under auths, without the surrounding map, and with a personal access token that can read the package as the password -- {"username":"<user>","password":"<token>"}. Put it in ~/.config/archie/env as ARCHIE_REGISTRY_AUTH so it stays out of config.toml. The field is file-owned and settled at boot, so a change takes effect on restart; a value that cannot be read logs a warning and pulls anonymously, which is the failure the task reports as a capability error.

The bundled update adapter snapshots the configured [containers].image before rebuilding it and restores that image if the restarted daemon fails its health check. A custom image reference is therefore supported directly in config.toml; ARCHIE_AGENT_IMAGE is only an override for deployments that cannot make that configuration file available to the update command.

7. Optional: External Compose NATS

External NATS changes only broker deployment. It does not change the worker executor. Start the optional service and configure the pinned Compose gateway:

docker compose up -d nats
[nats]
mode = "external"
url = "nats://172.19.0.1:4222"

[containers]
network = "archie-core_default"

The daemon hands workers the URL its own client connected to. The Compose service name is not resolvable from the native host, and localhost inside a worker is the worker itself; the pinned network gateway is reachable from both.

pull_policy = "always" cannot authenticate to a private registry because the daemon sends no registry credentials to the Docker API. Refresh a private image with the operator's Docker credentials and retain pull_policy = "missing":

docker compose pull agent

state_dir and other paths are native host paths. Point state_dir at the intended state directory (embedded NATS store, task logs).