1
Staging and QA
Sam Catlow edited this page 2026-10-11 14:40:58 +11:00

Staging and release QA

Guide home · Previous: project tracking · Next: release process

Staging is a disposable rehearsal of the supported installation. It consumes the packaged candidate artifacts and follows the deployment procedure intended for production.

Choose the first reference installation

The proposed first target is the installation Sam actually operates. Confirm its process supervisor and configuration before fixing the staging design. The current distribution supplies five Linux/amd64 host binaries, PostgreSQL, and a containerized agent runtime:

  • archie-state-store
  • archie-gateway
  • archie-ui
  • archie-messaging
  • archied
  • archie-agent, running in task containers

Match the deployed service layout, PostgreSQL version, configuration mechanism, artifact installation method, and image pull method. Keep each service's existing ownership and contract boundaries. Build one supported deployment path first; additional topologies can follow after that path is dependable.

Staging needs separate databases, credentials, forge repository, messaging destinations, and task storage. Use synthetic data or an approved sanitised snapshot. It must not consume production work queues or send test messages to real recipients.

Provision and reset it through automation. CI must be able to reach its artifact storage and registry. A reset should not require an undocumented server login.

Required release checks

Check Behaviour to prove Evidence
Fresh installation Packaged artifacts install and all services become usable Installer output, versions, manifest, service health
Upgrade Previous stable installation upgrades while retaining state and settings Before/after state assertions and migration logs
Representative workflow An event is accepted durably, dispatches a real agent/tool task, and produces a visible result Task ID, event trail, task outcome, resulting artifact
Chat/channel path A test message reaches the gateway, uses the configured model/tool path, and returns a reply Message/turn identifiers and outcome
Dashboard setting A changed setting persists and affects its consumer API/browser observation of the saved setting and effect
Recovery An unhealthy candidate triggers the documented recovery and work remains accounted for Failure injection, recovery logs, restored versions and task state
Release-specific QA Each changed user-facing behaviour works as described The concrete scenario and its result

Use test repositories and channels for outward actions. Include at least one real model/tool interaction in the appropriate end-to-end path; a fake provider alone cannot prove provider compatibility. Keep it bounded and capture cost where practical.

Service readiness is necessary, but the representative workflow proves that services can do useful work together. A service temporarily missing must degrade locally without losing accepted work or causing peer processes to exit.

Test upgrades and recovery honestly

For 0.1.0, rehearse the explicit transition from the existing legacy installation. Retain its original binaries, image digest, configuration, and approved database backup. Later releases rehearse upgrades from the preceding supported 0.x release.

A binary rollback does not reverse a database migration. For each schema-changing release, establish whether the previous binaries can safely use the migrated database. If they cannot, record and rehearse the required forward fix or database restore procedure. Database restore can lose work accepted since the snapshot, so it requires an explicit recovery decision and reconciliation plan.

Exercise interruption during installation and unhealthy startup through the existing updater mechanisms. Reuse its locking, journal, work-in-flight refusal, and independent watchdog. Confirm the actual deployment path behaves correctly rather than inferring it from source-level tests.

Record a QA result

For each candidate, retain:

  • Commit, intended version, pipeline run, artifact checksums, and image digests.
  • Tested topology and starting version.
  • Each required scenario with pass/fail and supporting logs or artifacts.
  • Known limitations accepted by Sam.
  • Recovery result and any manual actions required.

Do not store secrets or complete unsanitised request payloads in the evidence bundle. Keep candidate evidence available after the runner's temporary workspace disappears.

Complete when: automation can provision staging, install the candidate, execute the required scenarios, collect a result, and clean up. A failed scenario blocks promotion and links to a concrete issue.