Files
archie/e2e/README.md
T

112 lines
5.3 KiB
Markdown

# e2e
Validation of the whole stack, run by hand or by an agent, never by
`task check` or CI. It needs PostgreSQL 18, five services, Vite and a browser,
which the gate must not. Docker is not needed: without it tasks queue and
park, which is useful data for the dashboard.
| File | What it does |
| --- | --- |
| `stack.sh` | Starts PostgreSQL 18 (port 55432) and the five services as local processes, then Vite on 5173 |
| `seed.sh` | Seeds the stack through the dashboard API; safe to rerun |
| `run.cjs` | Runs `checks/*.cjs` in Chromium; screenshots, manifest and gallery to `out/<stamp>/` |
| `checks/` | One module per behaviour |
| `results/` | One committed report per release |
## Run
```bash
task e2e:up # build, start everything, seed
task e2e:check # list available checks (no stack needed)
task e2e:check -- list # explicitly list checks
task e2e:check -- audit-redirect # one check
task e2e:check -- all # all checks
task e2e:check -- --headed accessibility # watch a check in Chromium
task e2e:check -- --headed all # watch every check
task e2e:down # stop; e2e/stack.sh reset also deletes the data
```
Headless is the default. `--headed` opens visible Chromium and slows browser
actions to 150ms so you can follow them; assertions and saved evidence are the
same in both modes. The terminal prints each check before it starts. Headed mode
requires a graphical desktop.
Nothing reads the installed config: `stack.sh` points the XDG directories at
`${ARCHIE_DEV_DIR:-~/.local/state/archie-dev}` and writes its overlay from
`deployments/dev.toml`. Logs are in its `logs/`. `e2e/stack.sh restart archied`
restarts one service. Run `task e2e:down` before starting again: startup refuses
occupied service ports and checks that each launched process stays alive.
The seed creates a local test administrator: `operator@example.test`, password
`archie-local-e2e-password`. Use it at `http://localhost:5173`; the browser
checks sign in with the same account. These are test fixtures for this
loopback-only stack. To use an existing test account, set `ARCHIE_E2E_EMAIL`
and `ARCHIE_E2E_PASSWORD` for both seeding and browser checks. Seeding never
resets an existing account or deletes test data.
The first run needs `pnpm install --dir e2e` and `pnpm --dir e2e exec playwright
install chromium`. In the cloud environment, Chromium and Playwright are
preinstalled; run with `NODE_PATH=$(pnpm root -g)`.
## Release evidence
```bash
node e2e/run.cjs --record v1.54.0 all
```
writes `results/v1.54.0.md`, the pass, fail or skip of every check for
that release. Commit it with the release. Screenshots stay in `out/`.
## Writing a check
A module in `checks/` exports `async ({ page, go, shot, fail, skip, settled })`.
- `go(path)` loads a dashboard route: `waitUntil: 'load'` plus 1200ms.
Wait for elements, never for `networkidle`: the event stream never goes idle.
- `shot(label)` saves `<check>-<kebab>[-<w>x<h>].png` after a short settle,
and records `{file, w, h, check}` in the manifest. The label is slugified,
so `shot('/')` becomes `routes-home.png`; a viewport other than the
1440x900 default adds a suffix (`-390x844`). Repeated labels get `-2`,
`-3`, so every capture is kept.
- `fail(msg)` records a problem without stopping.
- `skip(reason)` stops the check and records a skip: a missing
precondition, never a failure.
- `settled(sel?)` waits for a selector or URL, then settles briefly.
It is the harness-owned wait; checks need no capture wrapper or
ad-hoc sleeps around screenshots.
Each check gets a fresh page at 1440x900 with the login `storageState`;
a check that needs a phone viewport resizes it itself to 390x844.
Those are the only two viewports in use. A throw, a `fail`, or any
console or page error fails the check. One module covers one behaviour;
families share a prefix: `audit-*` for the audit page (redirect,
filters, revert, runtime, views-export, retention-live, phone),
`workflow-*` for the canvas (loading, validation, phone-layout, logic),
`chat-composer` for chat input.
`routes` loads every static route and flags text with stray leading,
trailing or doubled spaces. `seeded` checks the seed is visible.
`/logs` is visited but skipped by the whitespace lint: log lines are
data, not copy.
## Output
Every run writes `out/<stamp>/` with the screenshots,
`results.json` (the manifest: `{check, ok, ms, problems, shots[]}`,
plus `skipped`/`skipReason` for skips) and `index.html` (a gallery
grouped by check with pass, fail or skip plus problems, for human
image review).
Exits: 0 when every check passes or skips, 1 when any check fails,
2 when the harness itself errors. `ARCHIE_E2E_URL` points the run at
a dashboard other than `http://localhost:5173`.
## Pitfalls
| Symptom | Cause | Fix |
| --- | --- | --- |
| State Store exits: server is older than PostgreSQL 18 | another server answered | `stack.sh` uses port 55432; set `ARCHIE_DEV_PGPORT` if that is taken |
| Startup says a port is in use | a stack or another process already owns it | Run `task e2e:down`, then `task e2e:up`; inspect any remaining listener with `ss -ltnp` |
| Your own shell is killed (exit 144) | `pkill -f` matched its own command line | `stack.sh stop` or `restart`, which kill by PID |
| Postgres aborts in the cloud | `/tmp/claude-0` is reset to mode 700 | `stack.sh` keeps the root cluster in `/var/lib/archie-dev` |