112 lines
5.3 KiB
Markdown
112 lines
5.3 KiB
Markdown
# e2e
|
|
|
|
Validation of the whole stack, run by hand or by an agent, never by
|
|
`task check` or CI. It needs PostgreSQL 18, five services, Vite and a browser,
|
|
which the gate must not. Docker is not needed: without it tasks queue and
|
|
park, which is useful data for the dashboard.
|
|
|
|
| File | What it does |
|
|
| --- | --- |
|
|
| `stack.sh` | Starts PostgreSQL 18 (port 55432) and the five services as local processes, then Vite on 5173 |
|
|
| `seed.sh` | Seeds the stack through the dashboard API; safe to rerun |
|
|
| `run.cjs` | Runs `checks/*.cjs` in Chromium; screenshots, manifest and gallery to `out/<stamp>/` |
|
|
| `checks/` | One module per behaviour |
|
|
| `results/` | One committed report per release |
|
|
|
|
## Run
|
|
|
|
```bash
|
|
task e2e:up # build, start everything, seed
|
|
task e2e:check # list available checks (no stack needed)
|
|
task e2e:check -- list # explicitly list checks
|
|
task e2e:check -- audit-redirect # one check
|
|
task e2e:check -- all # all checks
|
|
task e2e:check -- --headed accessibility # watch a check in Chromium
|
|
task e2e:check -- --headed all # watch every check
|
|
task e2e:down # stop; e2e/stack.sh reset also deletes the data
|
|
```
|
|
|
|
Headless is the default. `--headed` opens visible Chromium and slows browser
|
|
actions to 150ms so you can follow them; assertions and saved evidence are the
|
|
same in both modes. The terminal prints each check before it starts. Headed mode
|
|
requires a graphical desktop.
|
|
|
|
Nothing reads the installed config: `stack.sh` points the XDG directories at
|
|
`${ARCHIE_DEV_DIR:-~/.local/state/archie-dev}` and writes its overlay from
|
|
`deployments/dev.toml`. Logs are in its `logs/`. `e2e/stack.sh restart archied`
|
|
restarts one service. Run `task e2e:down` before starting again: startup refuses
|
|
occupied service ports and checks that each launched process stays alive.
|
|
|
|
The seed creates a local test administrator: `operator@example.test`, password
|
|
`archie-local-e2e-password`. Use it at `http://localhost:5173`; the browser
|
|
checks sign in with the same account. These are test fixtures for this
|
|
loopback-only stack. To use an existing test account, set `ARCHIE_E2E_EMAIL`
|
|
and `ARCHIE_E2E_PASSWORD` for both seeding and browser checks. Seeding never
|
|
resets an existing account or deletes test data.
|
|
|
|
The first run needs `pnpm install --dir e2e` and `pnpm --dir e2e exec playwright
|
|
install chromium`. In the cloud environment, Chromium and Playwright are
|
|
preinstalled; run with `NODE_PATH=$(pnpm root -g)`.
|
|
|
|
## Release evidence
|
|
|
|
```bash
|
|
node e2e/run.cjs --record v1.54.0 all
|
|
```
|
|
|
|
writes `results/v1.54.0.md`, the pass, fail or skip of every check for
|
|
that release. Commit it with the release. Screenshots stay in `out/`.
|
|
|
|
## Writing a check
|
|
|
|
A module in `checks/` exports `async ({ page, go, shot, fail, skip, settled })`.
|
|
|
|
- `go(path)` loads a dashboard route: `waitUntil: 'load'` plus 1200ms.
|
|
Wait for elements, never for `networkidle`: the event stream never goes idle.
|
|
- `shot(label)` saves `<check>-<kebab>[-<w>x<h>].png` after a short settle,
|
|
and records `{file, w, h, check}` in the manifest. The label is slugified,
|
|
so `shot('/')` becomes `routes-home.png`; a viewport other than the
|
|
1440x900 default adds a suffix (`-390x844`). Repeated labels get `-2`,
|
|
`-3`, so every capture is kept.
|
|
- `fail(msg)` records a problem without stopping.
|
|
- `skip(reason)` stops the check and records a skip: a missing
|
|
precondition, never a failure.
|
|
- `settled(sel?)` waits for a selector or URL, then settles briefly.
|
|
It is the harness-owned wait; checks need no capture wrapper or
|
|
ad-hoc sleeps around screenshots.
|
|
|
|
Each check gets a fresh page at 1440x900 with the login `storageState`;
|
|
a check that needs a phone viewport resizes it itself to 390x844.
|
|
Those are the only two viewports in use. A throw, a `fail`, or any
|
|
console or page error fails the check. One module covers one behaviour;
|
|
families share a prefix: `audit-*` for the audit page (redirect,
|
|
filters, revert, runtime, views-export, retention-live, phone),
|
|
`workflow-*` for the canvas (loading, validation, phone-layout, logic),
|
|
`chat-composer` for chat input.
|
|
|
|
`routes` loads every static route and flags text with stray leading,
|
|
trailing or doubled spaces. `seeded` checks the seed is visible.
|
|
`/logs` is visited but skipped by the whitespace lint: log lines are
|
|
data, not copy.
|
|
|
|
## Output
|
|
|
|
Every run writes `out/<stamp>/` with the screenshots,
|
|
`results.json` (the manifest: `{check, ok, ms, problems, shots[]}`,
|
|
plus `skipped`/`skipReason` for skips) and `index.html` (a gallery
|
|
grouped by check with pass, fail or skip plus problems, for human
|
|
image review).
|
|
|
|
Exits: 0 when every check passes or skips, 1 when any check fails,
|
|
2 when the harness itself errors. `ARCHIE_E2E_URL` points the run at
|
|
a dashboard other than `http://localhost:5173`.
|
|
|
|
## Pitfalls
|
|
|
|
| Symptom | Cause | Fix |
|
|
| --- | --- | --- |
|
|
| State Store exits: server is older than PostgreSQL 18 | another server answered | `stack.sh` uses port 55432; set `ARCHIE_DEV_PGPORT` if that is taken |
|
|
| Startup says a port is in use | a stack or another process already owns it | Run `task e2e:down`, then `task e2e:up`; inspect any remaining listener with `ss -ltnp` |
|
|
| Your own shell is killed (exit 144) | `pkill -f` matched its own command line | `stack.sh stop` or `restart`, which kill by PID |
|
|
| Postgres aborts in the cloud | `/tmp/claude-0` is reset to mode 700 | `stack.sh` keeps the root cluster in `/var/lib/archie-dev` |
|