test(e2e): split overloaded checks, standardize screenshots
This commit is contained in:
1 parent
61d3cbeb8a
commit
90b2b89bac
17 files changed
+879
-649
No files matched your search
+42
-10
@@ -9,7 +9,7 @@ park, which is useful data for the dashboard.
|
||||
| --- | --- |
|
||||
| `stack.sh` | Starts PostgreSQL 18 (port 55432) and the five services as local processes, then Vite on 5173 |
|
||||
| `seed.sh` | Seeds the stack through the dashboard API; safe to rerun |
|
||||
| `run.cjs` | Runs `checks/*.cjs` in Chromium; screenshots to `out/` |
|
||||
| `run.cjs` | Runs `checks/*.cjs` in Chromium; screenshots, manifest and gallery to `out/<stamp>/` |
|
||||
| `checks/` | One module per behaviour |
|
||||
| `results/` | One committed report per release |
|
||||
|
||||
@@ -19,7 +19,7 @@ park, which is useful data for the dashboard.
|
||||
task e2e:up # build, start everything, seed
|
||||
task e2e:check # list available checks (no stack needed)
|
||||
task e2e:check -- list # explicitly list checks
|
||||
task e2e:check -- workflow-phone # one check
|
||||
task e2e:check -- audit-redirect # one check
|
||||
task e2e:check -- all # all checks
|
||||
task e2e:check -- --headed accessibility # watch a check in Chromium
|
||||
task e2e:check -- --headed all # watch every check
|
||||
@@ -54,19 +54,52 @@ preinstalled; run with `NODE_PATH=$(pnpm root -g)`.
|
||||
node e2e/run.cjs --record v1.54.0 all
|
||||
```
|
||||
|
||||
writes `results/v1.54.0.md`, the pass or fail of every check for that
|
||||
release. Commit it with the release. Screenshots stay in `out/`.
|
||||
writes `results/v1.54.0.md`, the pass, fail or skip of every check for
|
||||
that release. Commit it with the release. Screenshots stay in `out/`.
|
||||
|
||||
## Writing a check
|
||||
|
||||
A module in `checks/` exports `async ({ page, go, shot, fail })`. `go(path)`
|
||||
loads a dashboard route, `shot(label)` saves a screenshot, `fail(msg)` records
|
||||
a problem without stopping. A throw, a `fail`, or any console or page error
|
||||
fails the check. Wait for elements, never for `networkidle`: the event stream
|
||||
never goes idle.
|
||||
A module in `checks/` exports `async ({ page, go, shot, fail, skip, settled })`.
|
||||
|
||||
- `go(path)` loads a dashboard route: `waitUntil: 'load'` plus 1200ms.
|
||||
Wait for elements, never for `networkidle`: the event stream never goes idle.
|
||||
- `shot(label)` saves `<check>-<kebab>[-<w>x<h>].png` after a short settle,
|
||||
and records `{file, w, h, check}` in the manifest. The label is slugified,
|
||||
so `shot('/')` becomes `routes-home.png`; a viewport other than the
|
||||
1440x900 default adds a suffix (`-390x844`). Repeated labels get `-2`,
|
||||
`-3`, so every capture is kept.
|
||||
- `fail(msg)` records a problem without stopping.
|
||||
- `skip(reason)` stops the check and records a skip: a missing
|
||||
precondition, never a failure.
|
||||
- `settled(sel?)` waits for a selector or URL, then settles briefly.
|
||||
It is the harness-owned wait; checks need no capture wrapper or
|
||||
ad-hoc sleeps around screenshots.
|
||||
|
||||
Each check gets a fresh page at 1440x900 with the login `storageState`;
|
||||
a check that needs a phone viewport resizes it itself to 390x844.
|
||||
Those are the only two viewports in use. A throw, a `fail`, or any
|
||||
console or page error fails the check. One module covers one behaviour;
|
||||
families share a prefix: `audit-*` for the audit page (redirect,
|
||||
filters, revert, runtime, views-export, retention-live, phone),
|
||||
`workflow-*` for the canvas (loading, validation, phone-layout, logic),
|
||||
`chat-composer` for chat input.
|
||||
|
||||
`routes` loads every static route and flags text with stray leading,
|
||||
trailing or doubled spaces. `seeded` checks the seed is visible.
|
||||
`/logs` is visited but skipped by the whitespace lint: log lines are
|
||||
data, not copy.
|
||||
|
||||
## Output
|
||||
|
||||
Every run writes `out/<stamp>/` with the screenshots,
|
||||
`results.json` (the manifest: `{check, ok, ms, problems, shots[]}`,
|
||||
plus `skipped`/`skipReason` for skips) and `index.html` (a gallery
|
||||
grouped by check with pass, fail or skip plus problems, for human
|
||||
image review).
|
||||
|
||||
Exits: 0 when every check passes or skips, 1 when any check fails,
|
||||
2 when the harness itself errors. `ARCHIE_E2E_URL` points the run at
|
||||
a dashboard other than `http://localhost:5173`.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
@@ -75,5 +108,4 @@ trailing or doubled spaces. `seeded` checks the seed is visible.
|
||||
| State Store exits: server is older than PostgreSQL 18 | another server answered | `stack.sh` uses port 55432; set `ARCHIE_DEV_PGPORT` if that is taken |
|
||||
| Startup says a port is in use | a stack or another process already owns it | Run `task e2e:down`, then `task e2e:up`; inspect any remaining listener with `ss -ltnp` |
|
||||
| Your own shell is killed (exit 144) | `pkill -f` matched its own command line | `stack.sh stop` or `restart`, which kill by PID |
|
||||
| `pnpm dlx shadcn-vue add` cannot fetch the registry | Node's fetch ignores the cloud agent proxy | `curl` the registry JSON and copy the files |
|
||||
| Postgres aborts in the cloud | `/tmp/claude-0` is reset to mode 700 | `stack.sh` keeps the root cluster in `/var/lib/archie-dev` |
|
||||
@@ -1,27 +1,30 @@
|
||||
// Keyboard and phone access: the task dialog fits and tabs in order, pages
|
||||
// title correctly, the skip link and palette focus main content, and
|
||||
// query-only tab switches keep focus where it is.
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
module.exports = async ({ page, go, shot }) => {
|
||||
await page.setViewportSize({ width: 360, height: 700 });
|
||||
module.exports = async ({ page, go, shot, settled }) => {
|
||||
await page.setViewportSize({ width: 390, height: 844 });
|
||||
await go('/tasks');
|
||||
await page.getByRole('button', { name: 'Jump to', exact: true }).click();
|
||||
await page.getByRole('option', { name: 'New task', exact: true }).click();
|
||||
const dialog = page.getByRole('dialog');
|
||||
await dialog.waitFor();
|
||||
await page.waitForTimeout(200);
|
||||
await settled();
|
||||
const bounds = await dialog.boundingBox();
|
||||
assert.ok(bounds.y >= 0 && bounds.y + bounds.height <= 700, 'New task dialog extends outside viewport');
|
||||
assert.ok(bounds.y >= 0 && bounds.y + bounds.height <= 844, 'New task dialog extends outside viewport');
|
||||
await shot('phone-task-dialog-top');
|
||||
const start = dialog.getByRole('button', { name: 'Start task', exact: true });
|
||||
await start.scrollIntoViewIfNeeded();
|
||||
const button = await start.boundingBox();
|
||||
assert.ok(button.y >= 0 && button.y + button.height <= 700, 'Start task cannot be reached by scrolling');
|
||||
assert.ok(button.y >= 0 && button.y + button.height <= 844, 'Start task cannot be reached by scrolling');
|
||||
await shot('phone-task-dialog');
|
||||
const close = dialog.getByRole('button', { name: 'Close', exact: true });
|
||||
await dialog.getByRole('button', { name: 'Cancel', exact: true }).focus();
|
||||
await page.keyboard.press('Tab');
|
||||
assert.ok(await close.evaluate(el => el === document.activeElement), 'Close is missing from keyboard tab order');
|
||||
const closeBounds = await close.boundingBox();
|
||||
assert.ok(closeBounds.y >= 0 && closeBounds.y + closeBounds.height <= 700, 'Close cannot be reached by keyboard');
|
||||
assert.ok(closeBounds.y >= 0 && closeBounds.y + closeBounds.height <= 844, 'Close cannot be reached by keyboard');
|
||||
await page.keyboard.press('Escape');
|
||||
for (const [path, title] of [['/tasks', 'Tasks'], ['/events', 'Events'], ['/account', 'Your account'], ['/settings/models', 'Models | Instance'], ['/org/models', 'Models | Organisation']]) {
|
||||
await go(path);
|
||||
@@ -34,33 +37,16 @@ module.exports = async ({ page, go, shot }) => {
|
||||
assert.equal(await page.evaluate(() => document.activeElement?.id), 'main-content');
|
||||
await page.getByRole('button', { name: 'Sections', exact: true }).click();
|
||||
await page.getByRole('navigation', { name: 'Sections', exact: true }).getByRole('link', { name: 'Tasks', exact: true }).click();
|
||||
await page.waitForTimeout(300);
|
||||
await page.waitForFunction(() => document.activeElement?.id === 'main-content', null, { timeout: 5000 }).catch(() => {});
|
||||
assert.equal(await page.evaluate(() => document.activeElement?.id), 'main-content', 'route navigation did not focus main');
|
||||
await page.getByRole('button', { name: 'Jump to', exact: true }).click();
|
||||
await page.getByRole('option', { name: 'Events', exact: true }).click();
|
||||
await page.waitForTimeout(300);
|
||||
await page.waitForFunction(() => document.activeElement?.id === 'main-content', null, { timeout: 5000 }).catch(() => {});
|
||||
assert.equal(await page.evaluate(() => document.activeElement?.id), 'main-content', 'palette navigation did not focus main');
|
||||
const composer = page.getByRole('textbox', { name: 'Message Archie', exact: true });
|
||||
await composer.fill('Composition in progress');
|
||||
const sent = [];
|
||||
await page.route('**/api/chat/stream', async route => {
|
||||
sent.push(route.request().postDataJSON().text);
|
||||
await route.fulfill({ contentType: 'text/event-stream', body: 'data: {"type":"done","text":"Test response"}\n\n' });
|
||||
});
|
||||
for (const event of [{ isComposing: true }, { keyCode: 229 }]) {
|
||||
await composer.dispatchEvent('keydown', { key: 'Enter', bubbles: true, cancelable: true, ...event });
|
||||
assert.equal(await composer.inputValue(), 'Composition in progress', 'IME confirmation submitted the draft');
|
||||
}
|
||||
assert.equal(sent.length, 0);
|
||||
await composer.press('Shift+Enter');
|
||||
assert.ok((await composer.inputValue()).includes('\n'), 'Shift+Enter no longer inserts a newline');
|
||||
await composer.fill('Ordinary Enter submits');
|
||||
const request = page.waitForRequest('**/api/chat/stream');
|
||||
await composer.press('Enter');
|
||||
assert.equal((await request).postDataJSON().text, 'Ordinary Enter submits');
|
||||
assert.equal(await composer.inputValue(), '');
|
||||
await shot('chat-composition');
|
||||
await page.getByRole('button', { name: 'Hide conversation', exact: true }).click();
|
||||
// The conversation panel is only open on some entries to this page; the
|
||||
// focus check below needs it closed either way.
|
||||
const hide = page.getByRole('button', { name: 'Hide conversation', exact: true });
|
||||
if (await hide.count()) await hide.click();
|
||||
const tab = page.getByRole('tab', { name: /^Map/ });
|
||||
await tab.click();
|
||||
assert.ok(await tab.evaluate(el => el === document.activeElement), 'query-only navigation stole focus');
|
||||
|
||||
@@ -0,0 +1,113 @@
|
||||
// Audit filters are server-evaluated through the URL contract: search,
|
||||
// category, time range, back/forward restore, denied, and the actor filter.
|
||||
const assert = require('node:assert/strict');
|
||||
const { trackAudit } = require('./helpers/audit-helpers.cjs');
|
||||
|
||||
module.exports = async ({ page, go, shot, fail }) => {
|
||||
const { requestMatching, waitForUrl } = trackAudit(page);
|
||||
|
||||
await go('/settings/audit');
|
||||
const status = await page.evaluate(async () => (await fetch('/api/auth/status')).json());
|
||||
const actor = status?.user?.id;
|
||||
assert.ok(actor, 'signed-in identity has no id');
|
||||
|
||||
const search = page.getByLabel('Search events', { exact: true });
|
||||
let request = await requestMatching(
|
||||
() => search.fill('repository'),
|
||||
(u) => u.searchParams.get('q') === 'repository',
|
||||
);
|
||||
assert.ok(request, 'search issued no matching /api/audit request');
|
||||
await waitForUrl((u) => u.searchParams.get('q') === 'repository');
|
||||
assert.equal((await search.inputValue()), 'repository');
|
||||
|
||||
await go('/settings/audit');
|
||||
request = await requestMatching(
|
||||
() => page.getByRole('button', { name: 'Runtime', exact: true }).click(),
|
||||
(u) => u.searchParams.get('categories') === 'runtime',
|
||||
);
|
||||
assert.ok(request, 'category filter issued no matching /api/audit request');
|
||||
await waitForUrl((u) => u.searchParams.get('categories') === 'runtime');
|
||||
assert.equal(
|
||||
await page.getByRole('button', { name: 'Runtime', exact: true }).getAttribute('aria-pressed'),
|
||||
'true',
|
||||
);
|
||||
|
||||
// Time: Any time drops the window from the request rather than guessing one.
|
||||
await page.getByRole('combobox', { name: 'Time range', exact: true }).click();
|
||||
request = await requestMatching(
|
||||
() => page.getByRole('option', { name: 'Any time', exact: true }).click(),
|
||||
(u) => u.searchParams.get('from') === null && u.searchParams.get('categories') === 'runtime',
|
||||
);
|
||||
assert.ok(request, 'Any time issued no matching /api/audit request');
|
||||
await waitForUrl((u) => u.searchParams.get('since') === 'any');
|
||||
assert.equal(await page.getByRole('combobox', { name: 'Time range', exact: true }).innerText(), 'Any time');
|
||||
|
||||
// Back/forward restore the controls the URL describes.
|
||||
const backRequest = page.waitForRequest(
|
||||
(r) => {
|
||||
const u = new URL(r.url());
|
||||
return u.pathname === '/api/audit' && u.searchParams.get('categories') === 'runtime' && !!u.searchParams.get('from');
|
||||
},
|
||||
{ timeout: 8000 },
|
||||
);
|
||||
await page.goBack();
|
||||
await backRequest;
|
||||
await waitForUrl((u) => u.searchParams.get('categories') === 'runtime' && u.searchParams.get('since') === null);
|
||||
assert.equal(await page.getByRole('combobox', { name: 'Time range', exact: true }).innerText(), 'Last 30 days');
|
||||
assert.equal(
|
||||
await page.getByRole('button', { name: 'Runtime', exact: true }).getAttribute('aria-pressed'),
|
||||
'true',
|
||||
'back did not restore the category toggle',
|
||||
);
|
||||
const forwardRequest = page.waitForRequest(
|
||||
(r) => {
|
||||
const u = new URL(r.url());
|
||||
return u.pathname === '/api/audit' && u.searchParams.get('categories') === 'runtime' && u.searchParams.get('from') === null;
|
||||
},
|
||||
{ timeout: 8000 },
|
||||
);
|
||||
await page.goForward();
|
||||
await forwardRequest;
|
||||
await waitForUrl((u) => u.searchParams.get('since') === 'any');
|
||||
assert.equal(await page.getByRole('combobox', { name: 'Time range', exact: true }).innerText(), 'Any time');
|
||||
|
||||
// Denied reaches the server through the same URL contract as the other
|
||||
// filters. It must both update the URL and issue a request.
|
||||
await go('/settings/audit');
|
||||
const denied = page.getByRole('button', { name: 'Denied', exact: true });
|
||||
request = await requestMatching(
|
||||
() => denied.click(),
|
||||
(u) => u.searchParams.get('outcome') === 'denied',
|
||||
3000,
|
||||
);
|
||||
const deniedUrl = new URL(page.url());
|
||||
if (!request || deniedUrl.searchParams.get('outcome') !== 'denied') {
|
||||
await shot('denied');
|
||||
fail(
|
||||
`Denied filter did not reach the server: URL ${page.url()} has outcome=${deniedUrl.searchParams.get('outcome') ?? 'absent'}, ` +
|
||||
`aria-pressed=${await denied.getAttribute('aria-pressed')}, no /api/audit request with outcome=denied`,
|
||||
);
|
||||
} else {
|
||||
assert.equal(await denied.getAttribute('aria-pressed'), 'true');
|
||||
}
|
||||
|
||||
// Advanced actor filter: server-evaluated, in the URL, shown as a chip.
|
||||
await go('/settings/audit');
|
||||
await page.getByRole('button', { name: 'Filters', exact: true }).click();
|
||||
const actorInput = page.getByLabel('Actor', { exact: true });
|
||||
request = await requestMatching(
|
||||
async () => {
|
||||
await actorInput.fill(actor);
|
||||
await actorInput.press('Enter');
|
||||
},
|
||||
(u) => u.searchParams.get('actor') === actor,
|
||||
);
|
||||
assert.ok(request, 'advanced actor filter issued no matching /api/audit request');
|
||||
await waitForUrl((u) => u.searchParams.get('actor') === actor);
|
||||
assert.match(
|
||||
await page.locator('ul[aria-label="Active filters"] li', { hasText: 'Actor:' }).first().innerText(),
|
||||
new RegExp(actor),
|
||||
);
|
||||
await shot('filters');
|
||||
await page.keyboard.press('Escape');
|
||||
};
|
||||
@@ -0,0 +1,42 @@
|
||||
// The audit page at phone width: secondary columns collapse, and the list,
|
||||
// an expanded row and the filters dialog fit without horizontal overflow.
|
||||
const assert = require('node:assert/strict');
|
||||
const { trackAudit } = require('./helpers/audit-helpers.cjs');
|
||||
|
||||
module.exports = async ({ page, go, shot, fail, settled }) => {
|
||||
trackAudit(page);
|
||||
|
||||
await page.setViewportSize({ width: 390, height: 844 });
|
||||
await go('/settings/audit');
|
||||
const assertNoOverflow = async (label) => {
|
||||
const { scrollWidth, innerWidth } = await page.evaluate(() => ({
|
||||
scrollWidth: document.documentElement.scrollWidth,
|
||||
innerWidth,
|
||||
}));
|
||||
if (scrollWidth > innerWidth) {
|
||||
fail(`${label} overflows 390px: documentElement.scrollWidth=${scrollWidth} > innerWidth=${innerWidth}`);
|
||||
}
|
||||
};
|
||||
await page.locator('[data-row-key]').first().waitFor();
|
||||
await assertNoOverflow('audit list');
|
||||
assert.equal(
|
||||
await page.locator('thead th', { hasText: 'Target' }).first().evaluate((el) => getComputedStyle(el).display),
|
||||
'none',
|
||||
'secondary columns are still rendered on the phone',
|
||||
);
|
||||
await shot('list');
|
||||
|
||||
const phoneRow = page.locator('[data-row-key]').first();
|
||||
await phoneRow.focus();
|
||||
await page.keyboard.press('Enter');
|
||||
await settled();
|
||||
assert.equal(await phoneRow.getAttribute('aria-expanded'), 'true', 'phone row did not expand');
|
||||
await assertNoOverflow('expanded change set');
|
||||
await shot('expanded');
|
||||
|
||||
await page.getByRole('button', { name: 'Filters', exact: true }).click();
|
||||
await page.getByRole('dialog').first().waitFor();
|
||||
await settled();
|
||||
await assertNoOverflow('advanced filters');
|
||||
await shot('filters');
|
||||
};
|
||||
@@ -0,0 +1,50 @@
|
||||
// A deep link to the old history route redirects to the audit page keeping
|
||||
// section, actor, since and the hash, and the feed loads with filter chips.
|
||||
const assert = require('node:assert/strict');
|
||||
const { trackAudit } = require('./helpers/audit-helpers.cjs');
|
||||
|
||||
module.exports = async ({ page, go, shot, fail }) => {
|
||||
const { lastRequest } = trackAudit(page);
|
||||
|
||||
await go('/settings/audit');
|
||||
const status = await page.evaluate(async () => (await fetch('/api/auth/status')).json());
|
||||
const actor = status?.user?.id;
|
||||
assert.ok(actor, 'signed-in identity has no id');
|
||||
|
||||
const deepLink = `/settings/history?section=container-runtime-policies&actor=${encodeURIComponent(actor)}&since=30d#deep`;
|
||||
await go(deepLink);
|
||||
assert.match(page.url(), /\/settings\/audit\?/, 'history did not redirect to the audit route');
|
||||
assert.match(page.url(), /section=container-runtime-policies/, 'redirect dropped section');
|
||||
assert.match(page.url(), new RegExp(`actor=${encodeURIComponent(actor)}`), 'redirect dropped actor');
|
||||
assert.match(page.url(), /since=30d/, 'redirect dropped since');
|
||||
assert.match(page.url(), /#deep$/, 'redirect dropped the hash');
|
||||
const redirected = lastRequest();
|
||||
assert.ok(redirected, 'the redirect issued no /api/audit request');
|
||||
assert.equal(redirected.searchParams.get('kind'), 'container-runtime-policies');
|
||||
assert.equal(redirected.searchParams.get('actor'), actor);
|
||||
|
||||
// The feed must be readable before anything below can mean anything. A 500
|
||||
// here is the whole page: say what the server answered instead of timing out.
|
||||
const probe = await page.evaluate(async () => {
|
||||
const response = await fetch('/api/audit?limit=1');
|
||||
return { status: response.status, body: await response.text() };
|
||||
});
|
||||
if (probe.status !== 200) {
|
||||
await shot('unavailable');
|
||||
fail(`GET /api/audit returned ${probe.status}: ${probe.body.slice(0, 200)}`);
|
||||
return;
|
||||
}
|
||||
|
||||
const kindChip = page.locator('ul[aria-label="Active filters"] li', { hasText: 'Kind:' });
|
||||
await kindChip.first().waitFor({ timeout: 8000 });
|
||||
assert.match(await kindChip.first().innerText(), /Container runtime policies/, 'kind chip missing its catalog title');
|
||||
assert.match(
|
||||
await page.locator('ul[aria-label="Active filters"] li', { hasText: 'Actor:' }).first().innerText(),
|
||||
new RegExp(actor),
|
||||
'actor chip missing',
|
||||
);
|
||||
await shot('redirect');
|
||||
await go('/settings/audit');
|
||||
await page.locator('[data-row-key]').first().waitFor();
|
||||
await shot('desktop');
|
||||
};
|
||||
@@ -0,0 +1,148 @@
|
||||
// Audit retention reads the stored resource, the live tail prepends new rows
|
||||
// without a reload and holds them behind a button while scrolled away, and a
|
||||
// retention save persists. The save stays last: it must not stop the rest.
|
||||
const assert = require('node:assert/strict');
|
||||
const { trackAudit } = require('./helpers/audit-helpers.cjs');
|
||||
|
||||
module.exports = async ({ page, go, shot, fail, settled }) => {
|
||||
const { loadCount } = trackAudit(page);
|
||||
|
||||
await go('/settings/audit');
|
||||
const resourceResponse = await page.request.get(`${new URL(page.url()).origin}/api/control-plane/resources/audit-retention`);
|
||||
assert.equal(resourceResponse.status(), 200, 'audit-retention resource is unreadable');
|
||||
const retentionResource = await resourceResponse.json();
|
||||
const retentionHours = (value) => {
|
||||
const match = String(value || '').match(/^(\d+)h$/);
|
||||
return match ? Number(match[1]) : 0;
|
||||
};
|
||||
const originalHours = retentionHours(retentionResource.resource.value.runtime_event_retention);
|
||||
|
||||
const openRetention = async () => {
|
||||
await page.getByRole('button', { name: 'Audit actions', exact: true }).click();
|
||||
await page.getByRole('menuitem', { name: 'Retention', exact: true }).click();
|
||||
const dialog = page.getByRole('dialog');
|
||||
await dialog.waitFor();
|
||||
await dialog.getByLabel('Runtime event retention (hours)', { exact: true }).waitFor();
|
||||
return dialog;
|
||||
};
|
||||
|
||||
let retentionDialog = await openRetention();
|
||||
assert.match(await retentionDialog.innerText(), /Audit retention/);
|
||||
assert.equal(
|
||||
await retentionDialog.getByLabel('Runtime event retention (hours)', { exact: true }).inputValue(),
|
||||
String(originalHours),
|
||||
'retention dialog did not read the stored resource',
|
||||
);
|
||||
await retentionDialog.getByRole('button', { name: 'Cancel', exact: true }).click();
|
||||
await retentionDialog.waitFor({ state: 'hidden', timeout: 5000 }).catch(() => {});
|
||||
assert.ok(!(await retentionDialog.isVisible().catch(() => false)), 'Cancel left the retention dialog open');
|
||||
|
||||
// ---- live tail -----------------------------------------------------------
|
||||
await go('/settings/audit');
|
||||
await page.evaluate(() => window.scrollTo(0, 0));
|
||||
await settled();
|
||||
const topKey = await page.locator('[data-row-key]').first().getAttribute('data-row-key');
|
||||
const loadsBeforeWrite = loadCount();
|
||||
let resource = await (await page.request.get(`${new URL(page.url()).origin}/api/control-plane/resources/audit-retention`)).json();
|
||||
const toggleRetention = async () => {
|
||||
const current = retentionHours(resource.resource.value.runtime_event_retention);
|
||||
const next = current === 2160 ? 2161 : 2160;
|
||||
const written = await page.request.post(
|
||||
`${new URL(page.url()).origin}/api/control-plane/resources/audit-retention/commands/replace`,
|
||||
{
|
||||
headers: { 'X-Archie-CSRF': '1', Origin: new URL(page.url()).origin },
|
||||
data: { value: { runtime_event_retention: `${next}h` }, expected_version: resource.resource.version },
|
||||
},
|
||||
);
|
||||
const body = await written.text();
|
||||
assert.equal(written.status(), 200, `retention write answered ${written.status()}: ${body.slice(0, 200)}`);
|
||||
resource = JSON.parse(body);
|
||||
};
|
||||
|
||||
await toggleRetention();
|
||||
let liveKey = null;
|
||||
for (let attempt = 0; attempt < 40; attempt += 1) {
|
||||
const key = await page.locator('[data-row-key]').first().getAttribute('data-row-key');
|
||||
if (key !== topKey) {
|
||||
liveKey = key;
|
||||
break;
|
||||
}
|
||||
await page.waitForTimeout(300);
|
||||
}
|
||||
if (!liveKey) {
|
||||
fail('live tail did not show a new row within 12s of a write');
|
||||
await shot('live');
|
||||
} else {
|
||||
assert.equal(loadCount(), loadsBeforeWrite, 'the live row required a page reload');
|
||||
assert.match(
|
||||
await page.locator(`[data-row-key="${liveKey}"]`).innerText(),
|
||||
/audit-retention/i,
|
||||
'the live row is not the write that caused it',
|
||||
);
|
||||
await shot('live');
|
||||
}
|
||||
|
||||
// Scrolled away, a new event is held behind a button instead of jumping the
|
||||
// list under the operator.
|
||||
await page.evaluate(() => window.scrollTo(0, 900));
|
||||
await settled();
|
||||
const scrolledTo = await page.evaluate(() => window.scrollY);
|
||||
assert.ok(scrolledTo > 80, 'the feed is too short to test held new events');
|
||||
const heldTop = await page.locator('[data-row-key]').first().getAttribute('data-row-key');
|
||||
await toggleRetention();
|
||||
const heldButton = page.getByRole('button', { name: /^\d+ new events?$/ }).first();
|
||||
let held = false;
|
||||
for (let attempt = 0; attempt < 40; attempt += 1) {
|
||||
if ((await heldButton.count()) && (await heldButton.isVisible().catch(() => false))) {
|
||||
held = true;
|
||||
break;
|
||||
}
|
||||
await page.waitForTimeout(300);
|
||||
}
|
||||
if (!held) {
|
||||
fail('held new events did not appear while scrolled away');
|
||||
await shot('held');
|
||||
} else {
|
||||
assert.equal(
|
||||
await page.locator('[data-row-key]').first().getAttribute('data-row-key'),
|
||||
heldTop,
|
||||
'a new row was prepended while the operator was scrolled away',
|
||||
);
|
||||
await shot('held');
|
||||
await heldButton.click();
|
||||
await page.waitForFunction(
|
||||
(top) => document.querySelector('[data-row-key]')?.getAttribute('data-row-key') !== top,
|
||||
heldTop,
|
||||
{ timeout: 8000 },
|
||||
).catch(() => {});
|
||||
await settled();
|
||||
assert.notEqual(
|
||||
await page.locator('[data-row-key]').first().getAttribute('data-row-key'),
|
||||
heldTop,
|
||||
'showing held events did not prepend the new row',
|
||||
);
|
||||
assert.ok((await page.evaluate(() => window.scrollY)) < 10, 'showing held events did not return to the top');
|
||||
}
|
||||
|
||||
// ---- retention: save -----------------------------------------------------
|
||||
// Kept last: the page error this currently raises must not stop the rest.
|
||||
await go('/settings/audit');
|
||||
retentionDialog = await openRetention();
|
||||
const hoursField = retentionDialog.getByLabel('Runtime event retention (hours)', { exact: true });
|
||||
const beforeSave = await (await page.request.get(`${new URL(page.url()).origin}/api/control-plane/resources/audit-retention`)).json();
|
||||
const beforeHours = retentionHours(beforeSave.resource.value.runtime_event_retention);
|
||||
const wanted = beforeHours === 2160 ? 2161 : 2160;
|
||||
await hoursField.fill(String(wanted));
|
||||
await retentionDialog.getByRole('button', { name: 'Save', exact: true }).click();
|
||||
await retentionDialog.waitFor({ state: 'hidden', timeout: 5000 }).catch(() => {});
|
||||
const stillOpen = await retentionDialog.isVisible().catch(() => false);
|
||||
const afterSave = await (await page.request.get(`${new URL(page.url()).origin}/api/control-plane/resources/audit-retention`)).json();
|
||||
const afterHours = retentionHours(afterSave.resource.value.runtime_event_retention);
|
||||
if (stillOpen || afterHours !== wanted) {
|
||||
await shot('retention-save');
|
||||
fail(
|
||||
`retention save did not complete: dialog ${stillOpen ? 'stayed open' : 'closed'}, ` +
|
||||
`stored retention is ${afterHours}h (wanted ${wanted}h), URL ${page.url()}`,
|
||||
);
|
||||
}
|
||||
};
|
||||
@@ -0,0 +1,69 @@
|
||||
// A revertable change set expands to its Before/After diff, offers one Revert
|
||||
// action whose dialog cancels cleanly, and toggles from the keyboard.
|
||||
const assert = require('node:assert/strict');
|
||||
const { DEFAULT_WINDOW_MS, trackAudit } = require('./helpers/audit-helpers.cjs');
|
||||
|
||||
module.exports = async ({ page, go, shot, fail, settled }) => {
|
||||
trackAudit(page);
|
||||
|
||||
await go('/settings/audit');
|
||||
await page.locator('[data-row-key]').first().waitFor();
|
||||
const from = new Date(Date.now() - DEFAULT_WINDOW_MS).toISOString();
|
||||
// The feed page size is 100: only rows in this response are rendered.
|
||||
const feed = await page.evaluate(
|
||||
async (iso) => (await fetch(`/api/audit?limit=100&from=${encodeURIComponent(iso)}`)).json(),
|
||||
from,
|
||||
);
|
||||
const records = feed.records || [];
|
||||
assert.ok(records.length, 'the seeded feed is empty');
|
||||
|
||||
const changeSet = records.find(
|
||||
(row) =>
|
||||
row.target?.kind === 'resource' &&
|
||||
(row.detail?.fields || []).length > 0 &&
|
||||
(row.detail?.version ?? 0) > 1,
|
||||
);
|
||||
if (!changeSet) {
|
||||
fail('no revertable settings change set in the feed; cannot exercise the diff or Revert');
|
||||
return;
|
||||
}
|
||||
|
||||
const row = page.locator(`[data-row-key="${changeSet.id}"]`);
|
||||
await row.waitFor();
|
||||
await row.scrollIntoViewIfNeeded();
|
||||
await row.locator('td').first().click();
|
||||
assert.equal(await row.getAttribute('aria-expanded'), 'true', 'click did not expand the change set');
|
||||
const detail = row.locator('xpath=following-sibling::tr[1]');
|
||||
await detail.getByText('Before:', { exact: false }).first().waitFor();
|
||||
assert.match(await detail.innerText(), /After:/, 'expanded change set has no after value');
|
||||
assert.equal(
|
||||
await detail.getByRole('button', { name: /^Revert \S/ }).count(),
|
||||
0,
|
||||
'the audit diff offers a per-field revert',
|
||||
);
|
||||
const revertAll = page.getByRole('button', { name: 'Revert', exact: true });
|
||||
assert.ok((await revertAll.count()) > 0, 'no Revert action on a revertable change set');
|
||||
await shot('change-set');
|
||||
|
||||
await revertAll.first().click();
|
||||
const dialog = page.getByRole('alertdialog');
|
||||
await dialog.waitFor();
|
||||
assert.match(await dialog.innerText(), /Revert this change\?/);
|
||||
assert.match(await dialog.innerText(), /previous version opens for review/i);
|
||||
await dialog.getByRole('button', { name: 'Cancel', exact: true }).click();
|
||||
await dialog.waitFor({ state: 'hidden', timeout: 5000 }).catch(() => {});
|
||||
assert.ok(!(await dialog.isVisible().catch(() => false)), 'Cancel left the revert dialog open');
|
||||
assert.equal(
|
||||
await page.getByRole('region', { name: 'Unsaved changes' }).count(),
|
||||
0,
|
||||
'Cancel touched the save bar',
|
||||
);
|
||||
|
||||
// Enter is the keyboard equivalent of clicking the row.
|
||||
await row.click();
|
||||
assert.equal(await row.getAttribute('aria-expanded'), 'false', 'the row did not collapse');
|
||||
await row.focus();
|
||||
await page.keyboard.press('Enter');
|
||||
await settled();
|
||||
assert.equal(await row.getAttribute('aria-expanded'), 'true', 'Enter did not expand the change set');
|
||||
};
|
||||
@@ -0,0 +1,35 @@
|
||||
// A runtime event with a task target expands to its metadata and links to
|
||||
// the task. The event comes from the seed; without it there is nothing to
|
||||
// exercise, so the check skips instead of failing.
|
||||
const assert = require('node:assert/strict');
|
||||
const { DEFAULT_WINDOW_MS, trackAudit } = require('./helpers/audit-helpers.cjs');
|
||||
|
||||
module.exports = async ({ page, go, skip }) => {
|
||||
trackAudit(page);
|
||||
|
||||
await go('/settings/audit');
|
||||
await page.locator('[data-row-key]').first().waitFor();
|
||||
const from = new Date(Date.now() - DEFAULT_WINDOW_MS).toISOString();
|
||||
// The feed page size is 100: only rows in this response are rendered.
|
||||
const feed = await page.evaluate(
|
||||
async (iso) => (await fetch(`/api/audit?limit=100&from=${encodeURIComponent(iso)}`)).json(),
|
||||
from,
|
||||
);
|
||||
const records = feed.records || [];
|
||||
const runtime = records.find((row) => row.category === 'runtime' && row.target?.kind === 'task' && row.target?.id);
|
||||
if (!runtime) {
|
||||
skip('no runtime event with a task target in the feed');
|
||||
}
|
||||
|
||||
const row = page.locator(`[data-row-key="${runtime.id}"]`);
|
||||
await row.waitFor();
|
||||
await row.scrollIntoViewIfNeeded();
|
||||
await row.locator('td').first().click();
|
||||
assert.equal(await row.getAttribute('aria-expanded'), 'true', 'runtime row did not expand');
|
||||
const detail = row.locator('xpath=following-sibling::tr[1]');
|
||||
await detail.locator('dl dt', { hasText: 'Target' }).first().waitFor();
|
||||
assert.match(await detail.innerText(), /Table/, 'runtime metadata has no table row');
|
||||
const link = detail.getByRole('link', { name: runtime.target.label || runtime.target.id, exact: true });
|
||||
assert.equal(await link.count(), 1, 'runtime target is not a link');
|
||||
assert.equal(await link.getAttribute('href'), `/tasks/${runtime.target.id}`);
|
||||
};
|
||||
@@ -0,0 +1,50 @@
|
||||
// Audit saved views round-trip through the URL, and CSV/JSON exports carry
|
||||
// the resolved window and arrive as attachments.
|
||||
const assert = require('node:assert/strict');
|
||||
const { trackAudit, exportOnce } = require('./helpers/audit-helpers.cjs');
|
||||
|
||||
module.exports = async ({ page, go, shot, fail, settled }) => {
|
||||
const { waitForUrl } = trackAudit(page);
|
||||
|
||||
await go('/settings/audit');
|
||||
await page.getByRole('button', { name: 'Runtime', exact: true }).click();
|
||||
await waitForUrl((u) => u.searchParams.get('categories') === 'runtime');
|
||||
|
||||
const viewName = `e2e-view-${Date.now()}`;
|
||||
await page.getByRole('button', { name: 'Audit actions', exact: true }).click();
|
||||
await page.getByRole('menuitem', { name: 'Save view…', exact: true }).click();
|
||||
const saveDialog = page.getByRole('dialog');
|
||||
await saveDialog.waitFor();
|
||||
await saveDialog.getByLabel('Name', { exact: true }).fill(viewName);
|
||||
await saveDialog.getByRole('button', { name: 'Save', exact: true }).click();
|
||||
await settled();
|
||||
await saveDialog.waitFor({ state: 'hidden', timeout: 5000 }).catch(() => {});
|
||||
|
||||
const savedView = page.getByRole('combobox', { name: 'Saved view', exact: true });
|
||||
await savedView.waitFor();
|
||||
assert.equal((await savedView.innerText()).trim(), viewName);
|
||||
await shot('saved-view');
|
||||
|
||||
await page.getByRole('button', { name: 'Runtime', exact: true }).click();
|
||||
await waitForUrl((u) => u.searchParams.get('categories') === null);
|
||||
await savedView.click();
|
||||
await page.getByRole('option', { name: viewName, exact: true }).click();
|
||||
await waitForUrl((u) => u.searchParams.get('categories') === 'runtime');
|
||||
assert.equal((await savedView.innerText()).trim(), viewName, 'applying the saved view did not restore it');
|
||||
|
||||
await page.getByRole('button', { name: 'Audit actions', exact: true }).click();
|
||||
await page.getByRole('menuitem', { name: 'Delete view', exact: true }).click();
|
||||
await savedView.waitFor({ state: 'detached', timeout: 5000 }).catch(() => {});
|
||||
assert.equal(await savedView.count(), 0, 'deleting the saved view left it in the picker');
|
||||
|
||||
await go('/settings/audit');
|
||||
await page.getByRole('button', { name: 'Export', exact: true }).click();
|
||||
await page.getByRole('menuitem', { name: 'CSV', exact: true }).waitFor();
|
||||
await shot('export');
|
||||
await page.keyboard.press('Escape');
|
||||
|
||||
const csvBody = await exportOnce(page, fail, 'CSV', 'csv');
|
||||
if (csvBody !== null) assert.ok(csvBody.split('\n')[0].trim().length > 0, 'CSV export is empty');
|
||||
const jsonBody = await exportOnce(page, fail, 'JSON', 'json');
|
||||
if (jsonBody !== null) assert.ok(jsonBody.trimStart().startsWith('['), 'JSON export is not an array');
|
||||
};
|
||||
@@ -1,536 +0,0 @@
|
||||
// The Audit page: deep-link redirect, server-evaluated filters, change-set and
|
||||
// runtime detail, revert, saved views, export, live tail and phone width.
|
||||
//
|
||||
// Rows come from the seeded stack's real feed. When a precondition is missing
|
||||
// (no revertable change set, no runtime task event) the check says so instead
|
||||
// of passing silently.
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
|
||||
const DEFAULT_WINDOW_MS = 30 * 24 * 60 * 60 * 1000;
|
||||
|
||||
function isAudit(u) {
|
||||
return u.pathname === '/api/audit';
|
||||
}
|
||||
|
||||
module.exports = async ({ page, go, shot, fail }) => {
|
||||
page.setDefaultTimeout(10000);
|
||||
const auditRequests = [];
|
||||
page.on('request', (request) => {
|
||||
try {
|
||||
const url = new URL(request.url());
|
||||
if (isAudit(url) || url.pathname === '/api/audit/export') auditRequests.push(url);
|
||||
} catch {
|
||||
// A non-URL request cannot be an audit call.
|
||||
}
|
||||
});
|
||||
let loads = 0;
|
||||
page.on('load', () => (loads += 1));
|
||||
|
||||
const requestMatching = async (action, predicate, timeout = 6000) => {
|
||||
const waiter = page
|
||||
.waitForRequest(
|
||||
(request) => {
|
||||
try {
|
||||
const url = new URL(request.url());
|
||||
return (isAudit(url) || url.pathname === '/api/audit/export') && predicate(url);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
},
|
||||
{ timeout },
|
||||
)
|
||||
.catch(() => null);
|
||||
await action();
|
||||
return waiter;
|
||||
};
|
||||
|
||||
const waitForUrl = (predicate, timeout = 8000) =>
|
||||
page.waitForURL((url) => predicate(new URL(url)), { timeout });
|
||||
|
||||
const lastRequest = () => auditRequests[auditRequests.length - 1] || null;
|
||||
|
||||
// ---- deep-link redirect ----------------------------------------------------
|
||||
await go('/settings/audit');
|
||||
const status = await page.evaluate(async () => (await fetch('/api/auth/status')).json());
|
||||
const actor = status?.user?.id;
|
||||
assert.ok(actor, 'signed-in identity has no id');
|
||||
|
||||
const deepLink = `/settings/history?section=container-runtime-policies&actor=${encodeURIComponent(actor)}&since=30d#deep`;
|
||||
await go(deepLink);
|
||||
assert.match(page.url(), /\/settings\/audit\?/, 'history did not redirect to the audit route');
|
||||
assert.match(page.url(), /section=container-runtime-policies/, 'redirect dropped section');
|
||||
assert.match(page.url(), new RegExp(`actor=${encodeURIComponent(actor)}`), 'redirect dropped actor');
|
||||
assert.match(page.url(), /since=30d/, 'redirect dropped since');
|
||||
assert.match(page.url(), /#deep$/, 'redirect dropped the hash');
|
||||
const redirected = lastRequest();
|
||||
assert.ok(redirected, 'the redirect issued no /api/audit request');
|
||||
assert.equal(redirected.searchParams.get('kind'), 'container-runtime-policies');
|
||||
assert.equal(redirected.searchParams.get('actor'), actor);
|
||||
|
||||
// The feed must be readable before anything below can mean anything. A 500
|
||||
// here is the whole page: say what the server answered instead of timing out.
|
||||
const probe = await page.evaluate(async () => {
|
||||
const response = await fetch('/api/audit?limit=1');
|
||||
return { status: response.status, body: await response.text() };
|
||||
});
|
||||
if (probe.status !== 200) {
|
||||
await shot('unavailable');
|
||||
fail(`GET /api/audit returned ${probe.status}: ${probe.body.slice(0, 200)}`);
|
||||
return;
|
||||
}
|
||||
|
||||
const kindChip = page.locator('ul[aria-label="Active filters"] li', { hasText: 'Kind:' });
|
||||
await kindChip.first().waitFor({ timeout: 8000 });
|
||||
assert.match(await kindChip.first().innerText(), /Container runtime policies/, 'kind chip missing its catalog title');
|
||||
assert.match(
|
||||
await page.locator('ul[aria-label="Active filters"] li', { hasText: 'Actor:' }).first().innerText(),
|
||||
new RegExp(actor),
|
||||
'actor chip missing',
|
||||
);
|
||||
await shot('redirect');
|
||||
await go('/settings/audit');
|
||||
await page.locator('[data-row-key]').first().waitFor();
|
||||
await shot('desktop');
|
||||
|
||||
// ---- filters ---------------------------------------------------------------
|
||||
await go('/settings/audit');
|
||||
const search = page.getByLabel('Search events', { exact: true });
|
||||
let request = await requestMatching(
|
||||
() => search.fill('repository'),
|
||||
(u) => u.searchParams.get('q') === 'repository',
|
||||
);
|
||||
assert.ok(request, 'search issued no matching /api/audit request');
|
||||
await waitForUrl((u) => u.searchParams.get('q') === 'repository');
|
||||
assert.equal((await search.inputValue()), 'repository');
|
||||
|
||||
await go('/settings/audit');
|
||||
request = await requestMatching(
|
||||
() => page.getByRole('button', { name: 'Runtime', exact: true }).click(),
|
||||
(u) => u.searchParams.get('categories') === 'runtime',
|
||||
);
|
||||
assert.ok(request, 'category filter issued no matching /api/audit request');
|
||||
await waitForUrl((u) => u.searchParams.get('categories') === 'runtime');
|
||||
assert.equal(
|
||||
await page.getByRole('button', { name: 'Runtime', exact: true }).getAttribute('aria-pressed'),
|
||||
'true',
|
||||
);
|
||||
|
||||
// Time: Any time drops the window from the request rather than guessing one.
|
||||
await page.getByRole('combobox', { name: 'Time range', exact: true }).click();
|
||||
request = await requestMatching(
|
||||
() => page.getByRole('option', { name: 'Any time', exact: true }).click(),
|
||||
(u) => u.searchParams.get('from') === null && u.searchParams.get('categories') === 'runtime',
|
||||
);
|
||||
assert.ok(request, 'Any time issued no matching /api/audit request');
|
||||
await waitForUrl((u) => u.searchParams.get('since') === 'any');
|
||||
assert.equal(await page.getByRole('combobox', { name: 'Time range', exact: true }).innerText(), 'Any time');
|
||||
|
||||
// Back/forward restore the controls the URL describes.
|
||||
const backRequest = page.waitForRequest(
|
||||
(r) => {
|
||||
const u = new URL(r.url());
|
||||
return u.pathname === '/api/audit' && u.searchParams.get('categories') === 'runtime' && !!u.searchParams.get('from');
|
||||
},
|
||||
{ timeout: 8000 },
|
||||
);
|
||||
await page.goBack();
|
||||
await backRequest;
|
||||
await waitForUrl((u) => u.searchParams.get('categories') === 'runtime' && u.searchParams.get('since') === null);
|
||||
assert.equal(await page.getByRole('combobox', { name: 'Time range', exact: true }).innerText(), 'Last 30 days');
|
||||
assert.equal(
|
||||
await page.getByRole('button', { name: 'Runtime', exact: true }).getAttribute('aria-pressed'),
|
||||
'true',
|
||||
'back did not restore the category toggle',
|
||||
);
|
||||
const forwardRequest = page.waitForRequest(
|
||||
(r) => {
|
||||
const u = new URL(r.url());
|
||||
return u.pathname === '/api/audit' && u.searchParams.get('categories') === 'runtime' && u.searchParams.get('from') === null;
|
||||
},
|
||||
{ timeout: 8000 },
|
||||
);
|
||||
await page.goForward();
|
||||
await forwardRequest;
|
||||
await waitForUrl((u) => u.searchParams.get('since') === 'any');
|
||||
assert.equal(await page.getByRole('combobox', { name: 'Time range', exact: true }).innerText(), 'Any time');
|
||||
|
||||
// Denied reaches the server through the same URL contract as the other
|
||||
// filters. It must both update the URL and issue a request.
|
||||
await go('/settings/audit');
|
||||
const denied = page.getByRole('button', { name: 'Denied', exact: true });
|
||||
request = await requestMatching(
|
||||
() => denied.click(),
|
||||
(u) => u.searchParams.get('outcome') === 'denied',
|
||||
3000,
|
||||
);
|
||||
const deniedUrl = new URL(page.url());
|
||||
if (!request || deniedUrl.searchParams.get('outcome') !== 'denied') {
|
||||
await shot('denied');
|
||||
fail(
|
||||
`Denied filter did not reach the server: URL ${page.url()} has outcome=${deniedUrl.searchParams.get('outcome') ?? 'absent'}, ` +
|
||||
`aria-pressed=${await denied.getAttribute('aria-pressed')}, no /api/audit request with outcome=denied`,
|
||||
);
|
||||
} else {
|
||||
assert.equal(await denied.getAttribute('aria-pressed'), 'true');
|
||||
}
|
||||
|
||||
// Advanced actor filter: server-evaluated, in the URL, shown as a chip.
|
||||
await go('/settings/audit');
|
||||
await page.getByRole('button', { name: 'Filters', exact: true }).click();
|
||||
const actorInput = page.getByLabel('Actor', { exact: true });
|
||||
request = await requestMatching(
|
||||
async () => {
|
||||
await actorInput.fill(actor);
|
||||
await actorInput.press('Enter');
|
||||
},
|
||||
(u) => u.searchParams.get('actor') === actor,
|
||||
);
|
||||
assert.ok(request, 'advanced actor filter issued no matching /api/audit request');
|
||||
await waitForUrl((u) => u.searchParams.get('actor') === actor);
|
||||
assert.match(
|
||||
await page.locator('ul[aria-label="Active filters"] li', { hasText: 'Actor:' }).first().innerText(),
|
||||
new RegExp(actor),
|
||||
);
|
||||
await shot('filters');
|
||||
await page.keyboard.press('Escape');
|
||||
|
||||
// ---- change set, revert dialog, keyboard --------------------------------
|
||||
await go('/settings/audit');
|
||||
await page.locator('[data-row-key]').first().waitFor();
|
||||
const from = new Date(Date.now() - DEFAULT_WINDOW_MS).toISOString();
|
||||
// The feed page size is 100: only rows in this response are rendered.
|
||||
const feed = await page.evaluate(
|
||||
async (iso) => (await fetch(`/api/audit?limit=100&from=${encodeURIComponent(iso)}`)).json(),
|
||||
from,
|
||||
);
|
||||
const records = feed.records || [];
|
||||
assert.ok(records.length, 'the seeded feed is empty');
|
||||
|
||||
const changeSet = records.find(
|
||||
(row) =>
|
||||
row.target?.kind === 'resource' &&
|
||||
(row.detail?.fields || []).length > 0 &&
|
||||
(row.detail?.version ?? 0) > 1,
|
||||
);
|
||||
if (!changeSet) {
|
||||
fail('no revertable settings change set in the feed; cannot exercise the diff or Revert');
|
||||
} else {
|
||||
const row = page.locator(`[data-row-key="${changeSet.id}"]`);
|
||||
await row.waitFor();
|
||||
await row.scrollIntoViewIfNeeded();
|
||||
await row.locator('td').first().click();
|
||||
assert.equal(await row.getAttribute('aria-expanded'), 'true', 'click did not expand the change set');
|
||||
const detail = row.locator('xpath=following-sibling::tr[1]');
|
||||
await detail.getByText('Before:', { exact: false }).first().waitFor();
|
||||
assert.match(await detail.innerText(), /After:/, 'expanded change set has no after value');
|
||||
assert.equal(
|
||||
await detail.getByRole('button', { name: /^Revert \S/ }).count(),
|
||||
0,
|
||||
'the audit diff offers a per-field revert',
|
||||
);
|
||||
const revertAll = page.getByRole('button', { name: 'Revert', exact: true });
|
||||
assert.ok((await revertAll.count()) > 0, 'no Revert action on a revertable change set');
|
||||
await shot('change-set');
|
||||
|
||||
await revertAll.first().click();
|
||||
const dialog = page.getByRole('alertdialog');
|
||||
await dialog.waitFor();
|
||||
assert.match(await dialog.innerText(), /Revert this change\?/);
|
||||
assert.match(await dialog.innerText(), /previous version opens for review/i);
|
||||
await dialog.getByRole('button', { name: 'Cancel', exact: true }).click();
|
||||
await dialog.waitFor({ state: 'hidden', timeout: 5000 }).catch(() => {});
|
||||
assert.ok(!(await dialog.isVisible().catch(() => false)), 'Cancel left the revert dialog open');
|
||||
assert.equal(
|
||||
await page.getByRole('region', { name: 'Unsaved changes' }).count(),
|
||||
0,
|
||||
'Cancel touched the save bar',
|
||||
);
|
||||
|
||||
// Enter is the keyboard equivalent of clicking the row.
|
||||
await row.click();
|
||||
assert.equal(await row.getAttribute('aria-expanded'), 'false', 'the row did not collapse');
|
||||
await row.focus();
|
||||
await page.keyboard.press('Enter');
|
||||
await page.waitForTimeout(200);
|
||||
assert.equal(await row.getAttribute('aria-expanded'), 'true', 'Enter did not expand the change set');
|
||||
}
|
||||
|
||||
// ---- runtime event detail ------------------------------------------------
|
||||
await go('/settings/audit');
|
||||
const runtime = records.find((row) => row.category === 'runtime' && row.target?.kind === 'task' && row.target?.id);
|
||||
if (!runtime) {
|
||||
fail('no runtime event with a task target in the feed; cannot exercise runtime detail');
|
||||
} else {
|
||||
const row = page.locator(`[data-row-key="${runtime.id}"]`);
|
||||
await row.waitFor();
|
||||
await row.scrollIntoViewIfNeeded();
|
||||
await row.locator('td').first().click();
|
||||
assert.equal(await row.getAttribute('aria-expanded'), 'true', 'runtime row did not expand');
|
||||
const detail = row.locator('xpath=following-sibling::tr[1]');
|
||||
await detail.locator('dl dt', { hasText: 'Target' }).first().waitFor();
|
||||
assert.match(await detail.innerText(), /Table/, 'runtime metadata has no table row');
|
||||
const link = detail.getByRole('link', { name: runtime.target.label || runtime.target.id, exact: true });
|
||||
assert.equal(await link.count(), 1, 'runtime target is not a link');
|
||||
assert.equal(await link.getAttribute('href'), `/tasks/${runtime.target.id}`);
|
||||
}
|
||||
|
||||
// ---- saved views ---------------------------------------------------------
|
||||
await go('/settings/audit');
|
||||
await page.getByRole('button', { name: 'Runtime', exact: true }).click();
|
||||
await waitForUrl((u) => u.searchParams.get('categories') === 'runtime');
|
||||
|
||||
const viewName = `e2e-view-${Date.now()}`;
|
||||
await page.getByRole('button', { name: 'Audit actions', exact: true }).click();
|
||||
await page.getByRole('menuitem', { name: 'Save view…', exact: true }).click();
|
||||
const saveDialog = page.getByRole('dialog');
|
||||
await saveDialog.waitFor();
|
||||
await saveDialog.getByLabel('Name', { exact: true }).fill(viewName);
|
||||
await saveDialog.getByRole('button', { name: 'Save', exact: true }).click();
|
||||
await page.waitForTimeout(400);
|
||||
|
||||
const savedView = page.getByRole('combobox', { name: 'Saved view', exact: true });
|
||||
await savedView.waitFor();
|
||||
assert.equal((await savedView.innerText()).trim(), viewName);
|
||||
await shot('saved-view');
|
||||
|
||||
await page.getByRole('button', { name: 'Runtime', exact: true }).click();
|
||||
await waitForUrl((u) => u.searchParams.get('categories') === null);
|
||||
await savedView.click();
|
||||
await page.getByRole('option', { name: viewName, exact: true }).click();
|
||||
await waitForUrl((u) => u.searchParams.get('categories') === 'runtime');
|
||||
assert.equal((await savedView.innerText()).trim(), viewName, 'applying the saved view did not restore it');
|
||||
|
||||
await page.getByRole('button', { name: 'Audit actions', exact: true }).click();
|
||||
await page.getByRole('menuitem', { name: 'Delete view', exact: true }).click();
|
||||
await page.waitForTimeout(300);
|
||||
assert.equal(await savedView.count(), 0, 'deleting the saved view left it in the picker');
|
||||
|
||||
// ---- export --------------------------------------------------------------
|
||||
// Export resolves its window at click time and opens a tab. A successful
|
||||
// export arrives as an attachment download; a failed one stays visible in
|
||||
// that tab, so the check accepts either and reads the response from it.
|
||||
await go('/settings/audit');
|
||||
const exportOnce = async (label, extension) => {
|
||||
await page.getByRole('button', { name: 'Export', exact: true }).click();
|
||||
const item = page.getByRole('menuitem', { name: label, exact: true });
|
||||
await item.waitFor();
|
||||
const downloadP = page.waitForEvent('download', { timeout: 15000 }).then((download) => ({ download })).catch(() => null);
|
||||
const popupP = page.waitForEvent('popup', { timeout: 15000 }).then((popup) => ({ popup })).catch(() => null);
|
||||
await item.click();
|
||||
const settled = await Promise.race([downloadP, popupP]);
|
||||
if (!settled) {
|
||||
fail(`${label} export neither downloaded nor opened a tab`);
|
||||
return null;
|
||||
}
|
||||
if (settled.popup) {
|
||||
await settled.popup.waitForLoadState('load').catch(() => {});
|
||||
const url = new URL(settled.popup.url());
|
||||
assert.equal(url.pathname, '/api/audit/export', `${label} opened ${url.href}`);
|
||||
assert.equal(url.searchParams.get('format'), extension);
|
||||
assert.ok(url.searchParams.get('from'), `${label} export has no explicit start`);
|
||||
assert.ok(url.searchParams.get('to'), `${label} export has no explicit end`);
|
||||
const response = await page.request.get(url.href);
|
||||
const body = await response.text();
|
||||
if (response.status() !== 200) {
|
||||
fail(`${label} export failed (${response.status()}): ${body.slice(0, 200)}`);
|
||||
return null;
|
||||
}
|
||||
if (!new RegExp(`attachment;.*\\.${extension}`).test(response.headers()['content-disposition'] || '')) {
|
||||
fail(`${label} export opened a tab without an attachment filename`);
|
||||
return null;
|
||||
}
|
||||
return body;
|
||||
}
|
||||
if (!new RegExp(`\\.${extension}$`).test(settled.download.suggestedFilename())) {
|
||||
fail(`${label} export did not download a .${extension} file`);
|
||||
return null;
|
||||
}
|
||||
const failure = await settled.download.failure();
|
||||
if (failure) {
|
||||
fail(`${label} download failed: ${failure}`);
|
||||
return null;
|
||||
}
|
||||
const saved = await settled.download.path();
|
||||
return saved ? fs.readFileSync(saved, 'utf8') : '';
|
||||
};
|
||||
|
||||
await page.getByRole('button', { name: 'Export', exact: true }).click();
|
||||
await page.getByRole('menuitem', { name: 'CSV', exact: true }).waitFor();
|
||||
await shot('export');
|
||||
await page.keyboard.press('Escape');
|
||||
|
||||
const csvBody = await exportOnce('CSV', 'csv');
|
||||
if (csvBody !== null) assert.ok(csvBody.split('\n')[0].trim().length > 0, 'CSV export is empty');
|
||||
const jsonBody = await exportOnce('JSON', 'json');
|
||||
if (jsonBody !== null) assert.ok(jsonBody.trimStart().startsWith('['), 'JSON export is not an array');
|
||||
|
||||
// ---- retention: read and cancel -----------------------------------------
|
||||
await go('/settings/audit');
|
||||
const resourceResponse = await page.request.get(`${new URL(page.url()).origin}/api/control-plane/resources/audit-retention`);
|
||||
assert.equal(resourceResponse.status(), 200, 'audit-retention resource is unreadable');
|
||||
const retentionResource = await resourceResponse.json();
|
||||
const retentionHours = (value) => {
|
||||
const match = String(value || '').match(/^(\d+)h$/);
|
||||
return match ? Number(match[1]) : 0;
|
||||
};
|
||||
const originalHours = retentionHours(retentionResource.resource.value.runtime_event_retention);
|
||||
|
||||
const openRetention = async () => {
|
||||
await page.getByRole('button', { name: 'Audit actions', exact: true }).click();
|
||||
await page.getByRole('menuitem', { name: 'Retention', exact: true }).click();
|
||||
const dialog = page.getByRole('dialog');
|
||||
await dialog.waitFor();
|
||||
await dialog.getByLabel('Runtime event retention (hours)', { exact: true }).waitFor();
|
||||
return dialog;
|
||||
};
|
||||
|
||||
let retentionDialog = await openRetention();
|
||||
assert.match(await retentionDialog.innerText(), /Audit retention/);
|
||||
assert.equal(
|
||||
await retentionDialog.getByLabel('Runtime event retention (hours)', { exact: true }).inputValue(),
|
||||
String(originalHours),
|
||||
'retention dialog did not read the stored resource',
|
||||
);
|
||||
await retentionDialog.getByRole('button', { name: 'Cancel', exact: true }).click();
|
||||
await retentionDialog.waitFor({ state: 'hidden', timeout: 5000 }).catch(() => {});
|
||||
assert.ok(!(await retentionDialog.isVisible().catch(() => false)), 'Cancel left the retention dialog open');
|
||||
|
||||
// ---- live tail -----------------------------------------------------------
|
||||
await go('/settings/audit');
|
||||
await page.evaluate(() => window.scrollTo(0, 0));
|
||||
await page.waitForTimeout(300);
|
||||
const topKey = await page.locator('[data-row-key]').first().getAttribute('data-row-key');
|
||||
const loadsBeforeWrite = loads;
|
||||
let resource = await (await page.request.get(`${new URL(page.url()).origin}/api/control-plane/resources/audit-retention`)).json();
|
||||
const toggleRetention = async () => {
|
||||
const current = retentionHours(resource.resource.value.runtime_event_retention);
|
||||
const next = current === 2160 ? 2161 : 2160;
|
||||
const written = await page.request.post(
|
||||
`${new URL(page.url()).origin}/api/control-plane/resources/audit-retention/commands/replace`,
|
||||
{
|
||||
headers: { 'X-Archie-CSRF': '1', Origin: new URL(page.url()).origin },
|
||||
data: { value: { runtime_event_retention: `${next}h` }, expected_version: resource.resource.version },
|
||||
},
|
||||
);
|
||||
const body = await written.text();
|
||||
assert.equal(written.status(), 200, `retention write answered ${written.status()}: ${body.slice(0, 200)}`);
|
||||
resource = JSON.parse(body);
|
||||
};
|
||||
|
||||
await toggleRetention();
|
||||
let liveKey = null;
|
||||
for (let attempt = 0; attempt < 40; attempt += 1) {
|
||||
const key = await page.locator('[data-row-key]').first().getAttribute('data-row-key');
|
||||
if (key !== topKey) {
|
||||
liveKey = key;
|
||||
break;
|
||||
}
|
||||
await page.waitForTimeout(300);
|
||||
}
|
||||
if (!liveKey) {
|
||||
fail('live tail did not show a new row within 12s of a write');
|
||||
await shot('live');
|
||||
} else {
|
||||
assert.equal(loads, loadsBeforeWrite, 'the live row required a page reload');
|
||||
assert.match(
|
||||
await page.locator(`[data-row-key="${liveKey}"]`).innerText(),
|
||||
/audit-retention/i,
|
||||
'the live row is not the write that caused it',
|
||||
);
|
||||
await shot('live');
|
||||
}
|
||||
|
||||
// Scrolled away, a new event is held behind a button instead of jumping the
|
||||
// list under the operator.
|
||||
await page.evaluate(() => window.scrollTo(0, 900));
|
||||
await page.waitForTimeout(300);
|
||||
const scrolledTo = await page.evaluate(() => window.scrollY);
|
||||
assert.ok(scrolledTo > 80, 'the feed is too short to test held new events');
|
||||
const heldTop = await page.locator('[data-row-key]').first().getAttribute('data-row-key');
|
||||
await toggleRetention();
|
||||
const heldButton = page.getByRole('button', { name: /^\d+ new events?$/ }).first();
|
||||
let held = false;
|
||||
for (let attempt = 0; attempt < 40; attempt += 1) {
|
||||
if ((await heldButton.count()) && (await heldButton.isVisible().catch(() => false))) {
|
||||
held = true;
|
||||
break;
|
||||
}
|
||||
await page.waitForTimeout(300);
|
||||
}
|
||||
if (!held) {
|
||||
fail('held new events did not appear while scrolled away');
|
||||
await shot('held');
|
||||
} else {
|
||||
assert.equal(
|
||||
await page.locator('[data-row-key]').first().getAttribute('data-row-key'),
|
||||
heldTop,
|
||||
'a new row was prepended while the operator was scrolled away',
|
||||
);
|
||||
await shot('held');
|
||||
await heldButton.click();
|
||||
await page.waitForTimeout(500);
|
||||
assert.notEqual(
|
||||
await page.locator('[data-row-key]').first().getAttribute('data-row-key'),
|
||||
heldTop,
|
||||
'showing held events did not prepend the new row',
|
||||
);
|
||||
assert.ok((await page.evaluate(() => window.scrollY)) < 10, 'showing held events did not return to the top');
|
||||
}
|
||||
|
||||
// ---- retention: save -----------------------------------------------------
|
||||
// Kept last: the page error this currently raises must not stop the rest.
|
||||
await go('/settings/audit');
|
||||
retentionDialog = await openRetention();
|
||||
const hoursField = retentionDialog.getByLabel('Runtime event retention (hours)', { exact: true });
|
||||
const beforeSave = await (await page.request.get(`${new URL(page.url()).origin}/api/control-plane/resources/audit-retention`)).json();
|
||||
const beforeHours = retentionHours(beforeSave.resource.value.runtime_event_retention);
|
||||
const wanted = beforeHours === 2160 ? 2161 : 2160;
|
||||
await hoursField.fill(String(wanted));
|
||||
await retentionDialog.getByRole('button', { name: 'Save', exact: true }).click();
|
||||
await retentionDialog.waitFor({ state: 'hidden', timeout: 5000 }).catch(() => {});
|
||||
const stillOpen = await retentionDialog.isVisible().catch(() => false);
|
||||
const afterSave = await (await page.request.get(`${new URL(page.url()).origin}/api/control-plane/resources/audit-retention`)).json();
|
||||
const afterHours = retentionHours(afterSave.resource.value.runtime_event_retention);
|
||||
if (stillOpen || afterHours !== wanted) {
|
||||
await shot('retention-save');
|
||||
fail(
|
||||
`retention save did not complete: dialog ${stillOpen ? 'stayed open' : 'closed'}, ` +
|
||||
`stored retention is ${afterHours}h (wanted ${wanted}h), URL ${page.url()}`,
|
||||
);
|
||||
}
|
||||
|
||||
// ---- phone width ---------------------------------------------------------
|
||||
await page.setViewportSize({ width: 390, height: 844 });
|
||||
await go('/settings/audit');
|
||||
const assertNoOverflow = async (label) => {
|
||||
const { scrollWidth, innerWidth } = await page.evaluate(() => ({
|
||||
scrollWidth: document.documentElement.scrollWidth,
|
||||
innerWidth,
|
||||
}));
|
||||
if (scrollWidth > innerWidth) {
|
||||
fail(`${label} overflows 390px: documentElement.scrollWidth=${scrollWidth} > innerWidth=${innerWidth}`);
|
||||
}
|
||||
};
|
||||
await assertNoOverflow('audit list');
|
||||
assert.equal(
|
||||
await page.locator('thead th', { hasText: 'Target' }).first().evaluate((el) => getComputedStyle(el).display),
|
||||
'none',
|
||||
'secondary columns are still rendered on the phone',
|
||||
);
|
||||
await shot('390-list');
|
||||
|
||||
const phoneRow = page.locator('[data-row-key]').first();
|
||||
await phoneRow.focus();
|
||||
await page.keyboard.press('Enter');
|
||||
await page.waitForTimeout(300);
|
||||
assert.equal(await phoneRow.getAttribute('aria-expanded'), 'true', 'phone row did not expand');
|
||||
await assertNoOverflow('expanded change set');
|
||||
await shot('390-expanded');
|
||||
|
||||
await page.getByRole('button', { name: 'Filters', exact: true }).click();
|
||||
await page.getByRole('dialog').first().waitFor();
|
||||
await page.waitForTimeout(300);
|
||||
await assertNoOverflow('advanced filters');
|
||||
await shot('390-filters');
|
||||
};
|
||||
@@ -0,0 +1,28 @@
|
||||
// The chat composer: IME confirmation never submits the draft, Shift+Enter
|
||||
// inserts a newline, and ordinary Enter submits and clears.
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
module.exports = async ({ page, go, shot }) => {
|
||||
await go('/events');
|
||||
const composer = page.getByRole('textbox', { name: 'Message Archie', exact: true });
|
||||
await composer.waitFor();
|
||||
await composer.fill('Composition in progress');
|
||||
const sent = [];
|
||||
await page.route('**/api/chat/stream', async route => {
|
||||
sent.push(route.request().postDataJSON().text);
|
||||
await route.fulfill({ contentType: 'text/event-stream', body: 'data: {"type":"done","text":"Test response"}\n\n' });
|
||||
});
|
||||
for (const event of [{ isComposing: true }, { keyCode: 229 }]) {
|
||||
await composer.dispatchEvent('keydown', { key: 'Enter', bubbles: true, cancelable: true, ...event });
|
||||
assert.equal(await composer.inputValue(), 'Composition in progress', 'IME confirmation submitted the draft');
|
||||
}
|
||||
assert.equal(sent.length, 0);
|
||||
await composer.press('Shift+Enter');
|
||||
assert.ok((await composer.inputValue()).includes('\n'), 'Shift+Enter no longer inserts a newline');
|
||||
await composer.fill('Ordinary Enter submits');
|
||||
const request = page.waitForRequest('**/api/chat/stream');
|
||||
await composer.press('Enter');
|
||||
assert.equal((await request).postDataJSON().text, 'Ordinary Enter submits');
|
||||
assert.equal(await composer.inputValue(), '');
|
||||
await shot('chat-composition');
|
||||
};
|
||||
@@ -0,0 +1,103 @@
|
||||
// Shared setup for the audit-* checks: feed request tracking, URL waits,
|
||||
// and the export download-or-popup accept. One behaviour per check module;
|
||||
// this file holds only what several of them need.
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
|
||||
const DEFAULT_WINDOW_MS = 30 * 24 * 60 * 60 * 1000;
|
||||
|
||||
function isAudit(u) {
|
||||
return u.pathname === '/api/audit';
|
||||
}
|
||||
|
||||
// Installs request tracking on a fresh check page. Returns the helpers the
|
||||
// audit checks share; a check destructures only what it uses.
|
||||
function trackAudit(page) {
|
||||
page.setDefaultTimeout(10000);
|
||||
const auditRequests = [];
|
||||
page.on('request', (request) => {
|
||||
try {
|
||||
const url = new URL(request.url());
|
||||
if (isAudit(url) || url.pathname === '/api/audit/export') auditRequests.push(url);
|
||||
} catch {
|
||||
// A non-URL request cannot be an audit call.
|
||||
}
|
||||
});
|
||||
let loads = 0;
|
||||
page.on('load', () => (loads += 1));
|
||||
|
||||
const requestMatching = async (action, predicate, timeout = 6000) => {
|
||||
const waiter = page
|
||||
.waitForRequest(
|
||||
(request) => {
|
||||
try {
|
||||
const url = new URL(request.url());
|
||||
return (isAudit(url) || url.pathname === '/api/audit/export') && predicate(url);
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
},
|
||||
{ timeout },
|
||||
)
|
||||
.catch(() => null);
|
||||
await action();
|
||||
return waiter;
|
||||
};
|
||||
|
||||
const waitForUrl = (predicate, timeout = 8000) =>
|
||||
page.waitForURL((url) => predicate(new URL(url)), { timeout });
|
||||
|
||||
const lastRequest = () => auditRequests[auditRequests.length - 1] || null;
|
||||
const loadCount = () => loads;
|
||||
|
||||
return { requestMatching, waitForUrl, lastRequest, loadCount };
|
||||
}
|
||||
|
||||
// Export resolves its window at click time and opens a tab. A successful
|
||||
// export arrives as an attachment download; a failed one stays visible in
|
||||
// that tab, so the check accepts either and reads the response from it.
|
||||
async function exportOnce(page, fail, label, extension) {
|
||||
await page.getByRole('button', { name: 'Export', exact: true }).click();
|
||||
const item = page.getByRole('menuitem', { name: label, exact: true });
|
||||
await item.waitFor();
|
||||
const downloadP = page.waitForEvent('download', { timeout: 15000 }).then((download) => ({ download })).catch(() => null);
|
||||
const popupP = page.waitForEvent('popup', { timeout: 15000 }).then((popup) => ({ popup })).catch(() => null);
|
||||
await item.click();
|
||||
const settled = await Promise.race([downloadP, popupP]);
|
||||
if (!settled) {
|
||||
fail(`${label} export neither downloaded nor opened a tab`);
|
||||
return null;
|
||||
}
|
||||
if (settled.popup) {
|
||||
await settled.popup.waitForLoadState('load').catch(() => {});
|
||||
const url = new URL(settled.popup.url());
|
||||
assert.equal(url.pathname, '/api/audit/export', `${label} opened ${url.href}`);
|
||||
assert.equal(url.searchParams.get('format'), extension);
|
||||
assert.ok(url.searchParams.get('from'), `${label} export has no explicit start`);
|
||||
assert.ok(url.searchParams.get('to'), `${label} export has no explicit end`);
|
||||
const response = await page.request.get(url.href);
|
||||
const body = await response.text();
|
||||
if (response.status() !== 200) {
|
||||
fail(`${label} export failed (${response.status()}): ${body.slice(0, 200)}`);
|
||||
return null;
|
||||
}
|
||||
if (!new RegExp(`attachment;.*\\.${extension}`).test(response.headers()['content-disposition'] || '')) {
|
||||
fail(`${label} export opened a tab without an attachment filename`);
|
||||
return null;
|
||||
}
|
||||
return body;
|
||||
}
|
||||
if (!new RegExp(`\\.${extension}$`).test(settled.download.suggestedFilename())) {
|
||||
fail(`${label} export did not download a .${extension} file`);
|
||||
return null;
|
||||
}
|
||||
const failure = await settled.download.failure();
|
||||
if (failure) {
|
||||
fail(`${label} download failed: ${failure}`);
|
||||
return null;
|
||||
}
|
||||
const saved = await settled.download.path();
|
||||
return saved ? fs.readFileSync(saved, 'utf8') : '';
|
||||
}
|
||||
|
||||
module.exports = { DEFAULT_WINDOW_MS, isAudit, trackAudit, exportOnce };
|
||||
@@ -1,5 +1,5 @@
|
||||
const assert=require('node:assert/strict');
|
||||
module.exports=async({page,go})=>{
|
||||
module.exports=async({page,go,shot})=>{
|
||||
await page.route('**/api/access/policies',async route=>{await new Promise(r=>setTimeout(r,2200));await route.continue()});
|
||||
await go('/access/policies');
|
||||
assert.equal(await page.getByText('No policies yet.',{exact:true}).count(),0);
|
||||
@@ -15,4 +15,5 @@ module.exports=async({page,go})=>{
|
||||
await page.unroute('**/api/tasks?**');
|
||||
await page.getByRole('button',{name:'Retry runs',exact:true}).click();
|
||||
await page.getByRole('alert').filter({hasText:'Could not load recent runs'}).waitFor({state:'hidden'});
|
||||
await shot('recovered');
|
||||
};
|
||||
@@ -1,6 +1,6 @@
|
||||
// The save response is mocked; this checks the page's save path without changing models.
|
||||
const assert=require('node:assert/strict');
|
||||
module.exports=async({page,go})=>{
|
||||
module.exports=async({page,go,shot})=>{
|
||||
let submitted;
|
||||
await page.route('**/api/control-plane/resources/model-aliases/commands/replace',async route=>{
|
||||
submitted=route.request().postDataJSON();
|
||||
@@ -14,4 +14,5 @@ module.exports=async({page,go})=>{
|
||||
await page.getByRole('button',{name:'Save',exact:true}).click();
|
||||
await page.getByRole('heading',{name:/^Review \d/}).waitFor({state:'hidden'});
|
||||
assert.equal(submitted?.value['e2e-test'],'test/model');
|
||||
await shot('saved');
|
||||
};
|
||||
@@ -1,13 +1,16 @@
|
||||
// Workflow definition behaviour at desktop width: legacy title IDs migrate
|
||||
// to slugs, invalid expressions flag the node and toolbar alike, a pinned
|
||||
// historical run keeps its original step IDs inspectable, and handoff
|
||||
// targets round-trip between the step form and YAML.
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
module.exports = async ({ page, go, shot }) => {
|
||||
module.exports = async ({ page, go, shot, settled }) => {
|
||||
await page.route('**/api/control-plane/resources/workflow-drafts', async route => {
|
||||
const response = await route.fetch();
|
||||
const data = await response.json();
|
||||
data.resource.value = { drafts: [] };
|
||||
await route.fulfill({ json: data });
|
||||
});
|
||||
const capture = async label => { await page.waitForTimeout(350); await shot(label); };
|
||||
const panTo = async node => {
|
||||
const flow = page.locator('.vue-flow');
|
||||
await flow.hover();
|
||||
@@ -16,71 +19,21 @@ module.exports = async ({ page, go, shot }) => {
|
||||
const delta = target.y + target.height / 2 - viewport.y - viewport.height / 2;
|
||||
if (Math.abs(delta) < 30) break;
|
||||
await page.mouse.wheel(0, delta * 2);
|
||||
await page.waitForTimeout(250);
|
||||
await settled();
|
||||
}
|
||||
};
|
||||
for (const width of [360, 390]) {
|
||||
await page.setViewportSize({ width, height: 844 });
|
||||
for (const workflow of ['implement', 'feasibility']) {
|
||||
await go(`/workflows?workflow=${workflow}`);
|
||||
const canvas = page.locator('.workflow-canvas');
|
||||
await canvas.waitFor();
|
||||
const dimensions = await page.evaluate(() => ({
|
||||
height: innerHeight, width: innerWidth,
|
||||
scrollHeight: document.documentElement.scrollHeight,
|
||||
scrollWidth: document.documentElement.scrollWidth,
|
||||
}));
|
||||
assert.equal(dimensions.scrollHeight, dimensions.height, 'workflow page scrolls vertically');
|
||||
assert.equal(dimensions.scrollWidth, dimensions.width, 'workflow page scrolls horizontally');
|
||||
const topbar = page.locator('header').first();
|
||||
const bar = await topbar.boundingBox();
|
||||
assert.equal(bar.y, 0, 'blank band above topbar');
|
||||
assert.equal(await topbar.evaluate(el => getComputedStyle(el).backdropFilter), 'none');
|
||||
assert.ok(!await canvas.getByText('Works on a repository', { exact: true }).count());
|
||||
const run = page.getByRole('button', { name: new RegExp(`^(Run|Test) ${workflow}$`) });
|
||||
assert.ok(await run.isVisible(), 'Run hidden on phone');
|
||||
const box = await canvas.boundingBox();
|
||||
const start = await page.locator('.vue-flow__node[data-id="start"]').boundingBox();
|
||||
assert.ok(start.y >= box.y && start.y < box.y + box.height, 'start is outside initial viewport');
|
||||
const nodes = await page.locator('.vue-flow__node').evaluateAll(elements => elements.map(el => ({
|
||||
left: el.getBoundingClientRect().left, right: el.getBoundingClientRect().right,
|
||||
})));
|
||||
assert.ok(nodes.every(node => node.left >= box.x && node.right <= box.x + box.width), 'branch lanes clipped horizontally');
|
||||
const controls = await page.getByRole('button', { name: 'Zoom in', exact: true }).boundingBox();
|
||||
const flow = await canvas.locator('.vue-flow').boundingBox();
|
||||
assert.ok(controls.y >= flow.y + flow.height, 'zoom control overlays graph');
|
||||
await capture(`${workflow}-${width}`);
|
||||
const step = page.locator('.vue-flow__node').filter({ hasText: 'Prepare worktree' }).first();
|
||||
assert.equal(await step.locator('[data-slot="badge"]').innerText(), 'Prepare worktree');
|
||||
await step.click();
|
||||
const dialog = page.getByRole('dialog');
|
||||
await dialog.waitFor();
|
||||
const dialogBox = await dialog.boundingBox();
|
||||
assert.ok(dialogBox.x >= 0 && dialogBox.x + dialogBox.width <= width, 'step editor exceeds phone width');
|
||||
const input = dialog.getByRole('textbox').first();
|
||||
if (await input.count()) assert.ok(await input.evaluate(el => el.getBoundingClientRect().height) >= 24, 'step form scaled down');
|
||||
await capture(`${workflow}-step-${width}`);
|
||||
await page.keyboard.press('Escape');
|
||||
await run.click();
|
||||
await page.getByRole('dialog').waitFor();
|
||||
await capture(`${workflow}-run-${width}`);
|
||||
await page.keyboard.press('Escape');
|
||||
}
|
||||
}
|
||||
// Reproduce a previously stored title ID without changing the test server.
|
||||
const response = await page.request.get(new URL('/api/control-plane/resources/workflow-definitions', page.url()).href);
|
||||
const resource = await response.json();
|
||||
const definition = resource.resource.value.definitions.find(entry => entry.id === 'feasibility');
|
||||
definition.yaml = definition.yaml.replace(/\bid: assess\b/, 'id: Write a Product Requirement Document').replaceAll('steps.assess.', 'steps.Write a Product Requirement Document.');
|
||||
await page.route('**/api/control-plane/resources/workflow-definitions', route => route.fulfill({ json: resource }));
|
||||
await page.setViewportSize({ width: 320, height: 700 });
|
||||
await go('/workflows?workflow=feasibility');
|
||||
await page.getByRole('tab', { name: 'YAML', exact: true }).click();
|
||||
const yaml = await page.getByLabel('Workflow YAML', { exact: true }).inputValue();
|
||||
assert.ok(yaml.includes('id: write-a-product-requirement-document'));
|
||||
assert.ok(yaml.includes('steps.write-a-product-requirement-document.result'));
|
||||
assert.ok(!yaml.includes('steps.Write a Product Requirement Document'));
|
||||
assert.equal(await page.evaluate(() => document.documentElement.scrollHeight), 700, 'YAML causes page scroll');
|
||||
await page.getByRole('tab', { name: 'Canvas', exact: true }).click();
|
||||
const approval = page.locator('.vue-flow__node').filter({ hasText: 'Approval' }).first();
|
||||
await panTo(approval);
|
||||
@@ -90,7 +43,7 @@ module.exports = async ({ page, go, shot }) => {
|
||||
const chip = dialog.getByRole('button', { name: 'write-a-product-requirement-document', exact: true });
|
||||
await chip.waitFor();
|
||||
await chip.click();
|
||||
await capture('legacy-reference-320');
|
||||
await shot('legacy-reference');
|
||||
await page.keyboard.press('Escape');
|
||||
await page.getByRole('tab', { name: 'YAML', exact: true }).click();
|
||||
const field = page.getByLabel('Workflow YAML', { exact: true });
|
||||
@@ -100,8 +53,11 @@ module.exports = async ({ page, go, shot }) => {
|
||||
await problem.waitFor();
|
||||
const toolbarProblem = page.getByRole('status').filter({ has: page.locator('svg') });
|
||||
assert.equal(await problem.evaluate(el => getComputedStyle(el).color), await toolbarProblem.evaluate(el => getComputedStyle(el).color));
|
||||
assert.equal(await page.evaluate(() => document.documentElement.scrollWidth), 320, 'invalid toolbar overflows phone');
|
||||
await capture('validation-320');
|
||||
const invalidPage = await page.evaluate(() => ({
|
||||
scrollWidth: document.documentElement.scrollWidth, width: innerWidth,
|
||||
}));
|
||||
assert.equal(invalidPage.scrollWidth, invalidPage.width, 'invalid toolbar overflows');
|
||||
await shot('validation');
|
||||
// A pinned historical run keeps its original step IDs and remains inspectable.
|
||||
const snapshot = definition.yaml;
|
||||
await page.route('**/api/tasks?*', route => {
|
||||
@@ -120,27 +76,27 @@ module.exports = async ({ page, go, shot }) => {
|
||||
const root = await page.locator('.vue-flow__node[data-id="workflow"]').boundingBox();
|
||||
const pinnedCanvas = await page.locator('.workflow-canvas').boundingBox();
|
||||
assert.ok(root.y >= pinnedCanvas.y, 'historical workflow header clipped above canvas');
|
||||
await capture('run-picker-320');
|
||||
await shot('run-picker');
|
||||
await page.locator('.vue-flow').hover();
|
||||
await page.mouse.wheel(0, 550);
|
||||
await page.waitForTimeout(300);
|
||||
await settled();
|
||||
const historicalStep = page.locator('.vue-flow__node').filter({ hasText: 'Write a Product Requirement Document' }).first();
|
||||
await panTo(historicalStep);
|
||||
await historicalStep.click();
|
||||
await page.getByRole('dialog').waitFor();
|
||||
await page.getByText('Historical failure', { exact: true }).first().waitFor();
|
||||
await capture('historical-run-320');
|
||||
await shot('historical-run');
|
||||
await page.keyboard.press('Escape');
|
||||
await picker.click();
|
||||
await page.getByRole('option', { name: 'Definition', exact: true }).click();
|
||||
assert.equal(await picker.innerText(), 'Definition');
|
||||
await capture('definition-picker-320');
|
||||
await shot('definition-picker');
|
||||
await go('/workflows?workflow=triage');
|
||||
const runPicker = page.getByRole('combobox', { name: 'Run shown on the canvas' });
|
||||
if (await runPicker.count()) { await runPicker.click(); await page.getByRole('option', { name: 'Definition', exact: true }).click(); }
|
||||
await page.locator('.vue-flow').hover();
|
||||
await page.mouse.wheel(0, 650);
|
||||
await page.waitForTimeout(300);
|
||||
await settled();
|
||||
const handoff = page.locator('.vue-flow__node').filter({ hasText: 'Hand off' }).first();
|
||||
await panTo(handoff);
|
||||
await handoff.click();
|
||||
@@ -149,7 +105,7 @@ module.exports = async ({ page, go, shot }) => {
|
||||
assert.equal(await target.inputValue(), '{{ steps.classify.result.workflow }}');
|
||||
await target.fill('{{ steps.classify.result.alternative }}');
|
||||
await target.press('Home');
|
||||
await capture('dynamic-handoff-320');
|
||||
await shot('dynamic-handoff');
|
||||
await page.keyboard.press('Escape');
|
||||
await page.getByRole('tab', { name: 'YAML', exact: true }).click();
|
||||
assert.ok((await page.getByLabel('Workflow YAML', { exact: true }).inputValue()).includes('steps.classify.result.alternative'));
|
||||
@@ -0,0 +1,83 @@
|
||||
// The workflow canvas at phone width: no page scroll, a clean topbar, lanes
|
||||
// inside the canvas, zoom controls below the graph, step and run dialogs
|
||||
// that fit, YAML without page scroll, and a pinned run header that stays
|
||||
// visible.
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
module.exports = async ({ page, go, shot, settled }) => {
|
||||
await page.setViewportSize({ width: 390, height: 844 });
|
||||
for (const workflow of ['implement', 'feasibility']) {
|
||||
await go(`/workflows?workflow=${workflow}`);
|
||||
const canvas = page.locator('.workflow-canvas');
|
||||
await canvas.waitFor();
|
||||
const dimensions = await page.evaluate(() => ({
|
||||
height: innerHeight, width: innerWidth,
|
||||
scrollHeight: document.documentElement.scrollHeight,
|
||||
scrollWidth: document.documentElement.scrollWidth,
|
||||
}));
|
||||
assert.equal(dimensions.scrollHeight, dimensions.height, 'workflow page scrolls vertically');
|
||||
assert.equal(dimensions.scrollWidth, dimensions.width, 'workflow page scrolls horizontally');
|
||||
const topbar = page.locator('header').first();
|
||||
const bar = await topbar.boundingBox();
|
||||
assert.equal(bar.y, 0, 'blank band above topbar');
|
||||
assert.equal(await topbar.evaluate(el => getComputedStyle(el).backdropFilter), 'none');
|
||||
assert.ok(!await canvas.getByText('Works on a repository', { exact: true }).count());
|
||||
const run = page.getByRole('button', { name: new RegExp(`^(Run|Test) ${workflow}$`) });
|
||||
assert.ok(await run.isVisible(), 'Run hidden on phone');
|
||||
const box = await canvas.boundingBox();
|
||||
const start = await page.locator('.vue-flow__node[data-id="start"]').boundingBox();
|
||||
assert.ok(start.y >= box.y && start.y < box.y + box.height, 'start is outside initial viewport');
|
||||
const nodes = await page.locator('.vue-flow__node').evaluateAll(elements => elements.map(el => ({
|
||||
left: el.getBoundingClientRect().left, right: el.getBoundingClientRect().right,
|
||||
})));
|
||||
assert.ok(nodes.every(node => node.left >= box.x && node.right <= box.x + box.width), 'branch lanes clipped horizontally');
|
||||
const controls = await page.getByRole('button', { name: 'Zoom in', exact: true }).boundingBox();
|
||||
const flow = await canvas.locator('.vue-flow').boundingBox();
|
||||
assert.ok(controls.y >= flow.y + flow.height, 'zoom control overlays graph');
|
||||
await shot(workflow);
|
||||
const step = page.locator('.vue-flow__node').filter({ hasText: 'Prepare worktree' }).first();
|
||||
assert.equal(await step.locator('[data-slot="badge"]').innerText(), 'Prepare worktree');
|
||||
await step.click();
|
||||
const dialog = page.getByRole('dialog');
|
||||
await dialog.waitFor();
|
||||
const dialogBox = await dialog.boundingBox();
|
||||
assert.ok(dialogBox.x >= 0 && dialogBox.x + dialogBox.width <= 390, 'step editor exceeds phone width');
|
||||
const input = dialog.getByRole('textbox').first();
|
||||
if (await input.count()) assert.ok(await input.evaluate(el => el.getBoundingClientRect().height) >= 24, 'step form scaled down');
|
||||
await shot(`${workflow}-step`);
|
||||
await page.keyboard.press('Escape');
|
||||
await run.click();
|
||||
await page.getByRole('dialog').waitFor();
|
||||
await shot(`${workflow}-run`);
|
||||
await page.keyboard.press('Escape');
|
||||
}
|
||||
// YAML must not make the page scroll at phone width.
|
||||
await go('/workflows?workflow=feasibility');
|
||||
await page.getByRole('tab', { name: 'YAML', exact: true }).click();
|
||||
await settled();
|
||||
const yamlPage = await page.evaluate(() => ({
|
||||
scrollHeight: document.documentElement.scrollHeight, height: innerHeight,
|
||||
}));
|
||||
assert.equal(yamlPage.scrollHeight, yamlPage.height, 'YAML causes page scroll');
|
||||
await page.getByRole('tab', { name: 'Canvas', exact: true }).click();
|
||||
// A pinned historical run keeps its header visible on the canvas.
|
||||
const response = await page.request.get(new URL('/api/control-plane/resources/workflow-definitions', page.url()).href);
|
||||
const resource = await response.json();
|
||||
const snapshot = resource.resource.value.definitions.find(entry => entry.id === 'feasibility').yaml;
|
||||
await page.route('**/api/tasks?*', route => {
|
||||
const workflow = new URL(route.request().url()).searchParams.get('workflow');
|
||||
return workflow === 'feasibility'
|
||||
? route.fulfill({ json: [{ id: 999999, workflow, title: 'Historical run', status: 'failed', workflow_definition_yaml: snapshot }] })
|
||||
: route.continue();
|
||||
});
|
||||
await page.route('**/api/tasks/999999/*?*', route => route.fulfill({ json: {} }));
|
||||
await page.route('**/api/tasks/999999/debug', route => route.fulfill({ json: {} }));
|
||||
await page.route('**/api/tasks/999999/attempts', route => route.fulfill({ json: { attempts: [{ attempt: 1, stages: [] }] } }));
|
||||
await go('/workflows?workflow=feasibility');
|
||||
const picker = page.getByRole('combobox', { name: 'Run shown on the canvas' });
|
||||
await picker.waitFor();
|
||||
assert.match(await picker.innerText(), /Run #999999/);
|
||||
const root = await page.locator('.vue-flow__node[data-id="workflow"]').boundingBox();
|
||||
const pinnedCanvas = await page.locator('.workflow-canvas').boundingBox();
|
||||
assert.ok(root.y >= pinnedCanvas.y, 'historical workflow header clipped above canvas');
|
||||
};
|
||||
+79
-10
@@ -1,13 +1,25 @@
|
||||
// Drive the dashboard of a running local stack (e2e/stack.sh) in Chromium. Each module in checks/ exports async ({ page, go, shot, fail }).
|
||||
// A check fails on a thrown error or any console or page error.
|
||||
// Drive the dashboard of a running local stack (e2e/stack.sh) in Chromium. Each module in checks/ exports async ({ page, go, shot, fail, skip, settled }).
|
||||
// A check fails on a thrown error or any console or page error. skip(reason)
|
||||
// records a skip for a missing precondition and never fails.
|
||||
//
|
||||
// node e2e/run.cjs [--headed] [--record VERSION] [list | all | check ...]
|
||||
//
|
||||
// Screenshots and results.json go to e2e/out/<stamp>/. --record also writes
|
||||
// e2e/results/VERSION.md, the committed evidence for a release.
|
||||
// Screenshots, results.json (the manifest) and index.html (the gallery) go to
|
||||
// e2e/out/<stamp>/. --record also writes e2e/results/VERSION.md, the committed
|
||||
// evidence for a release.
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
const DESKTOP = { width: 1440, height: 900 };
|
||||
|
||||
// Stable label naming: kebab-case, so a route like /org/members becomes
|
||||
// org-members and / becomes home.
|
||||
function slug(label) {
|
||||
return String(label).toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/^-+|-+$/g, '') || 'home';
|
||||
}
|
||||
|
||||
class SkipCheck extends Error {}
|
||||
|
||||
const base = process.env.ARCHIE_E2E_URL || 'http://localhost:5173';
|
||||
const args = process.argv.slice(2);
|
||||
const headed = args.includes('--headed');
|
||||
@@ -63,27 +75,84 @@ fs.mkdirSync(out, { recursive: true });
|
||||
page.on('console', (m) => m.type() === 'error' && errors.push(m.text()));
|
||||
// Wait for load, never networkidle: the event stream never goes idle.
|
||||
const go = async (p) => { await page.goto(base + p, { waitUntil: 'load' }); await page.waitForTimeout(1200); };
|
||||
const shot = (label) => page.screenshot({ path: path.join(out, `${name}-${label.replace(/[^a-z0-9-]+/gi, '_') || 'root'}.png`) });
|
||||
// Harness-owned wait: wait for a selector or URL, then settle briefly.
|
||||
const settled = async (sel, timeout = 10000) => {
|
||||
if (sel) {
|
||||
if (/^https?:/.test(sel)) await page.waitForURL(sel, { timeout });
|
||||
else if (sel.startsWith('/')) await page.waitForURL('**' + sel, { timeout });
|
||||
else await page.locator(sel).first().waitFor({ timeout });
|
||||
}
|
||||
await page.waitForTimeout(150);
|
||||
};
|
||||
const shots = [];
|
||||
// <check>-<kebab>[-<w>x<h>].png; the viewport suffix marks non-desktop
|
||||
// shots. A short settle precedes every capture so checks need no capture
|
||||
// wrapper of their own. Every capture is kept: a repeated label gets -2, -3...
|
||||
const shot = async (label) => {
|
||||
await page.waitForTimeout(150);
|
||||
const vp = page.viewportSize() || DESKTOP;
|
||||
const suffix = (vp.width === DESKTOP.width && vp.height === DESKTOP.height) ? '' : `-${vp.width}x${vp.height}`;
|
||||
const stem = `${name}-${slug(label)}${suffix}`;
|
||||
let file = `${stem}.png`;
|
||||
for (let n = 2; fs.existsSync(path.join(out, file)); n++) file = `${stem}-${n}.png`;
|
||||
await page.screenshot({ path: path.join(out, file) });
|
||||
shots.push({ file, w: vp.width, h: vp.height, check: name });
|
||||
return file;
|
||||
};
|
||||
const fail = (msg) => notes.push(msg);
|
||||
const skip = (reason) => { throw new SkipCheck(reason || 'precondition missing'); };
|
||||
const started = Date.now();
|
||||
let error = '';
|
||||
try { await check({ page, go, shot, fail }); } catch (e) { error = String(e && e.message || e); }
|
||||
let skipped = '';
|
||||
try { await check({ page, go, shot, fail, skip, settled }); } catch (e) {
|
||||
if (e instanceof SkipCheck) skipped = e.message;
|
||||
else error = String(e && e.message || e);
|
||||
}
|
||||
await page.close();
|
||||
const problems = [error, ...notes, ...errors].filter(Boolean);
|
||||
results.push({ check: name, ok: problems.length === 0, ms: Date.now() - started, problems });
|
||||
console.log(`${problems.length ? 'FAIL' : 'ok '} ${name}${problems.length ? '\n ' + problems.join('\n ') : ''}`);
|
||||
const problems = skipped ? [] : [error, ...notes, ...errors].filter(Boolean);
|
||||
const result = { check: name, ok: problems.length === 0, ms: Date.now() - started, problems, shots };
|
||||
if (skipped) { result.skipped = true; result.skipReason = skipped; }
|
||||
results.push(result);
|
||||
console.log(`${skipped ? 'SKIP' : problems.length ? 'FAIL' : 'ok '} ${name}${skipped ? ` (${skipped})` : problems.length ? '\n ' + problems.join('\n ') : ''}`);
|
||||
}
|
||||
await browser.close();
|
||||
fs.writeFileSync(path.join(out, 'results.json'), JSON.stringify(results, null, 2));
|
||||
fs.writeFileSync(path.join(out, 'index.html'), gallery(results));
|
||||
if (record) writeRecord(record, results);
|
||||
console.log(`evidence: ${out}`);
|
||||
process.exit(results.every((r) => r.ok) ? 0 : 1);
|
||||
})().catch((e) => { console.error(e); process.exit(2); });
|
||||
|
||||
function esc(s) {
|
||||
return String(s).replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>').replace(/"/g, '"');
|
||||
}
|
||||
|
||||
// Per-run gallery for human image review, grouped by check.
|
||||
function gallery(results) {
|
||||
const sections = results.map((r) => {
|
||||
const status = r.skipped ? 'skip' : r.ok ? 'pass' : 'fail';
|
||||
const head = `<h2>${esc(r.check)} <span class="${status}">${status}</span> <small>${r.ms}ms</small></h2>`;
|
||||
const skipNote = r.skipped ? `<p class="skip-reason">skipped: ${esc(r.skipReason || '')}</p>` : '';
|
||||
const probs = r.problems.length
|
||||
? `<ul class="problems">${r.problems.map((p) => `<li>${esc(p)}</li>`).join('')}</ul>` : '';
|
||||
const shots = r.shots.length
|
||||
? `<div class="shots">${r.shots.map((s) => `<figure><a href="${esc(s.file)}"><img src="${esc(s.file)}" loading="lazy"></a><figcaption>${esc(s.file)} ${s.w}x${s.h}</figcaption></figure>`).join('')}</div>`
|
||||
: '<p class="no-shots">no screenshots</p>';
|
||||
return `<section class="${status}">${head}${skipNote}${probs}${shots}</section>`;
|
||||
}).join('\n');
|
||||
const passed = results.filter((r) => r.ok && !r.skipped).length;
|
||||
const skipped = results.filter((r) => r.skipped).length;
|
||||
return `<!doctype html>
|
||||
<html><head><meta charset="utf-8"><title>e2e ${esc(stamp)}</title>
|
||||
<style>body{font-family:system-ui,sans-serif;max-width:1200px;margin:0 auto;padding:1em}h2 small{color:#666;font-weight:normal}.pass>h2 .pass,.fail>h2 .fail,.skip>h2 .skip{padding:.1em .5em;border-radius:.3em;font-size:.8em}.pass>h2 .pass{background:#dfd}.fail>h2 .fail{background:#fdd}.skip>h2 .skip{background:#eef}.problems{color:#a00}.skip-reason{color:#555}.no-shots{color:#888;font-style:italic}.shots{display:flex;flex-wrap:wrap;gap:1em}figure{margin:0;width:320px}img{width:100%;border:1px solid #ccc}figcaption{font-size:.8em;color:#555;word-break:break-all}section{border-top:1px solid #ddd;padding-bottom:1em}</style>
|
||||
</head><body><h1>e2e run ${esc(stamp)}</h1><p>${passed} passed, ${skipped} skipped, ${results.length - passed - skipped} failed against ${esc(base)}.</p>${sections}</body></html>
|
||||
`;
|
||||
}
|
||||
|
||||
function writeRecord(version, results) {
|
||||
const dir = path.join(__dirname, 'results');
|
||||
fs.mkdirSync(dir, { recursive: true });
|
||||
const rows = results.map((r) => `| ${r.check} | ${r.ok ? 'pass' : 'fail'} | ${r.problems.join('<br>').replace(/\|/g, '\\|') || ''} |`);
|
||||
const rows = results.map((r) => `| ${r.check} | ${r.skipped ? 'skip' : r.ok ? 'pass' : 'fail'} | ${r.problems.join('<br>').replace(/\|/g, '\\|') || ''} |`);
|
||||
const passed = results.filter((r) => r.ok).length;
|
||||
fs.writeFileSync(path.join(dir, `${version}.md`), [
|
||||
`# ${version}`, '',
|
||||
|
||||
Reference in new issue
Block a user