Usage reporting: per-identity attribution, cost, and an org usage page #1855

Closed
opened 2026-11-10 14:56:43 +00:00 by sam · 1 comment
Owner

Model usage is recorded per model call but nothing reads it: no query, no API, no page. The dashboard's token numbers come from per-task totals, which have no model, source or identity dimension, so nobody can see what an org actually consumed or what it cost.

Add the read side, org-scoped:

  • Aggregate usage rows by day, provider/model, source (task/chat/curator) and identity.
  • Attribute each call to the identity the work was requested as: the task's identity for task and MCP-sampling calls, the calling principal for chat and curator calls. Carry it in the usage record so an outbox replay keeps it, and backfill existing task rows from their task.
  • Price at read time from the model catalog list price, netting cache hits: uncached input, cached input, and output. A model with no catalog price reports tokens only, never a guessed cost.
  • One org usage page for admins and owners: date range, totals, a daily series, and breakdowns by model, source and identity. The system org's page is the instance view.
  • Cross-org reads are forbidden by construction: the store derives the org from the principal, with no org parameter and no all-orgs mode.

Done when:

  • A stats read returns per-org usage rows for a date range.
  • Recording a model call stores its identity, and an outbox replay keeps it.
  • The usage page shows tokens and cost for the selected range and breaks down by model, source and identity.
  • A viewer or developer is refused the usage read, and no caller can read another org's usage.
  • task check passes.
Model usage is recorded per model call but nothing reads it: no query, no API, no page. The dashboard's token numbers come from per-task totals, which have no model, source or identity dimension, so nobody can see what an org actually consumed or what it cost. Add the read side, org-scoped: - Aggregate usage rows by day, provider/model, source (task/chat/curator) and identity. - Attribute each call to the identity the work was requested as: the task's identity for task and MCP-sampling calls, the calling principal for chat and curator calls. Carry it in the usage record so an outbox replay keeps it, and backfill existing task rows from their task. - Price at read time from the model catalog list price, netting cache hits: uncached input, cached input, and output. A model with no catalog price reports tokens only, never a guessed cost. - One org usage page for admins and owners: date range, totals, a daily series, and breakdowns by model, source and identity. The system org's page is the instance view. - Cross-org reads are forbidden by construction: the store derives the org from the principal, with no org parameter and no all-orgs mode. Done when: - A stats read returns per-org usage rows for a date range. - Recording a model call stores its identity, and an outbox replay keeps it. - The usage page shows tokens and cost for the selected range and breaks down by model, source and identity. - A viewer or developer is refused the usage read, and no caller can read another org's usage. - task check passes.
sam self-assigned this 2026-11-10 14:56:44 +00:00
Author
Owner

Landed in 4728c8c2.

  • model_usage gains identity_id: task calls take it from the task row, chat and curator calls from the calling principal carried in the usage record, so an outbox replay keeps its attribution. Existing task rows are backfilled from tasks.
  • StatsService.UsageTotals returns the caller org's rows by UTC day, source, provider, model and identity. There is no org parameter and no all-orgs mode.
  • GET /api/usage prices tokens at the catalog list rate, netting cache hits; calls to models without a price report tokens only. The published catalog view now carries each model's cost.
  • New org Usage page behind a read_usage grant (admin and owner only): 7/30/90-day range, totals, daily series, and model, source and identity breakdowns. The org shell keeps it on the caller's own org, so an instance admin viewing another org is sent back instead of seeing their own numbers under it.

Verified: task check passed on the committed tree — Go tests, dashboard tests and typecheck, lint, vet, proto and sqlc generation checks.

Not verified: no browser check against a running instance, so the page has not been seen with real usage rows.

Landed in 4728c8c2. - model_usage gains identity_id: task calls take it from the task row, chat and curator calls from the calling principal carried in the usage record, so an outbox replay keeps its attribution. Existing task rows are backfilled from tasks. - StatsService.UsageTotals returns the caller org's rows by UTC day, source, provider, model and identity. There is no org parameter and no all-orgs mode. - GET /api/usage prices tokens at the catalog list rate, netting cache hits; calls to models without a price report tokens only. The published catalog view now carries each model's cost. - New org Usage page behind a read_usage grant (admin and owner only): 7/30/90-day range, totals, daily series, and model, source and identity breakdowns. The org shell keeps it on the caller's own org, so an instance admin viewing another org is sent back instead of seeing their own numbers under it. Verified: task check passed on the committed tree — Go tests, dashboard tests and typecheck, lint, vet, proto and sqlc generation checks. Not verified: no browser check against a running instance, so the page has not been seen with real usage rows.
sam closed this issue 2026-11-10 15:15:37 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sam/archie#1855