Files
archie/docs/prds/payload-field-mapping.md
T
sam aba035a396 feat(webui): payload field mapping -- bind JSON paths to named fields, preview before saving
Adds t2db.3: schema-by-example field mapping for the no-code playbook epic.
An operator picks a captured event, clicks through its payload to bind named
fields to JSON paths, and previews the resolution against a real event
before saving -- a missing, null, or type-changed path fails loudly (named
by field and event) rather than handing an agent a blank value.

- internal/domain/mapping: Mapping/Field types, path parsing/walking,
  Resolve (per-field failure reporting), Validate (incl. duplicate names)
- internal/store/mappings.go: SQLite persistence, mirrors captures.go
- internal/webui/api_mapping.go: CRUD + preview HTTP API
- ui/src/mappings/: dashboard editor (clickable payload tree, live preview)

Design decisions recorded in docs/prds/payload-field-mapping.md. t2db.4
(binding a mapping + matcher to a workflow) is out of scope here.
2026-08-22 19:18:15 +10:00

160 lines
7.2 KiB
Markdown

# Payload field mapping -- decision
**Status:** Decided, not yet implemented
**Date:** 2026-08-22
**Beads issue:** `archie-core-t2db.3`, depends on `t2db.1`, blocks `t2db.4`
Answers what a "mapping" is, where it lives, and what "resolve against a
real event" means precisely enough to implement. `t2db.4` (binding: matcher
+ mapping + workflow) is out of scope -- this bead ships the mapping
concept and its preview only, as a standalone entity `t2db.4` will later
reference by ID.
## Decision: a new `internal/domain/mapping` package, SQLite storage
`internal/domain/mapping` owns the mapping vocabulary and the pure
resolve/preview rule; `internal/store` persists it (mirrors
`captures.go`'s shape exactly -- inline schema const, narrow store
interface, `*store.Store` implementation); `internal/webui/api_mapping.go`
is the transport. This follows `docs/prds/event-capture-storage.md`'s own
placement precedent: `internal/store` is where this daemon's persistence
already lives, not `internal/infrastructure` (that layer today holds
cross-domain services -- config, eventbus, modelcatalog -- not this
daemon's primary store).
### Why a domain package and not just a store table
Resolving a path against a payload and deciding "missing vs null vs
type-changed" is a rule, not a query -- CLAUDE.md's domain layer owns
"language, state, rules, operations" precisely so this logic is testable
without a database and reusable by `t2db.4`'s runtime trigger path without
either package importing the other's transport concerns.
## Mapping shape
```go
type Mapping struct {
ID int64
Name string
SourceHint string // free-text label (e.g. "sentry"), organisational only -- not enforced against Capture.Source
Fields []Field
CreatedAt time.Time
UpdatedAt time.Time
}
type Field struct {
Name string // the name the workflow receives
Path string // dot/bracket path into the payload, see below
Type FieldType // expected type, checked at resolve time
Required bool // Required=false + missing is not a failure; Required=false + wrong type still is
}
type FieldType string // "string" | "number" | "bool" | "object" | "array" | "any"
```
`SourceHint` is advisory only: an operator authors a mapping by clicking
through one example capture, but the mapping is not pinned to that
capture's exact source string, because `t2db.4`'s matcher (not this bead)
decides which events a binding applies to. Pinning here would duplicate
that decision in the wrong layer.
## Path syntax
Dot-separated keys with `[N]` for array indices: `data.items[0].id`. No
wildcards, no filters, no external library -- the grammar is exactly what a
click-through UI needs to express (this key, then this key, then this
index), and every path in a mapping was generated by clicking a value in a
real payload, never hand-written against a spec. A hand-rolled parser
(~30 lines: split on `.`, strip `[N]` suffixes) is less risk than a new
dependency for a grammar this small.
## Resolve semantics (the preview AND the future runtime path)
```go
func Resolve(fields []Field, payload []byte) (values map[string]any, failures []Failure)
type Failure struct {
FieldName string
Path string
Reason string // "missing" | "null" | "type_mismatch"
}
```
Decode with `json.Decoder.UseNumber()` (matches `webhookguard.RedactPayload`'s
existing convention) so int/float distinction survives for type checking.
Walk each field's path independently -- one field's failure does not stop
resolution of the others, because the whole point of preview is showing
the operator every problem at once, not the first one.
- Path segment doesn't exist, or indexes past an array's length -> `missing`
- Path resolves to JSON `null` -> `null`
- Path resolves to a value whose JSON type doesn't match `Field.Type`
(`"any"` always matches) -> `type_mismatch`
- `Required: false` suppresses `missing` only, never `null` or
`type_mismatch` -- a field that's present but the wrong shape is always
worth surfacing, since silently coercing it is exactly the "agent
confabulates around a blank field" failure this bead exists to prevent.
**This is the same function both consumers use.** `t2db.3`'s preview calls
it against an operator-picked capture and renders every result, pass or
fail, in the UI -- that satisfies this bead's "fail loudly" acceptance
criterion, because at this stage the only runtime consumer of a mapping
*is* the human previewing it. `t2db.4` will call the identical function
when a live event triggers a binding and turn any non-empty `failures` into
a visible operator-facing failure (park the task / surface in the
inspector) instead of starting a workflow with blank fields -- that wiring
is `t2db.4`'s job, not duplicated here.
## Storage
`internal/store/mappings.go`, `field_mappings` table: `id, name,
source_hint, fields_json, created_at, updated_at`. `fields_json` is the
`[]Field` slice JSON-encoded into one column -- fields have no independent
query need (nothing filters "mappings with a field named X" today), so a
join table would be structure with no consumer, the same failure mode
`organisation.md` bans for empty layering.
`MappingStore` interface (narrow, mirrors `CaptureStore`):
`InsertMapping`, `ListMappings`, `GetMapping`, `UpdateMapping`,
`DeleteMapping`.
## HTTP surface
All under `/api/mappings`, token-gated like every other `/api/*` route (no
bypass -- unlike capture, nothing here accepts unauthenticated input):
- `GET /api/mappings` / `POST /api/mappings` -- list / create
- `GET|PATCH|DELETE /api/mappings/{id}`
- `POST /api/mappings/preview` -- body `{fields: []Field, capture_id:
int64}`, calls `mapping.Resolve` against that capture's stored (already
redacted) `Body` and returns `{values, failures}`. Deliberately takes
loose `fields` rather than a saved mapping ID, so an operator can iterate
before saving anything -- "never ship a mapping the operator has not seen
resolve against real data" requires preview to work pre-save.
Preview resolves against `CapturedEvent.Body`, which is already
redaction-passed (`webhookguard.RedactPayload`, per
`event-capture-storage.md`) -- a mapped field pointing at a
sensitive-keyed path previews as the literal string `"[redacted]"`, not the
real value. This is accepted, not fixed here: it is the stored ground
truth, the same payload `t2db.4`'s runtime path will resolve against, and a
preview mismatch would be worse than an honest one.
## Frontend
New `ui/src/mappings/` feature folder, same three-file split as
`ui/src/captures/`: `mappings.js` (page: list, fetch, create/edit flow),
`mapping-editor.js` (pure, unit-testable: renders a captured payload as a
clickable tree -- clicking a leaf appends a field row with `Path`
pre-filled -- plus the field table and preview-result rendering),
`mappings.css`. Creating a mapping starts from an existing captured event
(reuses `api.captures`), so the editor always has a real payload to click
through and preview against, never a blank form.
## What this does not decide
The binding entity, its matcher, its draft/pending_approval/armed state
machine, and turning a runtime `Resolve` failure into a parked task are
`t2db.4`. Per-source HMAC secret registration is `t2db.5`. This bead ships
the mapping concept, its storage, and its preview only.