* fix(postgres): scope OpenPRs to the caller's org A principal-carrying caller read every org's pr_open tasks. The query now honours the request scope; the daemon's unscoped review scan still sees all orgs. Issue: archie-core-kgxw.23 * fix(statestore): serve task logs only to the task's org ReadTaskLog and StreamTaskLogContent took a task ID and read the file with no org check. The State Store's task-log reader now confirms the task is visible under the caller's scope first; unscoped services read as before. Issue: archie-core-kgxw.24 * fix(postgres): scope ListPolicies to the caller's org Every org's policy text reached any principal-carrying caller. A scoped read now sees its own org's policies, plus instance policies when scoped to the system org. The engines and the lockout guard decide for every org, so they read the full chain unscoped: the dashboard engine loads without the request's principal and the guard lifts the caller's scope. Issue: archie-core-kgxw.25 * fix(postgres): scope identity reads to the caller's org ListIdentities and GetIdentity (and its alias fallback) returned every identity to any caller. A caller scoped to an org now sees only that org's members and assigned agents; the system org and unscoped services still see the whole instance. Issue: archie-core-kgxw.26 * fix(postgres): hide instance status from other orgs ListPresence, ListChannelStatus, ListApplyStatus and GetConfigSnapshot answered any org principal with instance-wide state. They now return nothing unless the request is unscoped or scoped to the system org. The dashboard reads the published config as itself, since its handlers project it for callers of every org. Issue: archie-core-kgxw.27 * feat(postgres): assign new agents an org and role at creation A bot or service account created through the API served no org until the next State Store boot granted it one. Create now assigns it to the org the request acts in with the developer role in the same transaction, so it is authorized immediately. Issue: archie-core-kgxw.14 * feat(edastore): create event records in a chosen workspace Sources are created in a workspace of the caller's org, the default one when none is named; event types and bindings take their source's workspace and mappings their event type's, the same rule the org backfill applied. The binding, mapping and event type contracts carry the workspace, the four list endpoints accept ?workspace=, and the Events page has a workspace selector that filters every step and picks where a new source goes. Issue: archie-core-kgxw.11 * feat(webui): authorize a record in its workspace The API built every access request with no workspace, so workspace policies and per-workspace roles never applied. A request naming a source, binding, mapping, event type or task now reads the record's workspace under the caller's scope and authorizes in it; tasks carry their workspace on the State Store contract for this. Issue: archie-core-kgxw.9 * feat(access): expose workspace environment to policies Policies can test resource.workspace.environment. The workspace attribute is the workspace entity, which carries its environment; the principal brings its org's workspaces from the State Store, so the dashboard and the daemon's run checks describe a resource's workspace without another lookup, and the daemon now authorizes runs in the task's or binding's workspace. Issue: archie-core-kgxw.10 * fix(webui): ask read_secret when a source secret is read Rotating a source's secret returns its value, but the route asked update, which every developer holds, so no policy could narrow who reads a secret. It now asks read_secret on the secret, which the shipped roles give admins and an object policy can narrow further. Issue: archie-core-kgxw.17 * refactor(org): delete the unused upgrade-incomplete sentinel Nothing returns org.ErrUpgradeIncomplete: the State Store finishes the org upgrade before it serves. Issue: archie-core-kgxw.18
311 lines
8.9 KiB
Go
311 lines
8.9 KiB
Go
// Package mapping owns the payload field-mapping vocabulary and its
|
|
// resolve/preview rule: binding named fields to JSON paths inside a
|
|
// captured webhook payload, and deciding -- loudly -- when a path a mapping
|
|
// depends on has drifted out from under it. See
|
|
package mapping
|
|
|
|
import (
|
|
"bytes"
|
|
"encoding/json"
|
|
"errors"
|
|
"fmt"
|
|
"strconv"
|
|
"strings"
|
|
"time"
|
|
|
|
"github.com/samcharles93/archie-core/internal/domain/eventtype"
|
|
"github.com/samcharles93/archie-core/internal/domain/org"
|
|
)
|
|
|
|
// FieldType is the expected JSON shape of a mapped field, checked at
|
|
// resolve time.
|
|
type FieldType string
|
|
|
|
const (
|
|
TypeString FieldType = "string"
|
|
TypeNumber FieldType = "number"
|
|
TypeBool FieldType = "bool"
|
|
TypeObject FieldType = "object"
|
|
TypeArray FieldType = "array"
|
|
TypeAny FieldType = "any"
|
|
)
|
|
|
|
// Failure reasons. A field either resolves to a value or fails for exactly
|
|
// one of these -- never both.
|
|
const (
|
|
ReasonMissing = "missing"
|
|
ReasonNull = "null"
|
|
ReasonTypeMismatch = "type_mismatch"
|
|
)
|
|
|
|
// Field binds one named field the workflow receives to a path into a
|
|
// captured payload. Required=false suppresses ReasonMissing only -- a
|
|
// present-but-wrong-shape value always fails, so a mapping never silently
|
|
// hands an agent a value it didn't expect.
|
|
type Field struct {
|
|
Name string `json:"name"`
|
|
Path string `json:"path"`
|
|
Type FieldType `json:"type"`
|
|
Required bool `json:"required"`
|
|
}
|
|
|
|
// Mapping is a named set of field bindings belonging to one event type and
|
|
// checked against its schema (CheckSchema). Any number of bindings may reuse
|
|
// it. MatchCount and LastMatchedAt are read-only: the store counts each
|
|
// capture the mapping resolved at dispatch.
|
|
type Mapping struct {
|
|
ID string `json:"id"`
|
|
Name string `json:"name"`
|
|
SourceHint string `json:"source_hint"`
|
|
EventTypeID string `json:"event_type_id"`
|
|
Fields []Field `json:"fields"`
|
|
// WorkspaceID is its event type's workspace; the store sets it.
|
|
WorkspaceID org.WorkspaceID `json:"workspace_id"`
|
|
MatchCount int64 `json:"match_count"`
|
|
LastMatchedAt time.Time `json:"last_matched_at"`
|
|
CreatedAt time.Time `json:"created_at"`
|
|
UpdatedAt time.Time `json:"updated_at"`
|
|
}
|
|
|
|
// validTypes is the set of FieldType values Validate accepts.
|
|
var validTypes = map[FieldType]bool{
|
|
TypeString: true, TypeNumber: true, TypeBool: true,
|
|
TypeObject: true, TypeArray: true, TypeAny: true,
|
|
}
|
|
|
|
// Validate checks a mapping is well-formed before it is persisted or
|
|
// previewed: a name, at least one field, and every field naming a
|
|
// non-empty field name, a non-empty path, and a recognised type. It does
|
|
// not check the mapping against any payload -- that is Resolve's job.
|
|
func (m Mapping) Validate() error {
|
|
if strings.TrimSpace(m.Name) == "" {
|
|
return fmt.Errorf("mapping: name is required")
|
|
}
|
|
if len(m.Fields) == 0 {
|
|
return fmt.Errorf("mapping: at least one field is required")
|
|
}
|
|
seen := make(map[string]bool, len(m.Fields))
|
|
for i, f := range m.Fields {
|
|
if strings.TrimSpace(f.Name) == "" {
|
|
return fmt.Errorf("mapping: field %d: name is required", i)
|
|
}
|
|
if strings.TrimSpace(f.Path) == "" {
|
|
return fmt.Errorf("mapping: field %d (%s): path is required", i, f.Name)
|
|
}
|
|
if !validTypes[f.Type] {
|
|
return fmt.Errorf("mapping: field %d (%s): unrecognised type %q", i, f.Name, f.Type)
|
|
}
|
|
// A duplicate name would let one field silently overwrite another's
|
|
// resolved value in Resolve's values map -- reject it here instead
|
|
// of losing data silently at resolve time.
|
|
if seen[f.Name] {
|
|
return fmt.Errorf("mapping: field %d: duplicate field name %q", i, f.Name)
|
|
}
|
|
seen[f.Name] = true
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// Failure describes one field that did not resolve cleanly against a payload --
|
|
// named by field and path so the operator knows exactly what drifted.
|
|
type Failure struct {
|
|
FieldName string `json:"field_name"`
|
|
Path string `json:"path"`
|
|
Reason string `json:"reason"`
|
|
}
|
|
|
|
// Resolve walks payload independently for each field, so one field's
|
|
// failure never stops the others from resolving -- preview must show every
|
|
// problem at once, not the first one. A field that fails never appears in
|
|
// values.
|
|
func Resolve(fields []Field, payload []byte) (values map[string]any, failures []Failure) {
|
|
values = make(map[string]any, len(fields))
|
|
|
|
var root any
|
|
dec := json.NewDecoder(bytes.NewReader(payload))
|
|
dec.UseNumber()
|
|
if err := dec.Decode(&root); err != nil {
|
|
for _, f := range fields {
|
|
failures = append(failures, Failure{FieldName: f.Name, Path: f.Path, Reason: ReasonMissing})
|
|
}
|
|
return values, failures
|
|
}
|
|
|
|
for _, f := range fields {
|
|
v, found := walk(root, parsePath(f.Path))
|
|
if !found {
|
|
if f.Required {
|
|
failures = append(failures, Failure{FieldName: f.Name, Path: f.Path, Reason: ReasonMissing})
|
|
}
|
|
continue
|
|
}
|
|
if v == nil {
|
|
failures = append(failures, Failure{FieldName: f.Name, Path: f.Path, Reason: ReasonNull})
|
|
continue
|
|
}
|
|
if !typeMatches(v, f.Type) {
|
|
failures = append(failures, Failure{FieldName: f.Name, Path: f.Path, Reason: ReasonTypeMismatch})
|
|
continue
|
|
}
|
|
values[f.Name] = normalize(v)
|
|
}
|
|
|
|
return values, failures
|
|
}
|
|
|
|
// pathSegment is a map key, an array index, or Invalid. walk always fails on
|
|
// Invalid.
|
|
type pathSegment struct {
|
|
Key string
|
|
Index int
|
|
Invalid bool
|
|
}
|
|
|
|
// parsePath splits "data.items[0].id" into
|
|
// [{Key:"data"} {Key:"items"} {Index:0} {Key:"id"}]. No wildcards, no
|
|
// filters -- the grammar is exactly what a click-through UI generates.
|
|
func parsePath(path string) []pathSegment {
|
|
var segs []pathSegment
|
|
for part := range strings.SplitSeq(path, ".") {
|
|
if part == "" {
|
|
continue
|
|
}
|
|
for {
|
|
open := strings.IndexByte(part, '[')
|
|
if open < 0 {
|
|
if part != "" {
|
|
segs = append(segs, pathSegment{Key: part, Index: -1})
|
|
}
|
|
break
|
|
}
|
|
if open > 0 {
|
|
segs = append(segs, pathSegment{Key: part[:open], Index: -1})
|
|
}
|
|
closeIdx := strings.IndexByte(part[open:], ']')
|
|
if closeIdx < 0 {
|
|
segs = append(segs, pathSegment{Invalid: true})
|
|
break
|
|
}
|
|
closeIdx += open
|
|
idx, err := strconv.Atoi(part[open+1 : closeIdx])
|
|
if err != nil {
|
|
segs = append(segs, pathSegment{Invalid: true})
|
|
} else {
|
|
segs = append(segs, pathSegment{Index: idx})
|
|
}
|
|
part = part[closeIdx+1:]
|
|
if part == "" {
|
|
break
|
|
}
|
|
}
|
|
}
|
|
return segs
|
|
}
|
|
|
|
// walk descends node by path. found=false means the path does not exist in
|
|
// the payload at all (missing), which is distinct from resolving to a JSON
|
|
// null (node == nil, found == true).
|
|
func walk(node any, path []pathSegment) (value any, found bool) {
|
|
cur := node
|
|
for _, seg := range path {
|
|
if seg.Invalid {
|
|
return nil, false
|
|
}
|
|
if cur == nil {
|
|
return nil, false
|
|
}
|
|
if seg.Index >= 0 {
|
|
arr, ok := cur.([]any)
|
|
if !ok || seg.Index >= len(arr) {
|
|
return nil, false
|
|
}
|
|
cur = arr[seg.Index]
|
|
continue
|
|
}
|
|
obj, ok := cur.(map[string]any)
|
|
if !ok {
|
|
return nil, false
|
|
}
|
|
v, ok := obj[seg.Key]
|
|
if !ok {
|
|
return nil, false
|
|
}
|
|
cur = v
|
|
}
|
|
return cur, true
|
|
}
|
|
|
|
// ErrSchema marks a mapping whose field paths or types disagree with its
|
|
// event type's inferred schema.
|
|
var ErrSchema = errors.New("mapping: field does not fit the event type's schema")
|
|
|
|
// CheckSchema refuses fields whose path the event type's schema lacks, or
|
|
// whose declared type differs from the schema's. Array indices match the
|
|
// schema's "[]" segment, and a path the example held as null accepts any type.
|
|
func CheckSchema(fields []Field, schema map[string]eventtype.ValueType) error {
|
|
for _, f := range fields {
|
|
got, ok := schema[schemaPath(f.Path)]
|
|
if !ok {
|
|
return fmt.Errorf("%w: %s: path %q is not in the event type", ErrSchema, f.Name, f.Path)
|
|
}
|
|
if f.Type != TypeAny && got != eventtype.TypeNull && string(got) != string(f.Type) {
|
|
return fmt.Errorf("%w: %s: path %q is %s, not %s", ErrSchema, f.Name, f.Path, got, f.Type)
|
|
}
|
|
}
|
|
return nil
|
|
}
|
|
|
|
// schemaPath rewrites a resolve path ("items[0].id") into the schema's form
|
|
// ("items[].id"). A malformed path has no schema form.
|
|
func schemaPath(path string) string {
|
|
var b strings.Builder
|
|
for _, seg := range parsePath(path) {
|
|
switch {
|
|
case seg.Invalid:
|
|
return ""
|
|
case seg.Index >= 0:
|
|
b.WriteString("[]")
|
|
default:
|
|
if b.Len() > 0 {
|
|
b.WriteByte('.')
|
|
}
|
|
b.WriteString(seg.Key)
|
|
}
|
|
}
|
|
return b.String()
|
|
}
|
|
|
|
func typeMatches(v any, want FieldType) bool {
|
|
if want == TypeAny {
|
|
return true
|
|
}
|
|
switch v.(type) {
|
|
case string:
|
|
return want == TypeString
|
|
case json.Number:
|
|
return want == TypeNumber
|
|
case bool:
|
|
return want == TypeBool
|
|
case map[string]any:
|
|
return want == TypeObject
|
|
case []any:
|
|
return want == TypeArray
|
|
default:
|
|
return false
|
|
}
|
|
}
|
|
|
|
// normalize converts json.Number to float64 so resolved values are ordinary
|
|
// JSON-encodable Go types for the preview response, matching
|
|
// encoding/json's own default number decoding.
|
|
func normalize(v any) any {
|
|
if n, ok := v.(json.Number); ok {
|
|
f, err := n.Float64()
|
|
if err != nil {
|
|
return fmt.Sprint(n)
|
|
}
|
|
return f
|
|
}
|
|
return v
|
|
}
|