feat(statestore): step_executions, StartStep/FinishStep under the run credential
Plan item 3 of docs/prds/execution-tree-state-machine.md (archie-core-1220), on top of the transition tables and ErrIllegalTransition enforcement (1214/1215, archie-core-qdbo.5): - Migration 0017 adds step_executions -- id, org_id, workspace_id, execution_id, attempt, parent_id, depth, kind, name, called_execution_id, status, detail, tokens_used, started_at, finished_at, indexed on (execution_id, attempt, id). Ownership fields are stamped from the execution's row; depth is derived from the parent, never supplied by a caller. Existing rows gain no backfilled steps. - task.Store (the workflow engine's mid-run facade) gains StartStep/FinishStep (5 methods, within the 8 cap), with storecontract.StepRecorder fronting them on the server; deadlineStore wraps both with its per-call timeout. - The store enforces each write as one transaction: the guarded row write under the shared step transition table plus the transition's own stage_start/stage_finish event row. Staleness is decided before legality, the way guardTransition decides it, and the store returns the persisted event for the caller's post-commit publish -- the EmitDurable convention (the daemon's event sink skips already-persisted rows by assigned ID), which is why workflow.Run stops emitting stage events beside the store and no longer discards step-write errors: a failed start or finish parks the execution, and the stage does not run unrecorded. An interrupted stage records interrupted rather than failed; a recording failure on that shutdown path is logged and left to RecoverStale, which owns the interrupted-execution sweep in plan item 6. - The run credential of docs/prds/orgs-and-access.md authorises StartStep and FinishStep on its own execution only: both requests carry the execution ID the grant check verifies (FinishStep carries it because the interceptor never reads the database), and FinishStep verifies handler-side that the step really is that execution's. authorizesTaskScopedCall's sanctioned widenings are now held as a lookup table so the next one stops growing a switch. - The PRD's wire contract is StartStep(execution, attempt, parent, kind, name) -> step id; the responses also carry the persisted event (needed for the post-commit publish) and FinishStep's request carries execution_id (needed for the grant check) -- both noted here as deliberate additions to the PRD's shorthand. called_execution_id exists in the table but not yet on the wire: its named consumer is the workflow-call recording of plan item 4 (#1216). Conformance drives both adapters; grant tests pin the own-execution scoping; the wire sentinel set gains ErrInvalidStep (InvalidArgument, "step execution invalid") for a step kind no consumer names.
This commit is contained in:
29 files changed
+2637
-937
No files matched your search
@@ -484,7 +484,7 @@
|
||||
"url": "/dev/prds/state-store-contract/",
|
||||
"title": "State Store — contract boundary \u0026 transport (ratification)",
|
||||
"section": "PRDs",
|
||||
"body": "# State Store — contract boundary \u0026 transport (ratification)\n\n**Status:** Approved\n**Revision:** 2e\n**Date:** 2026-09-06\n\nThis is the pre-implementation design document that Phase 2 subtasks `.4.2` (generalise\n`storerpc` as the State Store gRPC contract) and `.4.3` (stand up `archie-state-store`)\nimplement against.\n**Beads milestone:** archie-core-8cda.4.1\n**Parent:** `docs/prds/service-decomposition.md` (open-question Q4 **RESOLVED**)\n\nThis document fixes the **contract boundary** and **transport** for the State Store service.\nIt does not redesign Archie's domain model, does not invent a second store, and does **not**\nsplit `archie.db`. It records method ownership, DTO/wire representation, error semantics,\nlimits, mode/config shape, local-adapter compatibility, and cutover/deletion rules so that\n`.4.2` and `.4.3` have exactly one authoritative surface to build against.\n\n\u003e **Storage superseded.** The store behind this contract is PostgreSQL:\n\u003e `internal/infrastructure/postgres` implements the producer-owned interfaces in\n\u003e `internal/domain/storecontract` and `workflow.Store`, over the database\n\u003e `database_url` names. `internal/store`, the SQLite file and the local adapter\n\u003e are gone; where this document says `archie.db`, `*store.Store` or\n\u003e `internal/store`, read the State Store's PostgreSQL database and\n\u003e `internal/infrastructure/postgres`. The contract, ownership split, error\n\u003e semantics and cutover rules stand.\n\n\u003e **Current state — read before `.4.2` starts.** The §12 step 2 relocation (`archie-core-8cda.\n4.8`) has landed: `Task`/`Status*`/`Source*` and the 3-method `workflow.Store` now live in\n\u003e `internal/domain/workflow`, the six production workflow files import no `internal/store`\n\u003e symbol, and `storerpc`/the NATS transport assert against `workflow.Store` instead of\n\u003e `store.WorkflowStore`. The broader Phase 2 acceptance criterion (State Store as its own\n\u003e service behind gRPC) is still open — that is what `.4.2`/`.4.3` build next.\n\n\u003e **Relationship to migration-decisions §4 — not replaced.** `docs/architecture/\nmigration-decisions.md` §4 (`store.Task → WorkflowExecution`, `mutable Stage →\nStepExecution history`, `RetryCount/Attempt → Attempt records`, versioned Workflow defs)\n\u003e **remains the authoritative, open document for the long-term Workflow migration.** This PRD\n\u003e does **not** replace it. It only pulls the minimal `Task`/`Status`/`Source` type + interface\n\u003e relocation forward as a Phase 2 prerequisite (because the acceptance criterion requires it);\n\u003e the rest of §4 stays its own migration and stays open.\n\n---\n\n## 1. Decision (the invariant everything follows)\n\nArchie is composed such that a single **`archie-state-store`** service owns the one existing\nSQLite file (`archie.db`) and exposes it over **gRPC behind narrow typed contracts**. There is\nexactly one store, one file, one service. No database-per-service, no distributed\ntransactions, no second store abstraction.\n\nTwo adapter families satisfy the contract interfaces:\n\n- **Local adapter** — `*store.Store` (the in-process SQLite implementation).\n- **Remote adapter** — `*staterpc.Client` (wraps the generated gRPC client).\n\nEach adapter carries an explicit compile-time assertion against the contract interfaces it\nserves. Composition picks local vs. remote by the presence of `[services.state].target`\n(empty = local; set = dial the gRPC service). Default is **local** (the State Store is not yet\nextracted), which is the PRD §3 posture; it differs from the gateway's remote-default only\nbecause the gateway is already a separate deployment and the State Store is not.\n\n\u003e **Contract-ownership fork, decided (rev. 2c — resolves the domain contradiction).**\n\u003e Ownership splits by the consumer's _layer_, per the dependency contract's rule #2\n\u003e (\"a domain defines the smallest interfaces required to perform its work\") and rule #7\n\u003e (\"database rows … MUST NOT leak into domain models\"):\n\u003e\n\u003e - The **workflow domain** (`internal/domain/workflow`) is the only domain-layer consumer of\n\u003e store state. It defines its own **consumer-owned** narrow `Store` interface\n\u003e (Update/Transition/InsertEvent) and its own **domain-owned** `Task` type plus\n\u003e `Status`/`Source` vocabulary. It imports **zero** `internal/store` symbols — satisfying the\n\u003e Phase 2 acceptance criterion. This is the minimal bound of migration-decisions §4\n\u003e (`store.Task → WorkflowExecution`), pulled forward **because** the acceptance criterion\n\u003e requires it.\n\u003e - The **daemon / webui / intake** store surfaces (`TaskStore`, `CaptureStore`,\n\u003e `MappingStore`, `BindingStore`, `BindingDispatcher`, `BindingTaskCreator`) stay\n\u003e **producer-owned** in `internal/store`, because those consumers are application Line truncated
|
||||
"body": "# State Store — contract boundary \u0026 transport (ratification)\n\n**Status:** Approved\n**Revision:** 2e\n**Date:** 2026-09-06\n\nThis is the pre-implementation design document that Phase 2 subtasks `.4.2` (generalise\n`storerpc` as the State Store gRPC contract) and `.4.3` (stand up `archie-state-store`)\nimplement against.\n**Beads milestone:** archie-core-8cda.4.1\n**Parent:** `docs/prds/service-decomposition.md` (open-question Q4 **RESOLVED**)\n\nThis document fixes the **contract boundary** and **transport** for the State Store service.\nIt does not redesign Archie's domain model, does not invent a second store, and does **not**\nsplit `archie.db`. It records method ownership, DTO/wire representation, error semantics,\nlimits, mode/config shape, local-adapter compatibility, and cutover/deletion rules so that\n`.4.2` and `.4.3` have exactly one authoritative surface to build against.\n\n\u003e **Storage superseded.** The store behind this contract is PostgreSQL:\n\u003e `internal/infrastructure/postgres` implements the producer-owned interfaces in\n\u003e `internal/domain/storecontract` and `workflow.Store`, over the database\n\u003e `database_url` names. `internal/store`, the SQLite file and the local adapter\n\u003e are gone; where this document says `archie.db`, `*store.Store` or\n\u003e `internal/store`, read the State Store's PostgreSQL database and\n\u003e `internal/infrastructure/postgres`. The contract, ownership split, error\n\u003e semantics and cutover rules stand.\n\n\u003e **Current state — read before `.4.2` starts.** The §12 step 2 relocation (`archie-core-8cda.\n4.8`) has landed: `Task`/`Status*`/`Source*` and the 3-method `workflow.Store` now live in\n\u003e `internal/domain/workflow`, the six production workflow files import no `internal/store`\n\u003e symbol, and `storerpc`/the NATS transport assert against `workflow.Store` instead of\n\u003e `store.WorkflowStore`. The broader Phase 2 acceptance criterion (State Store as its own\n\u003e service behind gRPC) is still open — that is what `.4.2`/`.4.3` build next.\n\n\u003e **Relationship to migration-decisions §4 — not replaced.** `docs/architecture/\nmigration-decisions.md` §4 (`store.Task → WorkflowExecution`, `mutable Stage →\nStepExecution history`, `RetryCount/Attempt → Attempt records`, versioned Workflow defs)\n\u003e **remains the authoritative, open document for the long-term Workflow migration.** This PRD\n\u003e does **not** replace it. It only pulls the minimal `Task`/`Status`/`Source` type + interface\n\u003e relocation forward as a Phase 2 prerequisite (because the acceptance criterion requires it);\n\u003e the rest of §4 stays its own migration and stays open.\n\n---\n\n## 1. Decision (the invariant everything follows)\n\nArchie is composed such that a single **`archie-state-store`** service owns the one existing\nSQLite file (`archie.db`) and exposes it over **gRPC behind narrow typed contracts**. There is\nexactly one store, one file, one service. No database-per-service, no distributed\ntransactions, no second store abstraction.\n\nTwo adapter families satisfy the contract interfaces:\n\n- **Local adapter** — `*store.Store` (the in-process SQLite implementation).\n- **Remote adapter** — `*staterpc.Client` (wraps the generated gRPC client).\n\nEach adapter carries an explicit compile-time assertion against the contract interfaces it\nserves. Composition picks local vs. remote by the presence of `[services.state].target`\n(empty = local; set = dial the gRPC service). Default is **local** (the State Store is not yet\nextracted), which is the PRD §3 posture; it differs from the gateway's remote-default only\nbecause the gateway is already a separate deployment and the State Store is not.\n\n\u003e **Contract-ownership fork, decided (rev. 2c — resolves the domain contradiction).**\n\u003e Ownership splits by the consumer's _layer_, per the dependency contract's rule #2\n\u003e (\"a domain defines the smallest interfaces required to perform its work\") and rule #7\n\u003e (\"database rows … MUST NOT leak into domain models\"):\n\u003e\n\u003e - The **workflow domain** (`internal/domain/workflow`) is the only domain-layer consumer of\n\u003e store state. It defines its own **consumer-owned** narrow `Store` interface\n\u003e (Update/Transition/InsertEvent) and its own **domain-owned** `Task` type plus\n\u003e `Status`/`Source` vocabulary. It imports **zero** `internal/store` symbols — satisfying the\n\u003e Phase 2 acceptance criterion. This is the minimal bound of migration-decisions §4\n\u003e (`store.Task → WorkflowExecution`), pulled forward **because** the acceptance criterion\n\u003e requires it.\n\u003e - The **daemon / webui / intake** store surfaces (`TaskStore`, `CaptureStore`,\n\u003e `MappingStore`, `BindingStore`, `BindingDispatcher`, `BindingTaskCreator`) stay\n\u003e **producer-owned** in `internal/store`, because those consumers are application Line truncated
|
||||
},
|
||||
{
|
||||
"url": "/dev/prds/status-health-surface/",
|
||||
|
||||
@@ -266,13 +266,14 @@ _view_, not new RPCs — `Update`/`Transition`/`InsertEvent` are shared with
|
||||
| Dispatch | `ArmedBindingsForSource`, `RecordDispatch`, `ListUndispatchedCaptures` |
|
||||
| Playbook | `RecordPlaybookDispatch`, `DeletePlaybookDispatches` |
|
||||
| WorkflowCaller | `EnqueueCallTask`, `WorkflowCallStatus` |
|
||||
| Step executions | `StartStep`, `FinishStep` |
|
||||
| BindingTaskCreator | `EnqueueBindingTask` |
|
||||
| Config snapshot | `PutConfigSnapshot`, `GetConfigSnapshot` |
|
||||
| Apply status | `PutApplyStatus`, `ListApplyStatus` |
|
||||
| Channel status | `PutChannelStatus`, `ListChannelStatus` |
|
||||
| Task log | `ReadTaskLog`, `StreamTaskLogContent` |
|
||||
|
||||
That is **53 unique contract RPCs** across one service (the `TaskEvents.Close` method is dropped).
|
||||
That is **55 unique contract RPCs** across one service (the `TaskEvents.Close` method is dropped).
|
||||
`Close()` is **excluded** from the wire (it is server lifecycle, not a client call) — see §11.
|
||||
|
||||
The channel-status pair follows the config snapshot's split for the same reason: the writer is the process that HOSTS the channels -- the Messaging Service, which alone observes whether a channel is starting, running or failed -- and the reader is the UI process. Both RPCs are administrative, so a task-scoped grant reaches neither. A channel's state is runtime state rather than a settings document, which is why it is not a `ControlPlaneService` resource kind: a failed channel is not a document anybody edits. Asking for a reload travels on its own surface rather than through this one; see `docs/architecture/migration-decisions.md`, "Channel state to the dashboard".
|
||||
@@ -307,12 +308,18 @@ condition, and the panel that collapsed them told operators task logging was swi
|
||||
it was not. The Go facade is `storecontract.TaskLogStore` (2 methods), carried by
|
||||
`*staterpc.Client` alongside the rest.
|
||||
|
||||
The workflow-caller pair was added by `archie-core-t2db.41` (see
|
||||
`docs/prds/workflow-calls.md`). It is the one surface a task-scoped grant
|
||||
reaches beyond the three task.Store RPCs: a `workflow.call` step must start
|
||||
its callee and, with `wait: true`, read it back, and the callee is the
|
||||
caller's child. Both requests carry the caller's task ID, which the grant
|
||||
check verifies; `EnqueueCallTask` derives everything else from the caller's
|
||||
The step-execution pair was added by the execution tree's plan item 3 (see
|
||||
`docs/prds/execution-tree-state-machine.md`), the surface a task-scoped grant reaches for
|
||||
its own execution beyond the three `task.Store` RPCs and the workflow-caller pair: the
|
||||
container records its stages and its own agent calls with the run credential. Both requests
|
||||
carry the caller's execution ID, which the grant check verifies; `FinishStep` additionally
|
||||
verifies, handler-side, that the step really is that execution's. The store writes each
|
||||
transition's `stage_start`/`stage_finish` event row in the same transaction and returns it
|
||||
for the caller's post-commit bus publish. `called_execution_id` is stored for the `call`
|
||||
kind but not yet carried on the wire: the named consumer is the workflow-call recording
|
||||
that will set it.
|
||||
|
||||
The workflow-caller pair was added by `archie-core-t2db.41` (see everything else from the caller's
|
||||
row and refuses a caller that is not running or a call past the depth limit
|
||||
(`ErrCallCallerNotRunning` / `ErrCallDepthExceeded`), and `WorkflowCallStatus`
|
||||
re-checks the parent-child relation against the row before answering
|
||||
|
||||
@@ -383,6 +383,7 @@ func startStateStoreGRPC(t *testing.T, local *pgstore.TaskDB) string {
|
||||
staterpc.RegisterServer(server, staterpc.Deps{
|
||||
Tasks: local, Captures: eda, Mappings: eda, Bindings: eda,
|
||||
BindingDispatcher: eda, BindingTaskCreator: local, WorkflowCalls: local,
|
||||
Steps: local,
|
||||
})
|
||||
go func() { _ = server.Serve(listener) }()
|
||||
t.Cleanup(func() { server.Stop(); _ = listener.Close() })
|
||||
|
||||
@@ -346,6 +346,9 @@ func (b *boot) stateStoreDeps(grants *staterpc.TaskGrants) staterpc.Deps {
|
||||
if wc, ok := b.st.(storecontract.WorkflowCaller); ok {
|
||||
deps.WorkflowCalls = wc
|
||||
}
|
||||
if sr, ok := b.st.(storecontract.StepRecorder); ok {
|
||||
deps.Steps = sr
|
||||
}
|
||||
b.accessDeps(&deps)
|
||||
if css, ok := b.st.(storecontract.ConfigSnapshotStore); ok {
|
||||
deps.ConfigSnapshots = css
|
||||
|
||||
+1175
-875
File diff suppressed because it is too large.
Load diff
@@ -40,6 +40,8 @@ const (
|
||||
StateStoreService_ClaimByIssue_FullMethodName = "/state.v1.StateStoreService/ClaimByIssue"
|
||||
StateStoreService_Transition_FullMethodName = "/state.v1.StateStoreService/Transition"
|
||||
StateStoreService_Update_FullMethodName = "/state.v1.StateStoreService/Update"
|
||||
StateStoreService_StartStep_FullMethodName = "/state.v1.StateStoreService/StartStep"
|
||||
StateStoreService_FinishStep_FullMethodName = "/state.v1.StateStoreService/FinishStep"
|
||||
StateStoreService_Requeue_FullMethodName = "/state.v1.StateStoreService/Requeue"
|
||||
StateStoreService_ParkTask_FullMethodName = "/state.v1.StateStoreService/ParkTask"
|
||||
StateStoreService_RecoverStale_FullMethodName = "/state.v1.StateStoreService/RecoverStale"
|
||||
@@ -149,6 +151,10 @@ type StateStoreServiceClient interface {
|
||||
ClaimByIssue(ctx context.Context, in *ClaimByIssueRequest, opts ...grpc.CallOption) (*ClaimByIssueResponse, error)
|
||||
Transition(ctx context.Context, in *TransitionRequest, opts ...grpc.CallOption) (*TransitionResponse, error)
|
||||
Update(ctx context.Context, in *UpdateRequest, opts ...grpc.CallOption) (*UpdateResponse, error)
|
||||
// Step executions (docs/prds/execution-tree-state-machine.md). The run
|
||||
// credential authorises both on its own execution only.
|
||||
StartStep(ctx context.Context, in *StartStepRequest, opts ...grpc.CallOption) (*StartStepResponse, error)
|
||||
FinishStep(ctx context.Context, in *FinishStepRequest, opts ...grpc.CallOption) (*FinishStepResponse, error)
|
||||
Requeue(ctx context.Context, in *RequeueRequest, opts ...grpc.CallOption) (*RequeueResponse, error)
|
||||
ParkTask(ctx context.Context, in *ParkTaskRequest, opts ...grpc.CallOption) (*ParkTaskResponse, error)
|
||||
RecoverStale(ctx context.Context, in *RecoverStaleRequest, opts ...grpc.CallOption) (*RecoverStaleResponse, error)
|
||||
@@ -492,6 +498,26 @@ func (c *stateStoreServiceClient) Update(ctx context.Context, in *UpdateRequest,
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) StartStep(ctx context.Context, in *StartStepRequest, opts ...grpc.CallOption) (*StartStepResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(StartStepResponse)
|
||||
err := c.cc.Invoke(ctx, StateStoreService_StartStep_FullMethodName, in, out, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) FinishStep(ctx context.Context, in *FinishStepRequest, opts ...grpc.CallOption) (*FinishStepResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(FinishStepResponse)
|
||||
err := c.cc.Invoke(ctx, StateStoreService_FinishStep_FullMethodName, in, out, cOpts...)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (c *stateStoreServiceClient) Requeue(ctx context.Context, in *RequeueRequest, opts ...grpc.CallOption) (*RequeueResponse, error) {
|
||||
cOpts := append([]grpc.CallOption{grpc.StaticMethod()}, opts...)
|
||||
out := new(RequeueResponse)
|
||||
@@ -1229,6 +1255,10 @@ type StateStoreServiceServer interface {
|
||||
ClaimByIssue(context.Context, *ClaimByIssueRequest) (*ClaimByIssueResponse, error)
|
||||
Transition(context.Context, *TransitionRequest) (*TransitionResponse, error)
|
||||
Update(context.Context, *UpdateRequest) (*UpdateResponse, error)
|
||||
// Step executions (docs/prds/execution-tree-state-machine.md). The run
|
||||
// credential authorises both on its own execution only.
|
||||
StartStep(context.Context, *StartStepRequest) (*StartStepResponse, error)
|
||||
FinishStep(context.Context, *FinishStepRequest) (*FinishStepResponse, error)
|
||||
Requeue(context.Context, *RequeueRequest) (*RequeueResponse, error)
|
||||
ParkTask(context.Context, *ParkTaskRequest) (*ParkTaskResponse, error)
|
||||
RecoverStale(context.Context, *RecoverStaleRequest) (*RecoverStaleResponse, error)
|
||||
@@ -1425,6 +1455,12 @@ func (UnimplementedStateStoreServiceServer) Transition(context.Context, *Transit
|
||||
func (UnimplementedStateStoreServiceServer) Update(context.Context, *UpdateRequest) (*UpdateResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method Update not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) StartStep(context.Context, *StartStepRequest) (*StartStepResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method StartStep not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) FinishStep(context.Context, *FinishStepRequest) (*FinishStepResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method FinishStep not implemented")
|
||||
}
|
||||
func (UnimplementedStateStoreServiceServer) Requeue(context.Context, *RequeueRequest) (*RequeueResponse, error) {
|
||||
return nil, status.Error(codes.Unimplemented, "method Requeue not implemented")
|
||||
}
|
||||
@@ -2025,6 +2061,42 @@ func _StateStoreService_Update_Handler(srv interface{}, ctx context.Context, dec
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_StartStep_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(StartStepRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(StateStoreServiceServer).StartStep(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: StateStoreService_StartStep_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(StateStoreServiceServer).StartStep(ctx, req.(*StartStepRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_FinishStep_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(FinishStepRequest)
|
||||
if err := dec(in); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if interceptor == nil {
|
||||
return srv.(StateStoreServiceServer).FinishStep(ctx, in)
|
||||
}
|
||||
info := &grpc.UnaryServerInfo{
|
||||
Server: srv,
|
||||
FullMethod: StateStoreService_FinishStep_FullMethodName,
|
||||
}
|
||||
handler := func(ctx context.Context, req interface{}) (interface{}, error) {
|
||||
return srv.(StateStoreServiceServer).FinishStep(ctx, req.(*FinishStepRequest))
|
||||
}
|
||||
return interceptor(ctx, in, info, handler)
|
||||
}
|
||||
|
||||
func _StateStoreService_Requeue_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) {
|
||||
in := new(RequeueRequest)
|
||||
if err := dec(in); err != nil {
|
||||
@@ -3301,6 +3373,14 @@ var StateStoreService_ServiceDesc = grpc.ServiceDesc{
|
||||
MethodName: "Update",
|
||||
Handler: _StateStoreService_Update_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "StartStep",
|
||||
Handler: _StateStoreService_StartStep_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "FinishStep",
|
||||
Handler: _StateStoreService_FinishStep_Handler,
|
||||
},
|
||||
{
|
||||
MethodName: "Requeue",
|
||||
Handler: _StateStoreService_Requeue_Handler,
|
||||
|
||||
@@ -65,6 +65,16 @@ type TaskParker interface {
|
||||
ParkTask(ctx context.Context, taskID int64, from, detail, class string) error
|
||||
}
|
||||
|
||||
// StepRecorder records the StepExecutions of one running WorkflowExecution
|
||||
// (docs/prds/execution-tree-state-machine.md). It is deliberately separate
|
||||
// from TaskLifecycle, which is exactly at the interfacebloat cap: the same
|
||||
// two methods also sit on the workflow engine's mid-run facade (task.Store),
|
||||
// which is what a container's run credential calls.
|
||||
type StepRecorder interface {
|
||||
StartStep(ctx context.Context, s task.StepStart) (int64, events.Event, error)
|
||||
FinishStep(ctx context.Context, s task.StepFinish) (events.Event, error)
|
||||
}
|
||||
|
||||
// TaskArchiver removes one terminal task's local record with an optimistic
|
||||
// status guard. It is separate from the already broad lifecycle contract so
|
||||
// consumers that only run tasks do not acquire an operator-only capability.
|
||||
@@ -415,6 +425,12 @@ var (
|
||||
// it. Distinct from ErrStaleTransition, which covers a from that does
|
||||
// not match the row's actual status.
|
||||
ErrIllegalTransition = errors.New("store: illegal transition: status pair is not in the transition table")
|
||||
// ErrInvalidStep is returned when a step-execution request names a kind
|
||||
// the store does not record (docs/prds/execution-tree-state-machine.md:
|
||||
// stage, agent or call; a new kind needs a named consumer before it is
|
||||
// added) or a step whose identifying fields do not resolve. Distinct from
|
||||
// the transition sentinels, which cover a row's state.
|
||||
ErrInvalidStep = errors.New("store: invalid step execution request")
|
||||
// ErrBindingNotFound is returned when a binding ID does not exist.
|
||||
ErrBindingNotFound = errors.New("store: binding not found")
|
||||
// ErrBindingOverlap is returned when a binding's matcher overlaps an
|
||||
|
||||
@@ -21,9 +21,14 @@ import (
|
||||
// events it was asked to persist. A capture is durable provenance, so the
|
||||
// assertions below read this and not the lossy bus.
|
||||
type recordingStore struct {
|
||||
events []events.Event
|
||||
transitions []transition
|
||||
insertEr error
|
||||
events []events.Event
|
||||
transitions []transition
|
||||
steps []StepStart
|
||||
finishes []StepFinish
|
||||
insertEr error
|
||||
stepEr error
|
||||
stepNames map[int64]string
|
||||
stepAttempts map[int64]int
|
||||
}
|
||||
|
||||
// transition is one status change the engine asked the store for. Parking is
|
||||
@@ -46,6 +51,33 @@ func (s *recordingStore) InsertEvent(_ context.Context, e events.Event) (int64,
|
||||
return int64(len(s.events)), nil
|
||||
}
|
||||
|
||||
func (s *recordingStore) StartStep(_ context.Context, start StepStart) (int64, events.Event, error) {
|
||||
if s.stepEr != nil {
|
||||
return 0, events.Event{}, s.stepEr
|
||||
}
|
||||
s.steps = append(s.steps, start)
|
||||
id := int64(len(s.steps))
|
||||
if s.stepNames == nil {
|
||||
s.stepNames = map[int64]string{}
|
||||
s.stepAttempts = map[int64]int{}
|
||||
}
|
||||
s.stepNames[id] = start.Name
|
||||
s.stepAttempts[id] = start.Attempt
|
||||
return id, events.Event{ID: id, Kind: events.KindStageStart, Stage: start.Name, Attempt: start.Attempt}, nil
|
||||
}
|
||||
|
||||
func (s *recordingStore) FinishStep(_ context.Context, finish StepFinish) (events.Event, error) {
|
||||
if s.stepEr != nil {
|
||||
return events.Event{}, s.stepEr
|
||||
}
|
||||
s.finishes = append(s.finishes, finish)
|
||||
id := int64(len(s.finishes))
|
||||
return events.Event{
|
||||
ID: id, Kind: events.KindStageFinish,
|
||||
Stage: s.stepNames[finish.StepID], Attempt: s.stepAttempts[finish.StepID],
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *recordingStore) ofKind(kind string) []events.Event {
|
||||
var out []events.Event
|
||||
for _, e := range s.events {
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
package task
|
||||
|
||||
import "github.com/samcharles93/archie-core/internal/taskstate"
|
||||
|
||||
// The step-execution vocabulary of docs/prds/execution-tree-state-machine.md:
|
||||
// one stage run, or one agent or workflow.call step an execution's tree
|
||||
// records, started and finished through the State Store under the run's own
|
||||
// credential.
|
||||
|
||||
// Step kinds. `stage` is what workflow.Run records for every stage it runs;
|
||||
// `agent` is what the agent runtime records for one agent call; `call` is the
|
||||
// workflow.call step in a caller's tree, which names its callee's execution.
|
||||
// A new kind needs a named consumer before it is added.
|
||||
const (
|
||||
StepKindStage = "stage"
|
||||
StepKindAgent = "agent"
|
||||
StepKindCall = "call"
|
||||
)
|
||||
|
||||
// ValidStepKind reports whether kind is one the store records.
|
||||
func ValidStepKind(kind string) bool {
|
||||
switch kind {
|
||||
case StepKindStage, StepKindAgent, StepKindCall:
|
||||
return true
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
// StepStart names the step a caller is starting. ExecutionID is the
|
||||
// WorkflowExecution (the tasks row); Attempt is the run of it the step
|
||||
// belongs to; ParentID is 0 for a stage and names the enclosing StepExecution
|
||||
// otherwise, from which the store derives depth.
|
||||
type StepStart struct {
|
||||
ExecutionID int64
|
||||
Attempt int
|
||||
// ParentID is 0 when the step is a stage at the tree's root.
|
||||
ParentID int64
|
||||
Kind string
|
||||
Name string
|
||||
}
|
||||
|
||||
// StepFinish moves one step from From to its outcome. ExecutionID names the
|
||||
// run the step belongs to: the task-scoped grant check verifies it, and the
|
||||
// store verifies the step really is that execution's, so a caller cannot move
|
||||
// another run's step by naming it.
|
||||
type StepFinish struct {
|
||||
StepID int64
|
||||
ExecutionID int64
|
||||
// From and To are the step transition table's statuses (taskstate).
|
||||
From taskstate.StepStatus
|
||||
To taskstate.StepStatus
|
||||
// Detail is the step's outcome line: an error's text for a failed step, a
|
||||
// summary otherwise. Clipped store-side like every detail.
|
||||
Detail string
|
||||
// TokensUsed is the usage the step itself accounted. Stages record none;
|
||||
// the agent runtime records its own calls' usage.
|
||||
TokensUsed int64
|
||||
}
|
||||
@@ -160,6 +160,17 @@ type Store interface {
|
||||
Update(ctx context.Context, t *Task) error
|
||||
Transition(ctx context.Context, taskID int64, from, to, detail string) error
|
||||
InsertEvent(ctx context.Context, e events.Event) (int64, error)
|
||||
// StartStep records a StepExecution entering running
|
||||
// (docs/prds/execution-tree-state-machine.md): the store creates the row,
|
||||
// enforces the step transition table and the execution's own status, and
|
||||
// writes the transition's stage_start event in the same transaction. It
|
||||
// returns the step's id and the persisted event, which the caller publishes
|
||||
// to its bus after the write commits.
|
||||
StartStep(ctx context.Context, s StepStart) (int64, events.Event, error)
|
||||
// FinishStep moves one step to its outcome under the same table, writing
|
||||
// the stage_finish event in the same transaction, and returns that event
|
||||
// for the caller's post-commit publish.
|
||||
FinishStep(ctx context.Context, s StepFinish) (events.Event, error)
|
||||
}
|
||||
|
||||
// Definition is the operator-safe snapshot of an executable workflow. It
|
||||
|
||||
@@ -14,10 +14,22 @@ type (
|
||||
// Store is the narrow, consumer-owned subset of the State Store
|
||||
// contract that workflow stages call mid-run.
|
||||
Store = task.Store
|
||||
// StepStart and StepFinish are the step-execution writes the store
|
||||
// records mid-run (docs/prds/execution-tree-state-machine.md).
|
||||
StepStart = task.StepStart
|
||||
StepFinish = task.StepFinish
|
||||
// Definition is the operator-safe snapshot of an executable workflow.
|
||||
Definition = task.Definition
|
||||
)
|
||||
|
||||
// Step-execution kinds, defined beside the step writes in internal/domain/
|
||||
// workflow/task with the same aliasing reason.
|
||||
const (
|
||||
StepKindStage = task.StepKindStage
|
||||
StepKindAgent = task.StepKindAgent
|
||||
StepKindCall = task.StepKindCall
|
||||
)
|
||||
|
||||
// Task lifecycle statuses. Defined in internal/taskstate; these names are
|
||||
// the workflow domain's public spelling.
|
||||
const (
|
||||
|
||||
@@ -7,7 +7,6 @@ package workflow
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"log/slog"
|
||||
"regexp"
|
||||
@@ -20,6 +19,7 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||
"github.com/samcharles93/archie-core/internal/events"
|
||||
"github.com/samcharles93/archie-core/internal/skill"
|
||||
"github.com/samcharles93/archie-core/internal/taskstate"
|
||||
"github.com/samcharles93/archie-core/internal/tools"
|
||||
)
|
||||
|
||||
@@ -304,6 +304,20 @@ func Run(ctx context.Context, wf Workflow, tc *TaskContext) {
|
||||
for _, stage := range wf.Stages {
|
||||
t.Stage = stage.Name
|
||||
_ = tc.Store.Update(ctx, t)
|
||||
// Every stage is a StepExecution (docs/prds/execution-tree-state-machine.md):
|
||||
// the store records pending -> running and writes the stage_start event
|
||||
// in the same transaction, and refuses a start the state machine
|
||||
// forbids. A failed recording write parks the execution -- the step
|
||||
// does not run unrecorded, because a stage whose record never landed
|
||||
// would park on a reason no dashboard can trace to a step row.
|
||||
stepID, startEvent, err := tc.Store.StartStep(ctx, StepStart{
|
||||
ExecutionID: t.ID, Attempt: t.Attempt, Kind: task.StepKindStage, Name: stage.Name,
|
||||
})
|
||||
if err != nil {
|
||||
park(ctx, tc, fmt.Sprintf("stage %s could not be recorded: %v", stage.Name, err))
|
||||
return
|
||||
}
|
||||
publishEvent(tc, startEvent)
|
||||
// The stage is bound onto the task logger for exactly the stage's own
|
||||
// execution and removed afterwards, so every line a stage's code writes
|
||||
// is selectable by stage (internal/logging.Query.Stage). It is restored
|
||||
@@ -315,28 +329,52 @@ func Run(ctx context.Context, wf Workflow, tc *TaskContext) {
|
||||
stageLog := previousLog.With("stage", stage.Name)
|
||||
tc.Log = stageLog
|
||||
stageLog.Info("stage starting")
|
||||
tc.Emit(events.KindStageStart, stage.Name, "", nil)
|
||||
started := time.Now()
|
||||
|
||||
err := stage.Run(ctx, tc)
|
||||
data := map[string]any{"duration_ms": time.Since(started).Milliseconds()}
|
||||
if err != nil {
|
||||
data["error"] = err.Error()
|
||||
if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
|
||||
data["interrupted"] = true
|
||||
}
|
||||
}
|
||||
tc.Emit(events.KindStageFinish, stage.Name, "", data)
|
||||
err = stage.Run(ctx, tc)
|
||||
tc.Log = previousLog
|
||||
|
||||
if err != nil {
|
||||
// Daemon shutdown is not a workflow failure. Leave the task running
|
||||
// so Startup's existing crash recovery requeues it; parking here
|
||||
// would publish a false failure and require manual intervention.
|
||||
if ctx.Err() != nil {
|
||||
stageLog.Info("stage interrupted", "err", err)
|
||||
return
|
||||
// Daemon shutdown is not a workflow failure. The step records the
|
||||
// table's interrupted outcome -- the process knows the step it is
|
||||
// giving up -- and returns without parking: Startup's existing crash
|
||||
// recovery requeues the execution, and the next attempt starts fresh
|
||||
// steps. A recording failure on this path is logged and left to the
|
||||
// same recovery: the process is on its way out either way.
|
||||
if ctx.Err() != nil && err != nil {
|
||||
if finishEvent, recordErr := tc.Store.FinishStep(ctx, StepFinish{
|
||||
StepID: stepID, ExecutionID: t.ID,
|
||||
From: taskstate.StepRunning, To: taskstate.StepInterrupted, Detail: err.Error(),
|
||||
}); recordErr != nil {
|
||||
stageLog.Warn("interrupted step finish could not be recorded", "stage", stage.Name, "err", recordErr)
|
||||
} else {
|
||||
publishEvent(tc, finishEvent)
|
||||
}
|
||||
stageLog.Info("stage interrupted", "err", err)
|
||||
return
|
||||
}
|
||||
|
||||
finishTo := taskstate.StepSucceeded
|
||||
finishDetail := ""
|
||||
if err != nil {
|
||||
finishTo = taskstate.StepFailed
|
||||
finishDetail = err.Error()
|
||||
}
|
||||
// The outcome event is the store's own stage_finish row, written in the
|
||||
// FinishStep transaction -- not a second copy emitted beside it. A
|
||||
// failed recording write parks the execution, like a failed start.
|
||||
finishEvent, recordErr := tc.Store.FinishStep(ctx, StepFinish{
|
||||
StepID: stepID, ExecutionID: t.ID,
|
||||
From: taskstate.StepRunning, To: finishTo, Detail: finishDetail,
|
||||
})
|
||||
if recordErr != nil {
|
||||
stageLog.Error("stage finish could not be recorded", "stage", stage.Name, "err", recordErr)
|
||||
park(ctx, tc, fmt.Sprintf("stage %s could not be recorded: %v", stage.Name, recordErr))
|
||||
return
|
||||
}
|
||||
publishEvent(tc, finishEvent)
|
||||
|
||||
if err != nil {
|
||||
// Parking here publishes the failure; the store has already
|
||||
// recorded the step that produced it.
|
||||
t.ParkReason = fmt.Sprintf("stage %s: %v", stage.Name, err)
|
||||
park(ctx, tc, t.ParkReason)
|
||||
return
|
||||
@@ -351,6 +389,16 @@ func Run(ctx context.Context, wf Workflow, tc *TaskContext) {
|
||||
park(ctx, tc, "workflow ended without an outcome (definition bug)")
|
||||
}
|
||||
|
||||
// publishEvent puts an already-persisted event on the run's bus after its
|
||||
// write committed: the assigned ID is what the daemon's event sink reads to
|
||||
// skip the second insert, the same convention EmitDurable established.
|
||||
func publishEvent(tc *TaskContext, e events.Event) {
|
||||
if tc.Bus == nil || e.ID == 0 {
|
||||
return
|
||||
}
|
||||
tc.Bus.Publish(e)
|
||||
}
|
||||
|
||||
func finish(ctx context.Context, tc *TaskContext, log *slog.Logger) {
|
||||
t := tc.Task
|
||||
if tc.Outcome.Status == StatusParked {
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
package workflow
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"slices"
|
||||
"testing"
|
||||
|
||||
task "github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||
"github.com/samcharles93/archie-core/internal/events"
|
||||
)
|
||||
|
||||
// The stage-recording half of docs/prds/execution-tree-state-machine.md:
|
||||
// workflow.Run records every stage it runs as a StepExecution through the
|
||||
// store, and a failed step write parks the execution -- the step does not run
|
||||
// unrecorded.
|
||||
|
||||
// TestRunRecordsEveryStageAsAStepExecution pins the recording shape: one
|
||||
// StartStep before each stage's body and one FinishStep with that stage's
|
||||
// outcome, both of kind stage, against the task's own execution and attempt.
|
||||
func TestRunRecordsEveryStageAsAStepExecution(t *testing.T) {
|
||||
store := &recordingStore{}
|
||||
stage := func(name string) Stage {
|
||||
return Stage{Name: name, Run: func(context.Context, *TaskContext) error { return nil }}
|
||||
}
|
||||
wf := Workflow{Name: "recording", Stages: []Stage{stage("plan"), stage("implement")}}
|
||||
tc := &TaskContext{
|
||||
Task: &Task{ID: 41, Attempt: 3, Owner: "acme", Repo: "widgets", Status: StatusRunning},
|
||||
Store: store, Log: slog.New(slog.DiscardHandler),
|
||||
}
|
||||
|
||||
Run(context.Background(), wf, tc)
|
||||
|
||||
if len(store.steps) != 2 || len(store.finishes) != 2 {
|
||||
t.Fatalf("steps = %+v finishes = %+v, want one recorded start and finish per stage", store.steps, store.finishes)
|
||||
}
|
||||
for i, step := range store.steps {
|
||||
if step.ExecutionID != 41 || step.Attempt != 3 {
|
||||
t.Errorf("stage %d: ExecutionID/Attempt = %d/%d, want the run's own 41/3", i, step.ExecutionID, step.Attempt)
|
||||
}
|
||||
if step.Kind != task.StepKindStage {
|
||||
t.Errorf("stage %d: kind = %q, want %q", i, step.Kind, task.StepKindStage)
|
||||
}
|
||||
if want := []string{"plan", "implement"}[i]; step.Name != want {
|
||||
t.Errorf("stage %d: name = %q, want %q", i, step.Name, want)
|
||||
}
|
||||
}
|
||||
if got := store.finishes[0].To; got != "succeeded" {
|
||||
t.Errorf("stage finish to = %q, want succeeded", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestRunPublishesTheRecordedEvents pins the post-commit half: the events the
|
||||
// store returned, already persisted, are what reach the bus -- not a second,
|
||||
// unpersisted copy emitted beside the store write.
|
||||
func TestRunPublishesThePersistedStepEvents(t *testing.T) {
|
||||
store := &recordingStore{}
|
||||
wf := Workflow{Name: "recording", Stages: []Stage{{
|
||||
Name: "implement", Run: func(context.Context, *TaskContext) error { return nil },
|
||||
}}}
|
||||
bus := events.NewBus()
|
||||
sub := bus.Subscribe(16)
|
||||
defer sub.Close()
|
||||
tc := &TaskContext{
|
||||
Task: &Task{ID: 7, Attempt: 1, Owner: "acme", Repo: "widgets", Status: StatusRunning},
|
||||
Store: store, Bus: bus, Log: slog.New(slog.DiscardHandler),
|
||||
}
|
||||
|
||||
Run(context.Background(), wf, tc)
|
||||
|
||||
var seen []string
|
||||
ids := map[string]int64{}
|
||||
for len(sub.C) > 0 {
|
||||
e := <-sub.C
|
||||
seen = append(seen, e.Kind)
|
||||
ids[e.Kind] = e.ID
|
||||
}
|
||||
for _, want := range []string{events.KindStageStart, events.KindStageFinish} {
|
||||
if !slices.Contains(seen, want) {
|
||||
t.Errorf("bus saw %v, want %q among them: the persisted events are what reach the bus", seen, want)
|
||||
}
|
||||
if ids[want] == 0 {
|
||||
t.Errorf("bus event %q carries no ID: an event without the store's assigned ID would be inserted a second time by the daemon's event sink", want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// TestRunParksWhenAStepWriteFails is the PRD's persistence rule: a failed
|
||||
// step write parks the execution, and the step does not run unrecorded.
|
||||
func TestRunParksWhenAStepWriteFails(t *testing.T) {
|
||||
store := &recordingStore{stepEr: errStepWrite}
|
||||
stageRan := false
|
||||
wf := Workflow{Name: "recording", Stages: []Stage{{
|
||||
Name: "implement",
|
||||
Run: func(context.Context, *TaskContext) error { stageRan = true; return nil },
|
||||
}}}
|
||||
tc := &TaskContext{
|
||||
Task: &Task{ID: 9, Attempt: 1, Owner: "acme", Repo: "widgets", Status: StatusRunning},
|
||||
Store: store, Log: slog.New(slog.DiscardHandler),
|
||||
}
|
||||
|
||||
Run(context.Background(), wf, tc)
|
||||
|
||||
if stageRan {
|
||||
t.Fatal("the stage ran despite its recording write failing; the step must not run unrecorded")
|
||||
}
|
||||
if len(store.transitions) == 0 || store.transitions[0].to != StatusParked {
|
||||
t.Fatalf("transitions = %+v, want the execution parked", store.transitions)
|
||||
}
|
||||
if len(store.finishes) != 0 {
|
||||
t.Errorf("finishes = %+v, want none: the stage never ran", store.finishes)
|
||||
}
|
||||
}
|
||||
|
||||
var errStepWrite = errors.New("recording write failed")
|
||||
@@ -221,6 +221,18 @@ func (d deadlineStore) InsertEvent(ctx context.Context, e events.Event) (int64,
|
||||
return d.Store.InsertEvent(ctx, e)
|
||||
}
|
||||
|
||||
func (d deadlineStore) StartStep(ctx context.Context, s task.StepStart) (int64, events.Event, error) {
|
||||
ctx, cancel := d.withDeadline(ctx)
|
||||
defer cancel()
|
||||
return d.Store.StartStep(ctx, s)
|
||||
}
|
||||
|
||||
func (d deadlineStore) FinishStep(ctx context.Context, s task.StepFinish) (events.Event, error) {
|
||||
ctx, cancel := d.withDeadline(ctx)
|
||||
defer cancel()
|
||||
return d.Store.FinishStep(ctx, s)
|
||||
}
|
||||
|
||||
// Trees constructs the identity-scoped worktree RPC client.
|
||||
func (t *Transport) Trees(identity, grant string, timeout time.Duration) RemoteTrees {
|
||||
return &worktreerpc.Client{Conn: t.conn, Timeout: timeout, Identity: identity, Grant: grant}
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
-- Step executions: one durable record per stage run and per agent or
|
||||
-- workflow.call step inside a WorkflowExecution
|
||||
-- (docs/prds/execution-tree-state-machine.md). Statuses are taskstate's
|
||||
-- StepStatus vocabulary and move only through the step transition table the
|
||||
-- store enforces; depth is derived from parent_id, never supplied by a
|
||||
-- caller. Existing rows gain no backfilled steps: their stage history stays
|
||||
-- in transitions and events.
|
||||
|
||||
-- +goose Up
|
||||
|
||||
CREATE TABLE step_executions (
|
||||
id bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
org_id text NOT NULL DEFAULT 'default',
|
||||
workspace_id text NOT NULL DEFAULT 'default',
|
||||
execution_id bigint NOT NULL,
|
||||
attempt bigint NOT NULL,
|
||||
parent_id bigint,
|
||||
depth int NOT NULL DEFAULT 0,
|
||||
kind text NOT NULL,
|
||||
name text NOT NULL,
|
||||
called_execution_id bigint NOT NULL DEFAULT 0,
|
||||
status text NOT NULL DEFAULT 'pending',
|
||||
detail text NOT NULL DEFAULT '',
|
||||
tokens_used bigint NOT NULL DEFAULT 0,
|
||||
started_at timestamptz,
|
||||
finished_at timestamptz
|
||||
);
|
||||
CREATE INDEX idx_step_executions_attempt ON step_executions (execution_id, attempt, id);
|
||||
|
||||
-- +goose Down
|
||||
|
||||
DROP TABLE step_executions;
|
||||
@@ -306,6 +306,24 @@ type Source struct {
|
||||
WorkspaceID string
|
||||
}
|
||||
|
||||
type StepExecution struct {
|
||||
ID int64
|
||||
OrgID string
|
||||
WorkspaceID string
|
||||
ExecutionID int64
|
||||
Attempt int64
|
||||
ParentID pgtype.Int8
|
||||
Depth int32
|
||||
Kind string
|
||||
Name string
|
||||
CalledExecutionID int64
|
||||
Status string
|
||||
Detail string
|
||||
TokensUsed int64
|
||||
StartedAt pgtype.Timestamptz
|
||||
FinishedAt pgtype.Timestamptz
|
||||
}
|
||||
|
||||
type SysAudit struct {
|
||||
ID int64
|
||||
At time.Time
|
||||
|
||||
@@ -0,0 +1,165 @@
|
||||
// Code generated by sqlc. DO NOT EDIT.
|
||||
// versions:
|
||||
// sqlc v1.31.1
|
||||
// source: steps.sql
|
||||
|
||||
package postgresdb
|
||||
|
||||
import (
|
||||
"context"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
)
|
||||
|
||||
const finishStepExecution = `-- name: FinishStepExecution :execrows
|
||||
UPDATE step_executions
|
||||
SET status = $1, detail = $2, tokens_used = $3, finished_at = now()
|
||||
WHERE id = $4 AND status = $5
|
||||
`
|
||||
|
||||
type FinishStepExecutionParams struct {
|
||||
Status string
|
||||
Detail string
|
||||
TokensUsed int64
|
||||
ID int64
|
||||
Status_2 string
|
||||
}
|
||||
|
||||
func (q *Queries) FinishStepExecution(ctx context.Context, arg FinishStepExecutionParams) (int64, error) {
|
||||
result, err := q.db.Exec(ctx, finishStepExecution,
|
||||
arg.Status,
|
||||
arg.Detail,
|
||||
arg.TokensUsed,
|
||||
arg.ID,
|
||||
arg.Status_2,
|
||||
)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return result.RowsAffected(), nil
|
||||
}
|
||||
|
||||
const insertStepExecution = `-- name: InsertStepExecution :one
|
||||
INSERT INTO step_executions (org_id, workspace_id, execution_id, attempt, parent_id, depth, kind, name, called_execution_id, status)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, 'pending')
|
||||
RETURNING id
|
||||
`
|
||||
|
||||
type InsertStepExecutionParams struct {
|
||||
OrgID string
|
||||
WorkspaceID string
|
||||
ExecutionID int64
|
||||
Attempt int64
|
||||
ParentID pgtype.Int8
|
||||
Depth int32
|
||||
Kind string
|
||||
Name string
|
||||
CalledExecutionID int64
|
||||
}
|
||||
|
||||
func (q *Queries) InsertStepExecution(ctx context.Context, arg InsertStepExecutionParams) (int64, error) {
|
||||
row := q.db.QueryRow(ctx, insertStepExecution,
|
||||
arg.OrgID,
|
||||
arg.WorkspaceID,
|
||||
arg.ExecutionID,
|
||||
arg.Attempt,
|
||||
arg.ParentID,
|
||||
arg.Depth,
|
||||
arg.Kind,
|
||||
arg.Name,
|
||||
arg.CalledExecutionID,
|
||||
)
|
||||
var id int64
|
||||
err := row.Scan(&id)
|
||||
return id, err
|
||||
}
|
||||
|
||||
const lockExecutionForStep = `-- name: LockExecutionForStep :one
|
||||
|
||||
SELECT id, status, org_id, workspace_id, owner, repo, issue_number, workflow, attempt
|
||||
FROM tasks WHERE id = $1 FOR UPDATE
|
||||
`
|
||||
|
||||
type LockExecutionForStepRow struct {
|
||||
ID int64
|
||||
Status string
|
||||
OrgID string
|
||||
WorkspaceID string
|
||||
Owner string
|
||||
Repo string
|
||||
IssueNumber int64
|
||||
Workflow string
|
||||
Attempt int64
|
||||
}
|
||||
|
||||
// Step-execution queries (docs/prds/execution-tree-state-machine.md). The
|
||||
// store method owns the transaction that pairs each step transition with its
|
||||
// event row; these are only the guarded row writes and reads.
|
||||
// FOR UPDATE locks the execution's row, so the status check, the step insert
|
||||
// and the event append are one serialized operation per execution -- the way
|
||||
// guardTransition locks a task's row for its transition. The row also carries
|
||||
// every field the transition's event needs (org, workspace, the task
|
||||
// identity) and the attempt the step must belong to.
|
||||
func (q *Queries) LockExecutionForStep(ctx context.Context, id int64) (LockExecutionForStepRow, error) {
|
||||
row := q.db.QueryRow(ctx, lockExecutionForStep, id)
|
||||
var i LockExecutionForStepRow
|
||||
err := row.Scan(
|
||||
&i.ID,
|
||||
&i.Status,
|
||||
&i.OrgID,
|
||||
&i.WorkspaceID,
|
||||
&i.Owner,
|
||||
&i.Repo,
|
||||
&i.IssueNumber,
|
||||
&i.Workflow,
|
||||
&i.Attempt,
|
||||
)
|
||||
return i, err
|
||||
}
|
||||
|
||||
const lockStepExecution = `-- name: LockStepExecution :one
|
||||
SELECT id, org_id, workspace_id, execution_id, attempt, parent_id, depth, kind, name, called_execution_id, status, detail, tokens_used, started_at, finished_at
|
||||
FROM step_executions WHERE id = $1 FOR UPDATE
|
||||
`
|
||||
|
||||
// FOR UPDATE, shared by a start's parent check and a finish's guarded write:
|
||||
// the row is held for the caller's transaction, so the state a check reads is
|
||||
// the state the guarded write sees.
|
||||
func (q *Queries) LockStepExecution(ctx context.Context, id int64) (StepExecution, error) {
|
||||
row := q.db.QueryRow(ctx, lockStepExecution, id)
|
||||
var i StepExecution
|
||||
err := row.Scan(
|
||||
&i.ID,
|
||||
&i.OrgID,
|
||||
&i.WorkspaceID,
|
||||
&i.ExecutionID,
|
||||
&i.Attempt,
|
||||
&i.ParentID,
|
||||
&i.Depth,
|
||||
&i.Kind,
|
||||
&i.Name,
|
||||
&i.CalledExecutionID,
|
||||
&i.Status,
|
||||
&i.Detail,
|
||||
&i.TokensUsed,
|
||||
&i.StartedAt,
|
||||
&i.FinishedAt,
|
||||
)
|
||||
return i, err
|
||||
}
|
||||
|
||||
const startStepExecution = `-- name: StartStepExecution :execrows
|
||||
UPDATE step_executions SET status = 'running', started_at = now()
|
||||
WHERE id = $1 AND status = 'pending'
|
||||
`
|
||||
|
||||
// The pending -> running half of StartStep, guarded by the row's own status:
|
||||
// the step transition table routes the pair, so a row that is not pending is
|
||||
// stale rather than silently restarted.
|
||||
func (q *Queries) StartStepExecution(ctx context.Context, id int64) (int64, error) {
|
||||
result, err := q.db.Exec(ctx, startStepExecution, id)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
return result.RowsAffected(), nil
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
-- Step-execution queries (docs/prds/execution-tree-state-machine.md). The
|
||||
-- store method owns the transaction that pairs each step transition with its
|
||||
-- event row; these are only the guarded row writes and reads.
|
||||
|
||||
-- name: LockExecutionForStep :one
|
||||
-- FOR UPDATE locks the execution's row, so the status check, the step insert
|
||||
-- and the event append are one serialized operation per execution -- the way
|
||||
-- guardTransition locks a task's row for its transition. The row also carries
|
||||
-- every field the transition's event needs (org, workspace, the task
|
||||
-- identity) and the attempt the step must belong to.
|
||||
SELECT id, status, org_id, workspace_id, owner, repo, issue_number, workflow, attempt
|
||||
FROM tasks WHERE id = $1 FOR UPDATE;
|
||||
|
||||
-- name: InsertStepExecution :one
|
||||
INSERT INTO step_executions (org_id, workspace_id, execution_id, attempt, parent_id, depth, kind, name, called_execution_id, status)
|
||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, 'pending')
|
||||
RETURNING id;
|
||||
|
||||
-- name: StartStepExecution :execrows
|
||||
-- The pending -> running half of StartStep, guarded by the row's own status:
|
||||
-- the step transition table routes the pair, so a row that is not pending is
|
||||
-- stale rather than silently restarted.
|
||||
UPDATE step_executions SET status = 'running', started_at = now()
|
||||
WHERE id = $1 AND status = 'pending';
|
||||
|
||||
-- name: LockStepExecution :one
|
||||
-- FOR UPDATE, shared by a start's parent check and a finish's guarded write:
|
||||
-- the row is held for the caller's transaction, so the state a check reads is
|
||||
-- the state the guarded write sees.
|
||||
SELECT id, org_id, workspace_id, execution_id, attempt, parent_id, depth, kind, name, called_execution_id, status, detail, tokens_used, started_at, finished_at
|
||||
FROM step_executions WHERE id = $1 FOR UPDATE;
|
||||
|
||||
-- name: FinishStepExecution :execrows
|
||||
UPDATE step_executions
|
||||
SET status = $1, detail = $2, tokens_used = $3, finished_at = now()
|
||||
WHERE id = $4 AND status = $5;
|
||||
@@ -0,0 +1,232 @@
|
||||
package postgres
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgtype"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
task "github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||
"github.com/samcharles93/archie-core/internal/events"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/postgresdb"
|
||||
"github.com/samcharles93/archie-core/internal/taskstate"
|
||||
)
|
||||
|
||||
// The step-execution writes of docs/prds/execution-tree-state-machine.md.
|
||||
// Each is one transaction: the guarded row write, and the domain event row
|
||||
// the step transition produces, so the events table records exactly the
|
||||
// transitions the step row does. The events land in the table; the caller
|
||||
// publishes the returned event to its bus after the write commits, and the
|
||||
// daemon's event sink skips already-persisted rows by their assigned ID --
|
||||
// the EmitDurable convention.
|
||||
|
||||
// StartStep records a StepExecution entering running: the row is created
|
||||
// pending, guarded into running under the shared step transition table, and
|
||||
// the stage_start event is appended in the same transaction. The execution's
|
||||
// row is locked for the whole write, so its status check cannot race the
|
||||
// guarded updates, and org and workspace are stamped from it -- the request's
|
||||
// ownership fields are ignored, like every owned record.
|
||||
func (s *Store) StartStep(ctx context.Context, start task.StepStart) (int64, events.Event, error) {
|
||||
if !task.ValidStepKind(start.Kind) {
|
||||
return 0, events.Event{}, fmt.Errorf("%w: unknown step kind %q", storecontract.ErrInvalidStep, start.Kind)
|
||||
}
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return 0, events.Event{}, err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
q := postgresdb.New(tx)
|
||||
|
||||
execution, err := q.LockExecutionForStep(ctx, start.ExecutionID)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
// The execution the caller names does not exist: its view of the run
|
||||
// is stale, matching what guardTransition returns for a missing row.
|
||||
return 0, events.Event{}, storecontract.ErrStaleTransition
|
||||
}
|
||||
if err != nil {
|
||||
return 0, events.Event{}, err
|
||||
}
|
||||
if execution.Attempt != int64(start.Attempt) {
|
||||
// A step belongs to exactly one attempt; a caller naming one the
|
||||
// execution no longer runs is describing a run that is over.
|
||||
return 0, events.Event{}, storecontract.ErrStaleTransition
|
||||
}
|
||||
|
||||
depth, err := s.startParent(ctx, q, start)
|
||||
if err != nil {
|
||||
return 0, events.Event{}, err
|
||||
}
|
||||
// A step cannot reach running while its execution is not running, and
|
||||
// never under a terminal parent. Both are the state machine's rules, so
|
||||
// they refuse as illegal transitions.
|
||||
if err := taskstate.CheckStepStart(execution.Status, depth.underTerminal); err != nil {
|
||||
return 0, events.Event{}, fmt.Errorf("%w: %w", storecontract.ErrIllegalTransition, err)
|
||||
}
|
||||
|
||||
stepID, err := q.InsertStepExecution(ctx, postgresdb.InsertStepExecutionParams{
|
||||
OrgID: execution.OrgID, WorkspaceID: execution.WorkspaceID,
|
||||
ExecutionID: execution.ID, Attempt: execution.Attempt,
|
||||
ParentID: stepParentID(start.ParentID), Depth: int32(depth.depth),
|
||||
Kind: start.Kind, Name: start.Name, CalledExecutionID: 0,
|
||||
})
|
||||
if err != nil {
|
||||
return 0, events.Event{}, err
|
||||
}
|
||||
n, err := q.StartStepExecution(ctx, stepID)
|
||||
if err != nil {
|
||||
return 0, events.Event{}, err
|
||||
}
|
||||
if n == 0 {
|
||||
return 0, events.Event{}, storecontract.ErrStaleTransition
|
||||
}
|
||||
|
||||
event := stepEvent(events.KindStageStart, execution.ID,
|
||||
execution.Owner, execution.Repo, execution.IssueNumber, execution.Workflow,
|
||||
int(execution.Attempt), start.Name, nil)
|
||||
eventID, err := insertEventQ(ctx, q, event)
|
||||
if err != nil {
|
||||
return 0, events.Event{}, err
|
||||
}
|
||||
event.ID = eventID
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return 0, events.Event{}, err
|
||||
}
|
||||
return stepID, event, nil
|
||||
}
|
||||
|
||||
// stepParent resolves the enclosing StepExecution a child step names: its
|
||||
// row is locked in the same transaction, the tree must stay inside one
|
||||
// execution and one attempt, and depth is derived from the parent, never
|
||||
// supplied by the caller.
|
||||
func (s *Store) startParent(ctx context.Context, q *postgresdb.Queries, start task.StepStart) (stepParentState, error) {
|
||||
if start.ParentID == 0 {
|
||||
return stepParentState{}, nil
|
||||
}
|
||||
parent, err := q.LockStepExecution(ctx, start.ParentID)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return stepParentState{}, storecontract.ErrStaleTransition
|
||||
}
|
||||
if err != nil {
|
||||
return stepParentState{}, err
|
||||
}
|
||||
if parent.ExecutionID != start.ExecutionID || parent.Attempt != int64(start.Attempt) {
|
||||
// Trees never nest across executions, and a parent from another
|
||||
// attempt is a tree the caller is not running.
|
||||
return stepParentState{}, storecontract.ErrStaleTransition
|
||||
}
|
||||
return stepParentState{
|
||||
depth: int(parent.Depth) + 1,
|
||||
underTerminal: taskstate.StepTerminal(taskstate.StepStatus(parent.Status)),
|
||||
}, nil
|
||||
}
|
||||
|
||||
type stepParentState struct {
|
||||
depth int
|
||||
underTerminal bool
|
||||
}
|
||||
|
||||
// FinishStep moves one step to its outcome under the shared step transition
|
||||
// table, writing the stage_finish event in the same transaction and returning
|
||||
// it for the caller's post-commit publish. A step that is missing, that is
|
||||
// not the named execution's, or whose status does not match from is stale; an
|
||||
// off-table from->to pair is refused as an illegal transition.
|
||||
func (s *Store) FinishStep(ctx context.Context, finish task.StepFinish) (events.Event, error) {
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return events.Event{}, err
|
||||
}
|
||||
defer func() { _ = tx.Rollback(ctx) }()
|
||||
q := postgresdb.New(tx)
|
||||
|
||||
step, err := q.LockStepExecution(ctx, finish.StepID)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return events.Event{}, storecontract.ErrStaleTransition
|
||||
}
|
||||
if err != nil {
|
||||
return events.Event{}, err
|
||||
}
|
||||
// The step belongs to another run, or the caller believes a state the row
|
||||
// does not hold. Either way its view is stale, and writing would move
|
||||
// another execution's record.
|
||||
if step.ExecutionID != finish.ExecutionID || taskstate.StepStatus(step.Status) != finish.From {
|
||||
return events.Event{}, storecontract.ErrStaleTransition
|
||||
}
|
||||
// Staleness is decided first, the way guardTransition decides it: a caller
|
||||
// that is wrong about the row's state gets the stale sentinel even when
|
||||
// its pair is also unroutable.
|
||||
if !taskstate.CanStepTransition(finish.From, finish.To) {
|
||||
return events.Event{}, fmt.Errorf("%w: %s -> %s is not a step transition",
|
||||
storecontract.ErrIllegalTransition, finish.From, finish.To)
|
||||
}
|
||||
execution, err := q.TaskByID(ctx, step.ExecutionID)
|
||||
if err != nil {
|
||||
return events.Event{}, err
|
||||
}
|
||||
|
||||
durationMS := int64(0)
|
||||
if step.StartedAt.Valid {
|
||||
durationMS = time.Since(step.StartedAt.Time).Milliseconds()
|
||||
}
|
||||
n, err := q.FinishStepExecution(ctx, postgresdb.FinishStepExecutionParams{
|
||||
Status: string(finish.To), Detail: clip(finish.Detail, 4000), TokensUsed: finish.TokensUsed,
|
||||
ID: finish.StepID, Status_2: string(finish.From),
|
||||
})
|
||||
if err != nil {
|
||||
return events.Event{}, err
|
||||
}
|
||||
if n == 0 {
|
||||
return events.Event{}, storecontract.ErrStaleTransition
|
||||
}
|
||||
|
||||
event := stepEvent(events.KindStageFinish, execution.ID,
|
||||
execution.Owner, execution.Repo, execution.IssueNumber, execution.Workflow,
|
||||
int(step.Attempt), step.Name,
|
||||
stepEventData(finish.To, durationMS, finish.Detail))
|
||||
eventID, err := insertEventQ(ctx, q, event)
|
||||
if err != nil {
|
||||
return events.Event{}, err
|
||||
}
|
||||
event.ID = eventID
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return events.Event{}, err
|
||||
}
|
||||
return event, nil
|
||||
}
|
||||
|
||||
// stepEvent builds the event row one step transition writes, stamped with the
|
||||
// execution's identity the way TaskContext.Emit stamped it -- the same fields
|
||||
// the timeline and the stage statistics read back.
|
||||
func stepEvent(kind string, executionID int64, owner, repo string, issue int64, wf string, attempt int, stage string, data map[string]any) events.Event {
|
||||
return events.Event{
|
||||
Kind: kind, TaskID: executionID,
|
||||
Repo: owner + "/" + repo, Issue: int(issue), Workflow: wf,
|
||||
Attempt: attempt, Stage: stage, Data: data,
|
||||
}
|
||||
}
|
||||
|
||||
// stepEventData reads a stage_finish event's data the way the dashboard
|
||||
// (stageOutcome) and the stage statistics (StageStats) do: duration always,
|
||||
// the error text only when the step reported failure, and the interruption
|
||||
// marker only for an interrupted step.
|
||||
func stepEventData(to taskstate.StepStatus, durationMS int64, detail string) map[string]any {
|
||||
data := map[string]any{"duration_ms": durationMS}
|
||||
switch to {
|
||||
case taskstate.StepFailed:
|
||||
data["error"] = detail
|
||||
case taskstate.StepInterrupted:
|
||||
data["interrupted"] = true
|
||||
}
|
||||
return data
|
||||
}
|
||||
|
||||
// stepParentID encodes the nullable parent id an insert takes: 0 means "no
|
||||
// parent" -- a stage at the tree's root -- which the column stores as NULL
|
||||
// rather than as a reference to a row that does not exist.
|
||||
func stepParentID(parentID int64) pgtype.Int8 {
|
||||
return pgtype.Int8{Int64: parentID, Valid: parentID != 0}
|
||||
}
|
||||
@@ -0,0 +1,249 @@
|
||||
package postgres
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"testing"
|
||||
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
task "github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||
"github.com/samcharles93/archie-core/internal/events"
|
||||
"github.com/samcharles93/archie-core/internal/taskstate"
|
||||
)
|
||||
|
||||
// The step-execution writes (docs/prds/execution-tree-state-machine.md): each
|
||||
// of StartStep and FinishStep is one transaction -- guarded row write plus the
|
||||
// domain event row -- under the step transition table, with the execution's
|
||||
// own status gating whether a step may run at all.
|
||||
|
||||
// runningExecution enqueues and claims one task, the state a step runs in.
|
||||
func runningExecution(t *testing.T, s *Store) *workflow.Task {
|
||||
t.Helper()
|
||||
seed, err := s.EnqueueChatTask(t.Context(), "acme", "widgets", "t", "b", "implement", "")
|
||||
if err != nil || seed == nil {
|
||||
t.Fatalf("EnqueueChatTask: %+v %v", seed, err)
|
||||
}
|
||||
claimed, err := s.ClaimNext(t.Context())
|
||||
if err != nil || claimed == nil {
|
||||
t.Fatalf("ClaimNext: %+v %v", claimed, err)
|
||||
}
|
||||
return claimed
|
||||
}
|
||||
|
||||
func TestStartStepRecordsAStageEnteringRunning(t *testing.T) {
|
||||
s := storeFor(t)
|
||||
execution := runningExecution(t, s)
|
||||
|
||||
stepID, startEvent, err := s.StartStep(t.Context(), task.StepStart{
|
||||
ExecutionID: execution.ID, Attempt: execution.Attempt, Kind: task.StepKindStage, Name: "implement",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("StartStep: %v", err)
|
||||
}
|
||||
if startEvent.Kind != events.KindStageStart || startEvent.Stage != "implement" ||
|
||||
startEvent.TaskID != execution.ID || startEvent.Attempt != execution.Attempt ||
|
||||
startEvent.Repo != "acme/widgets" || startEvent.ID == 0 {
|
||||
t.Fatalf("stage_start event = %+v, want the transition's row", startEvent)
|
||||
}
|
||||
|
||||
eventsForTask, err := s.TaskEvents(t.Context(), execution.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("TaskEvents: %v", err)
|
||||
}
|
||||
if n := len(eventsForTask); n != 1 {
|
||||
t.Fatalf("events after StartStep = %d, want exactly the one the transition wrote", n)
|
||||
}
|
||||
|
||||
step, err := s.FinishStep(t.Context(), task.StepFinish{
|
||||
StepID: stepID, ExecutionID: execution.ID,
|
||||
From: taskstate.StepRunning, To: taskstate.StepSucceeded, Detail: "implemented",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("FinishStep: %v", err)
|
||||
}
|
||||
if step.Kind != events.KindStageFinish || step.Stage != "implement" || step.ID == 0 {
|
||||
t.Fatalf("stage_finish event = %+v, want the transition's row", step)
|
||||
}
|
||||
if _, ok := step.Data["duration_ms"]; !ok {
|
||||
t.Errorf("stage_finish data = %v, want duration_ms: the stage statistics read it", step.Data)
|
||||
}
|
||||
if _, has := step.Data["error"]; has {
|
||||
t.Errorf("stage_finish data = %v, want no error key for a succeeded step", step.Data)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFinishStepFailedCarriesTheErrorTheDashboardRenders(t *testing.T) {
|
||||
s := storeFor(t)
|
||||
execution := runningExecution(t, s)
|
||||
|
||||
stepID, _, err := s.StartStep(t.Context(), task.StepStart{
|
||||
ExecutionID: execution.ID, Attempt: execution.Attempt, Kind: task.StepKindStage, Name: "review",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("StartStep: %v", err)
|
||||
}
|
||||
step, err := s.FinishStep(t.Context(), task.StepFinish{
|
||||
StepID: stepID, ExecutionID: execution.ID,
|
||||
From: taskstate.StepRunning, To: taskstate.StepFailed, Detail: "review found 3 findings",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("FinishStep: %v", err)
|
||||
}
|
||||
if step.Data["error"] != "review found 3 findings" {
|
||||
t.Errorf("stage_finish data = %v, want the error text stageOutcome renders", step.Data)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStartStepRefusesWhatTheStateMachineForbids(t *testing.T) {
|
||||
tests := []struct {
|
||||
name string
|
||||
seed func(s *Store, taskID int64) error
|
||||
start task.StepStart
|
||||
want error
|
||||
}{
|
||||
{
|
||||
name: "an execution that is not running",
|
||||
start: task.StepStart{Kind: task.StepKindStage, Name: "plan", Attempt: 1},
|
||||
want: storecontract.ErrIllegalTransition,
|
||||
},
|
||||
{
|
||||
name: "an execution that does not exist",
|
||||
start: task.StepStart{Kind: task.StepKindStage, Name: "plan", Attempt: 1},
|
||||
want: storecontract.ErrStaleTransition,
|
||||
},
|
||||
{
|
||||
name: "an attempt the execution no longer runs",
|
||||
start: task.StepStart{Kind: task.StepKindStage, Name: "plan", Attempt: 99},
|
||||
want: storecontract.ErrStaleTransition,
|
||||
},
|
||||
{
|
||||
name: "an unknown step kind",
|
||||
start: task.StepStart{Kind: "quantum", Name: "plan", Attempt: 1},
|
||||
want: storecontract.ErrInvalidStep,
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
s := storeFor(t)
|
||||
execution := runningExecution(t, s)
|
||||
if tt.name == "an execution that is not running" {
|
||||
// Back on the queue: nothing may run under it. The seed rides
|
||||
// the legal running->queued edge, not a hand-written pair.
|
||||
if err := s.Transition(t.Context(), execution.ID, taskstate.Running, taskstate.Queued, "requeue"); err != nil {
|
||||
t.Fatalf("seed queued: %v", err)
|
||||
}
|
||||
}
|
||||
start := tt.start
|
||||
start.ExecutionID = execution.ID
|
||||
if tt.name == "an execution that does not exist" {
|
||||
start.ExecutionID = execution.ID + 999999
|
||||
}
|
||||
if _, _, err := s.StartStep(t.Context(), start); !errors.Is(err, tt.want) {
|
||||
t.Fatalf("StartStep = %v, want %v", err, tt.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestStartStepUnderATerminalParentIsIllegal(t *testing.T) {
|
||||
s := storeFor(t)
|
||||
execution := runningExecution(t, s)
|
||||
|
||||
parentID, _, err := s.StartStep(t.Context(), task.StepStart{
|
||||
ExecutionID: execution.ID, Attempt: execution.Attempt, Kind: task.StepKindStage, Name: "review",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("StartStep parent: %v", err)
|
||||
}
|
||||
if _, err := s.FinishStep(t.Context(), task.StepFinish{
|
||||
StepID: parentID, ExecutionID: execution.ID,
|
||||
From: taskstate.StepRunning, To: taskstate.StepSucceeded, Detail: "done",
|
||||
}); err != nil {
|
||||
t.Fatalf("FinishStep parent: %v", err)
|
||||
}
|
||||
|
||||
_, _, err = s.StartStep(t.Context(), task.StepStart{
|
||||
ExecutionID: execution.ID, Attempt: execution.Attempt,
|
||||
ParentID: parentID, Kind: task.StepKindAgent, Name: "review:2",
|
||||
})
|
||||
if !errors.Is(err, storecontract.ErrIllegalTransition) {
|
||||
t.Fatalf("StartStep under a terminal parent = %v, want ErrIllegalTransition", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStepDepthIsDerivedFromTheParent(t *testing.T) {
|
||||
s := storeFor(t)
|
||||
execution := runningExecution(t, s)
|
||||
|
||||
parentID, _, err := s.StartStep(t.Context(), task.StepStart{
|
||||
ExecutionID: execution.ID, Attempt: execution.Attempt, Kind: task.StepKindStage, Name: "implement",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("StartStep stage: %v", err)
|
||||
}
|
||||
childID, _, err := s.StartStep(t.Context(), task.StepStart{
|
||||
ExecutionID: execution.ID, Attempt: execution.Attempt,
|
||||
ParentID: parentID, Kind: task.StepKindAgent, Name: "implement",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("StartStep agent child: %v", err)
|
||||
}
|
||||
var depths map[int64]int
|
||||
if err := s.pool.QueryRow(t.Context(),
|
||||
`SELECT jsonb_object_agg(id, depth) FROM step_executions WHERE execution_id = $1`,
|
||||
execution.ID).Scan(&depths); err != nil {
|
||||
t.Fatalf("read depths: %v", err)
|
||||
}
|
||||
if depths[parentID] != 0 || depths[childID] != 1 {
|
||||
t.Fatalf("depths = %v, want the stage at 0 and its child at 1", depths)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFinishStepRefusesStaleAndForeign(t *testing.T) {
|
||||
s := storeFor(t)
|
||||
execution := runningExecution(t, s)
|
||||
other := runningExecution(t, s)
|
||||
|
||||
stepID, _, err := s.StartStep(t.Context(), task.StepStart{
|
||||
ExecutionID: execution.ID, Attempt: execution.Attempt, Kind: task.StepKindStage, Name: "plan",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("StartStep: %v", err)
|
||||
}
|
||||
|
||||
if _, err := s.FinishStep(t.Context(), task.StepFinish{
|
||||
StepID: stepID, ExecutionID: other.ID,
|
||||
From: taskstate.StepRunning, To: taskstate.StepSucceeded,
|
||||
}); !errors.Is(err, storecontract.ErrStaleTransition) {
|
||||
t.Fatalf("FinishStep naming another execution = %v, want ErrStaleTransition", err)
|
||||
}
|
||||
if _, err := s.FinishStep(t.Context(), task.StepFinish{
|
||||
StepID: stepID, ExecutionID: execution.ID,
|
||||
From: taskstate.StepSucceeded, To: taskstate.StepFailed,
|
||||
}); !errors.Is(err, storecontract.ErrStaleTransition) {
|
||||
t.Fatalf("FinishStep with an untruthful from = %v, want ErrStaleTransition", err)
|
||||
}
|
||||
if _, err := s.FinishStep(t.Context(), task.StepFinish{
|
||||
StepID: stepID, ExecutionID: execution.ID,
|
||||
From: taskstate.StepRunning, To: taskstate.StepPending,
|
||||
}); !errors.Is(err, storecontract.ErrIllegalTransition) {
|
||||
t.Fatalf("FinishStep off-table pair = %v, want ErrIllegalTransition", err)
|
||||
}
|
||||
if _, err := s.FinishStep(t.Context(), task.StepFinish{
|
||||
StepID: stepID + 999999, ExecutionID: execution.ID,
|
||||
From: taskstate.StepRunning, To: taskstate.StepSucceeded,
|
||||
}); !errors.Is(err, storecontract.ErrStaleTransition) {
|
||||
t.Fatalf("FinishStep on a missing step = %v, want ErrStaleTransition", err)
|
||||
}
|
||||
}
|
||||
|
||||
// A late FinishStep for a step another path already finished is stale, the
|
||||
// PRD's named wire behaviour: errors.Is matches on the client too.
|
||||
func TestFinishStepOffTablePairIsIllegalOverTheTable(t *testing.T) {
|
||||
if !taskstate.CanStepTransition(taskstate.StepPending, taskstate.StepRunning) {
|
||||
t.Fatal("pending -> running must be in the step transition table")
|
||||
}
|
||||
if taskstate.CanStepTransition(taskstate.StepSucceeded, taskstate.StepRunning) {
|
||||
t.Fatal("succeeded -> running must not be in the step transition table: terminal is terminal")
|
||||
}
|
||||
}
|
||||
@@ -124,6 +124,7 @@ func taskFromRow(t postgresdb.Task) *workflow.Task {
|
||||
// Compile-time checks: *Store satisfies every surface the SQLite store did.
|
||||
var (
|
||||
_ storecontract.TaskStore = (*Store)(nil)
|
||||
_ storecontract.StepRecorder = (*Store)(nil)
|
||||
_ storecontract.BindingTaskCreator = (*Store)(nil)
|
||||
_ storecontract.ConfigSnapshotStore = (*Store)(nil)
|
||||
_ storecontract.ChannelStatusStore = (*Store)(nil)
|
||||
|
||||
@@ -107,6 +107,41 @@ func (c *Client) Transition(ctx context.Context, taskID int64, from, to, detail
|
||||
return unmapError(err)
|
||||
}
|
||||
|
||||
// StartStep and FinishStep record the run's step executions
|
||||
// (docs/prds/execution-tree-state-machine.md) under the run credential, which
|
||||
// the State Store's grant interceptor scopes to the execution the requests
|
||||
// name. Each returns the transition's persisted event for the caller's
|
||||
// post-commit publish.
|
||||
func (c *Client) StartStep(ctx context.Context, s task.StepStart) (int64, events.Event, error) {
|
||||
r, err := c.client.StartStep(ctx, &pb.StartStepRequest{
|
||||
ExecutionId: s.ExecutionID, Attempt: int64(s.Attempt),
|
||||
ParentId: s.ParentID, Kind: s.Kind, Name: s.Name,
|
||||
})
|
||||
if err != nil {
|
||||
return 0, events.Event{}, unmapError(err)
|
||||
}
|
||||
event := events.Event{}
|
||||
if r.Event != nil {
|
||||
event = eventValue(r.Event)
|
||||
}
|
||||
return r.StepId, event, nil
|
||||
}
|
||||
|
||||
func (c *Client) FinishStep(ctx context.Context, s task.StepFinish) (events.Event, error) {
|
||||
r, err := c.client.FinishStep(ctx, &pb.FinishStepRequest{
|
||||
ExecutionId: s.ExecutionID, StepId: s.StepID,
|
||||
From: string(s.From), To: string(s.To), Detail: s.Detail, TokensUsed: s.TokensUsed,
|
||||
})
|
||||
if err != nil {
|
||||
return events.Event{}, unmapError(err)
|
||||
}
|
||||
event := events.Event{}
|
||||
if r.Event != nil {
|
||||
event = eventValue(r.Event)
|
||||
}
|
||||
return event, nil
|
||||
}
|
||||
|
||||
func (c *Client) Update(ctx context.Context, t *task.Task) error {
|
||||
_, err := c.client.Update(ctx, &pb.UpdateRequest{Task: taskProto(t)})
|
||||
return unmapError(err)
|
||||
|
||||
@@ -14,6 +14,7 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/domain/binding"
|
||||
"github.com/samcharles93/archie-core/internal/domain/mapping"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
"github.com/samcharles93/archie-core/internal/events"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
|
||||
@@ -26,6 +27,7 @@ import (
|
||||
// docs/prds/state-store-contract.md §11's conformance requirement.
|
||||
type contract interface {
|
||||
storecontract.TaskStore
|
||||
storecontract.StepRecorder
|
||||
storecontract.BindingTaskCreator
|
||||
storecontract.ConfigSnapshotStore
|
||||
storecontract.ApplyStatusStore
|
||||
@@ -81,7 +83,7 @@ func remoteTaskStore(t *testing.T, local *pgstore.TaskDB, logs storecontract.Tas
|
||||
listener := bufconn.Listen(1 << 20)
|
||||
server := grpc.NewServer()
|
||||
deps := Deps{
|
||||
Tasks: local, BindingTaskCreator: local, WorkflowCalls: local,
|
||||
Tasks: local, BindingTaskCreator: local, WorkflowCalls: local, Steps: local,
|
||||
ConfigSnapshots: local, ApplyStatus: local,
|
||||
TaskLogs: logs,
|
||||
}
|
||||
@@ -133,6 +135,37 @@ func TestStateStoreConformance(t *testing.T) {
|
||||
if err := c.Transition(ctx, task.ID, task.Status, "running", "started"); err != nil {
|
||||
t.Fatalf("Transition: %v", err)
|
||||
}
|
||||
current, err := c.TaskByID(ctx, task.ID)
|
||||
if err != nil || current == nil {
|
||||
t.Fatalf("TaskByID: %+v %v", current, err)
|
||||
}
|
||||
// The step-execution writes (docs/prds/execution-tree-state-machine.md):
|
||||
// both adapters record the same guarded pair, and each response
|
||||
// carries the transition's own persisted event.
|
||||
stepID, startEvent, err := c.StartStep(ctx, workflow.StepStart{
|
||||
ExecutionID: task.ID, Attempt: current.Attempt, Kind: workflow.StepKindStage, Name: "implement",
|
||||
})
|
||||
if err != nil || stepID == 0 {
|
||||
t.Fatalf("StartStep: %d %v", stepID, err)
|
||||
}
|
||||
if startEvent.Kind != "stage_start" || startEvent.TaskID != task.ID || startEvent.ID == 0 {
|
||||
t.Fatalf("StartStep event = %+v, want the persisted stage_start row", startEvent)
|
||||
}
|
||||
finishEvent, err := c.FinishStep(ctx, workflow.StepFinish{
|
||||
StepID: stepID, ExecutionID: task.ID,
|
||||
From: "running", To: "succeeded", Detail: "done", TokensUsed: 12,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("FinishStep: %v", err)
|
||||
}
|
||||
if finishEvent.Kind != "stage_finish" || finishEvent.ID == 0 || finishEvent.Data["duration_ms"] == nil {
|
||||
t.Fatalf("FinishStep event = %+v, want the persisted stage_finish row with a duration", finishEvent)
|
||||
}
|
||||
if _, err := c.FinishStep(ctx, workflow.StepFinish{
|
||||
StepID: stepID, ExecutionID: task.ID, From: "running", To: "succeeded",
|
||||
}); !errors.Is(err, storecontract.ErrStaleTransition) {
|
||||
t.Fatalf("replayed FinishStep = %v, want ErrStaleTransition over both adapters", err)
|
||||
}
|
||||
task.Plan = "the plan"
|
||||
task.Status = "running"
|
||||
task.PRNumber = 401
|
||||
@@ -256,24 +289,29 @@ func TestStateStoreConformance(t *testing.T) {
|
||||
t.Fatalf("OpenPRs: %v", err)
|
||||
}
|
||||
|
||||
// TaskEvents / EventsSince / stats.
|
||||
// TaskEvents / EventsSince / stats. The two step events the
|
||||
// battery's step half wrote come first (their transitions ran
|
||||
// first), then the two the battery inserted directly.
|
||||
evs, err := c.TaskEvents(ctx, task.ID)
|
||||
if err != nil || len(evs) != 2 || evs[0].Data["duration_ms"] != 12.0 {
|
||||
if err != nil || len(evs) != 4 {
|
||||
t.Fatalf("TaskEvents: %+v %v", evs, err)
|
||||
}
|
||||
if evs[0].Attempt != 2 {
|
||||
t.Errorf("TaskEvents attempt = %d, want 2 (provenance must survive the wire)", evs[0].Attempt)
|
||||
if evs[2].Data["duration_ms"] != 12.0 {
|
||||
t.Fatalf("TaskEvents stage_finish = %+v, want the battery's own duration_ms", evs[2])
|
||||
}
|
||||
if evs[1].Kind != events.KindConfigCaptured || evs[1].Attempt != 2 || evs[1].Data["schema"] != events.ConfigCapturedSchema {
|
||||
t.Errorf("config_captured event = %+v, want its kind, attempt and schema preserved", evs[1])
|
||||
if evs[2].Attempt != 2 {
|
||||
t.Errorf("TaskEvents attempt = %d, want 2 (provenance must survive the wire)", evs[2].Attempt)
|
||||
}
|
||||
if _, renamed := evs[1].Data["config"]; renamed {
|
||||
if evs[3].Kind != events.KindConfigCaptured || evs[3].Attempt != 2 || evs[3].Data["schema"] != events.ConfigCapturedSchema {
|
||||
t.Errorf("config_captured event = %+v, want its kind, attempt and schema preserved", evs[3])
|
||||
}
|
||||
if _, renamed := evs[3].Data["config"]; renamed {
|
||||
t.Error(`the document crossed under "config"; the producer writes it under "document"`)
|
||||
}
|
||||
if len(evs[1].Data) != 2 {
|
||||
if len(evs[3].Data) != 2 {
|
||||
t.Errorf("config_captured data keys = %v, want exactly schema and document", evs[1].Data)
|
||||
}
|
||||
doc, ok := evs[1].Data["document"].(map[string]any)
|
||||
doc, ok := evs[3].Data["document"].(map[string]any)
|
||||
if !ok || doc["bot_user"] != "archie" {
|
||||
t.Errorf("config_captured payload = %#v, want the decoded document under its own key", evs[1].Data)
|
||||
}
|
||||
@@ -365,7 +403,7 @@ func TestStateStoreConformance(t *testing.T) {
|
||||
// Archive. RecoverStale above may have requeued task if it was
|
||||
// still running, so re-fetch its current status rather than
|
||||
// trusting the in-memory snapshot.
|
||||
current, err := c.TaskByID(ctx, task.ID)
|
||||
current, err = c.TaskByID(ctx, task.ID)
|
||||
if err != nil || current == nil {
|
||||
t.Fatalf("TaskByID before archive: %+v %v", current, err)
|
||||
}
|
||||
|
||||
@@ -77,6 +77,7 @@ func TestUnmapErrorSentinelFidelity(t *testing.T) {
|
||||
}{
|
||||
{name: "stale transition", store: storecontract.ErrStaleTransition, rehyd: storecontract.ErrStaleTransition},
|
||||
{name: "illegal transition", store: storecontract.ErrIllegalTransition, rehyd: storecontract.ErrIllegalTransition},
|
||||
{name: "invalid step", store: storecontract.ErrInvalidStep, rehyd: storecontract.ErrInvalidStep},
|
||||
{name: "binding not found", store: storecontract.ErrBindingNotFound, rehyd: storecontract.ErrBindingNotFound},
|
||||
{name: "mapping not found", store: storecontract.ErrMappingNotFound, rehyd: storecontract.ErrMappingNotFound},
|
||||
{name: "binding overlap", store: storecontract.ErrBindingOverlap, rehyd: storecontract.ErrBindingOverlap},
|
||||
|
||||
@@ -26,6 +26,7 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/domain/storepkg"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||
"github.com/samcharles93/archie-core/internal/logging"
|
||||
"github.com/samcharles93/archie-core/internal/taskstate"
|
||||
)
|
||||
|
||||
// Unavailable-capability errors, mirroring gatewayrpc's missing-session-store
|
||||
@@ -96,6 +97,11 @@ type Deps struct {
|
||||
// answers both RPCs with codes.Unavailable, which is the honest answer
|
||||
// for a store that owns no task table.
|
||||
WorkflowCalls storecontract.WorkflowCaller
|
||||
// Steps is the step-execution write surface
|
||||
// (docs/prds/execution-tree-state-machine.md): the guarded row writes the
|
||||
// workflow engine's run credential calls. Optional: nil answers both RPCs
|
||||
// with codes.Unavailable.
|
||||
Steps storecontract.StepRecorder
|
||||
// PlaybookDispatcher is the side-effecting-action idempotency ledger
|
||||
// (docs/prds/eda-playbook-engine.md gap 2). Optional: nil disables the
|
||||
// two playbook dispatch RPCs with codes.Unavailable.
|
||||
@@ -189,6 +195,40 @@ func (s *server) Transition(ctx context.Context, r *pb.TransitionRequest) (*pb.T
|
||||
return &pb.TransitionResponse{}, nil
|
||||
}
|
||||
|
||||
// StartStep and FinishStep are the step-execution writes
|
||||
// (docs/prds/execution-tree-state-machine.md). Each is the store's own
|
||||
// transaction -- guarded row write plus the event row -- and the store's
|
||||
// response carries the persisted event back for the caller's post-commit
|
||||
// publish.
|
||||
func (s *server) StartStep(ctx context.Context, r *pb.StartStepRequest) (*pb.StartStepResponse, error) {
|
||||
if s.deps.Steps == nil {
|
||||
return nil, status.Error(codes.Unavailable, "step recorder unavailable")
|
||||
}
|
||||
stepID, event, err := s.deps.Steps.StartStep(ctx, task.StepStart{
|
||||
ExecutionID: r.ExecutionId, Attempt: int(r.Attempt),
|
||||
ParentID: r.ParentId, Kind: r.Kind, Name: r.Name,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, s.logErr("StartStep", err)
|
||||
}
|
||||
return &pb.StartStepResponse{StepId: stepID, Event: eventProto(event)}, nil
|
||||
}
|
||||
|
||||
func (s *server) FinishStep(ctx context.Context, r *pb.FinishStepRequest) (*pb.FinishStepResponse, error) {
|
||||
if s.deps.Steps == nil {
|
||||
return nil, status.Error(codes.Unavailable, "step recorder unavailable")
|
||||
}
|
||||
event, err := s.deps.Steps.FinishStep(ctx, task.StepFinish{
|
||||
StepID: r.StepId, ExecutionID: r.ExecutionId,
|
||||
From: taskstate.StepStatus(r.From), To: taskstate.StepStatus(r.To),
|
||||
Detail: r.Detail, TokensUsed: r.TokensUsed,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, s.logErr("FinishStep", err)
|
||||
}
|
||||
return &pb.FinishStepResponse{Event: eventProto(event)}, nil
|
||||
}
|
||||
|
||||
func (s *server) Update(ctx context.Context, r *pb.UpdateRequest) (*pb.UpdateResponse, error) {
|
||||
if r.Task == nil {
|
||||
return nil, status.Error(codes.InvalidArgument, "task is required")
|
||||
|
||||
@@ -95,32 +95,74 @@ func (g *TaskGrants) taskFor(token string) int64 {
|
||||
// (including RegisterTaskGrant/RevokeTaskGrant themselves) falls through to
|
||||
// false: a task grant can only ever narrow, never expand.
|
||||
//
|
||||
// The two workflow.call RPCs are the one sanctioned widening
|
||||
// (docs/prds/workflow-calls.md): a workflow.call step must start its callee
|
||||
// and read it back, and the callee is the caller's child. Both requests
|
||||
// carry the caller's own task ID, which the grant check verifies; the
|
||||
// parent-child relation is a handler-side row check, so this interceptor
|
||||
// never reads the database.
|
||||
// The two workflow.call RPCs and the two step-execution writes are the
|
||||
// sanctioned widenings: a workflow.call step must start its callee and read it
|
||||
// back, the callee being the caller's child (docs/prds/workflow-calls.md), and
|
||||
// the container records its stages and its own agent calls under the run
|
||||
// credential of docs/prds/execution-tree-state-machine.md. The step requests
|
||||
// carry the caller's own task ID, which the grant check verifies; the step's
|
||||
// membership in that execution is a handler-side row check, so this
|
||||
// interceptor never reads the database.
|
||||
func authorizesTaskScopedCall(fullMethod string, req any, taskID int64) bool {
|
||||
switch fullMethod {
|
||||
case pb.StateStoreService_Update_FullMethodName:
|
||||
target := taskScopedTargets[fullMethod]
|
||||
return target != nil && target(req) == taskID
|
||||
}
|
||||
|
||||
// taskScopedTargets maps each RPC a task grant may ever authorize to the
|
||||
// function naming the task ID its request targets (0 for a request it cannot
|
||||
// read). Held as data so the sanctioned widenings stop growing a switch: a
|
||||
// new authorized RPC registers one row here, and the deny-by-default rule
|
||||
// above covers everything else.
|
||||
var taskScopedTargets = map[string]func(any) int64{
|
||||
pb.StateStoreService_Update_FullMethodName: func(req any) int64 {
|
||||
r, ok := req.(*pb.UpdateRequest)
|
||||
return ok && r.Task != nil && r.Task.Id == taskID
|
||||
case pb.StateStoreService_Transition_FullMethodName:
|
||||
if !ok || r.Task == nil {
|
||||
return 0
|
||||
}
|
||||
return r.Task.Id
|
||||
},
|
||||
pb.StateStoreService_Transition_FullMethodName: func(req any) int64 {
|
||||
r, ok := req.(*pb.TransitionRequest)
|
||||
return ok && r.TaskId == taskID
|
||||
case pb.StateStoreService_InsertEvent_FullMethodName:
|
||||
if !ok {
|
||||
return 0
|
||||
}
|
||||
return r.TaskId
|
||||
},
|
||||
pb.StateStoreService_InsertEvent_FullMethodName: func(req any) int64 {
|
||||
r, ok := req.(*pb.InsertEventRequest)
|
||||
return ok && r.Event != nil && r.Event.TaskId == taskID
|
||||
case pb.StateStoreService_EnqueueCallTask_FullMethodName:
|
||||
if !ok || r.Event == nil {
|
||||
return 0
|
||||
}
|
||||
return r.Event.TaskId
|
||||
},
|
||||
pb.StateStoreService_StartStep_FullMethodName: func(req any) int64 {
|
||||
r, ok := req.(*pb.StartStepRequest)
|
||||
if !ok {
|
||||
return 0
|
||||
}
|
||||
return r.ExecutionId
|
||||
},
|
||||
pb.StateStoreService_FinishStep_FullMethodName: func(req any) int64 {
|
||||
r, ok := req.(*pb.FinishStepRequest)
|
||||
if !ok {
|
||||
return 0
|
||||
}
|
||||
return r.ExecutionId
|
||||
},
|
||||
pb.StateStoreService_EnqueueCallTask_FullMethodName: func(req any) int64 {
|
||||
r, ok := req.(*pb.EnqueueCallTaskRequest)
|
||||
return ok && r.CallerTaskId == taskID
|
||||
case pb.StateStoreService_WorkflowCallStatus_FullMethodName:
|
||||
if !ok {
|
||||
return 0
|
||||
}
|
||||
return r.CallerTaskId
|
||||
},
|
||||
pb.StateStoreService_WorkflowCallStatus_FullMethodName: func(req any) int64 {
|
||||
r, ok := req.(*pb.WorkflowCallStatusRequest)
|
||||
return ok && r.CallerTaskId == taskID
|
||||
default:
|
||||
return false
|
||||
}
|
||||
if !ok {
|
||||
return 0
|
||||
}
|
||||
return r.CallerTaskId
|
||||
},
|
||||
}
|
||||
|
||||
// UnaryInterceptor separates administrator access from task-scoped worker
|
||||
|
||||
@@ -17,8 +17,10 @@ import (
|
||||
"github.com/samcharles93/archie-core/internal/domain/mapping"
|
||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||
task "github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||
"github.com/samcharles93/archie-core/internal/events"
|
||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/pgstore"
|
||||
"github.com/samcharles93/archie-core/internal/taskstate"
|
||||
)
|
||||
|
||||
// grantsServer wires TaskGrants' Unary/Stream interceptors -- the same
|
||||
@@ -41,7 +43,7 @@ func grantsServer(t *testing.T, adminToken string) (grants *TaskGrants, dial fun
|
||||
Tasks: local, ConfigSnapshots: local, ApplyStatus: local,
|
||||
Captures: eda, Mappings: eda, Bindings: eda,
|
||||
BindingDispatcher: eda, PlaybookDispatcher: eda, EventTypes: eda,
|
||||
BindingTaskCreator: local, WorkflowCalls: local,
|
||||
BindingTaskCreator: local, WorkflowCalls: local, Steps: local,
|
||||
Grants: grants,
|
||||
})
|
||||
go func() { _ = server.Serve(listener) }()
|
||||
@@ -385,3 +387,73 @@ func TestOnlyAdminCanRegisterOrRevokeTaskGrants(t *testing.T) {
|
||||
t.Fatal("a task-scoped grant must not be able to revoke grants")
|
||||
}
|
||||
}
|
||||
|
||||
// TestRunCredentialRecordsItsOwnSteps: the run credential of
|
||||
// docs/prds/execution-tree-state-machine.md authorises StartStep and
|
||||
// FinishStep on its own execution only -- the container records its stages
|
||||
// and its own agent calls with it -- and never another execution's.
|
||||
func TestRunCredentialRecordsItsOwnSteps(t *testing.T) {
|
||||
const adminToken = "daemon-admin-token"
|
||||
grants, dial := grantsServer(t, adminToken)
|
||||
_ = grants
|
||||
admin := dial(t, adminToken)
|
||||
ctx := t.Context()
|
||||
|
||||
own, err := admin.EnqueueChatTask(ctx, "acme", "widget", "a", "body", "implement", "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := admin.ClaimNext(ctx); err != nil {
|
||||
t.Fatalf("ClaimNext: %v", err)
|
||||
}
|
||||
// The claim increments the attempt, so the run the grant names is read
|
||||
// back after it, the way the container's own request does.
|
||||
own, err = admin.TaskByID(ctx, own.ID)
|
||||
if err != nil || own == nil {
|
||||
t.Fatalf("TaskByID: %+v %v", own, err)
|
||||
}
|
||||
other, err := admin.EnqueueChatTask(ctx, "acme", "widget", "b", "body", "implement", "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
workerToken, err := admin.RegisterTaskGrant(ctx, own.ID, time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("RegisterTaskGrant: %v", err)
|
||||
}
|
||||
worker := dial(t, workerToken)
|
||||
|
||||
stepID, startEvent, err := worker.StartStep(ctx, task.StepStart{
|
||||
ExecutionID: own.ID, Attempt: own.Attempt, Kind: task.StepKindStage, Name: "implement",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("task grant should authorize StartStep on its own execution: %v", err)
|
||||
}
|
||||
if startEvent.Kind != events.KindStageStart || startEvent.TaskID != own.ID {
|
||||
t.Fatalf("start event = %+v, want the transition's stage_start row", startEvent)
|
||||
}
|
||||
finishEvent, err := worker.FinishStep(ctx, task.StepFinish{
|
||||
StepID: stepID, ExecutionID: own.ID,
|
||||
From: taskstate.StepRunning, To: taskstate.StepSucceeded,
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatalf("task grant should authorize FinishStep on its own execution: %v", err)
|
||||
}
|
||||
if finishEvent.Kind != events.KindStageFinish {
|
||||
t.Fatalf("finish event = %+v, want the transition's stage_finish row", finishEvent)
|
||||
}
|
||||
|
||||
// The refusal is the interceptor's, not the store's: the codes tell the
|
||||
// difference, so a store-level refusal could not mask a missing grant rule.
|
||||
if _, _, err := worker.StartStep(ctx, task.StepStart{
|
||||
ExecutionID: other.ID, Attempt: other.Attempt, Kind: task.StepKindStage, Name: "plan",
|
||||
}); status.Code(err) != codes.PermissionDenied {
|
||||
t.Fatalf("StartStep on another execution = %v, want PermissionDenied from the grant check", err)
|
||||
}
|
||||
if _, err := worker.FinishStep(ctx, task.StepFinish{
|
||||
StepID: stepID, ExecutionID: other.ID,
|
||||
From: taskstate.StepRunning, To: taskstate.StepSucceeded,
|
||||
}); status.Code(err) != codes.PermissionDenied {
|
||||
t.Fatalf("FinishStep naming another execution = %v, want PermissionDenied from the grant check", err)
|
||||
}
|
||||
}
|
||||
@@ -334,6 +334,7 @@ func dayTokensValue(d *pb.DayTokens) storecontract.DayTokens {
|
||||
const (
|
||||
msgStaleTransition = "stale transition"
|
||||
msgIllegalTransition = "illegal transition"
|
||||
msgInvalidStep = "step execution invalid"
|
||||
msgBindingNotFound = "binding not found"
|
||||
msgMappingNotFound = "mapping not found"
|
||||
msgEventTypeNotFound = "event type not found"
|
||||
@@ -447,6 +448,7 @@ var wireErrors = []struct {
|
||||
}{
|
||||
{storecontract.ErrStaleTransition, codes.FailedPrecondition, msgStaleTransition},
|
||||
{storecontract.ErrIllegalTransition, codes.FailedPrecondition, msgIllegalTransition},
|
||||
{storecontract.ErrInvalidStep, codes.InvalidArgument, msgInvalidStep},
|
||||
{storecontract.ErrBindingOverlap, codes.FailedPrecondition, msgBindingOverlap},
|
||||
{storecontract.ErrBindingTransition, codes.FailedPrecondition, msgBindingTransition},
|
||||
{storecontract.ErrSourceSigningStale, codes.FailedPrecondition, msgSourceSigning},
|
||||
|
||||
@@ -40,6 +40,10 @@ service StateStoreService {
|
||||
rpc ClaimByIssue(ClaimByIssueRequest) returns (ClaimByIssueResponse);
|
||||
rpc Transition(TransitionRequest) returns (TransitionResponse);
|
||||
rpc Update(UpdateRequest) returns (UpdateResponse);
|
||||
// Step executions (docs/prds/execution-tree-state-machine.md). The run
|
||||
// credential authorises both on its own execution only.
|
||||
rpc StartStep(StartStepRequest) returns (StartStepResponse);
|
||||
rpc FinishStep(FinishStepRequest) returns (FinishStepResponse);
|
||||
rpc Requeue(RequeueRequest) returns (RequeueResponse);
|
||||
rpc ParkTask(ParkTaskRequest) returns (ParkTaskResponse);
|
||||
rpc RecoverStale(RecoverStaleRequest) returns (RecoverStaleResponse);
|
||||
@@ -518,6 +522,42 @@ message TransitionRequest {
|
||||
}
|
||||
message TransitionResponse {}
|
||||
|
||||
message StartStepRequest {
|
||||
// execution_id is the WorkflowExecution (the tasks row) the step belongs
|
||||
// to, and the task ID the run credential is scoped to.
|
||||
int64 execution_id = 1;
|
||||
int64 attempt = 2;
|
||||
// parent_id is 0 when the step is a stage; otherwise it names the enclosing
|
||||
// StepExecution, from which the store derives depth.
|
||||
int64 parent_id = 3;
|
||||
// kind is stage, agent or call.
|
||||
string kind = 4;
|
||||
string name = 5;
|
||||
}
|
||||
message StartStepResponse {
|
||||
int64 step_id = 1;
|
||||
// event is the transition's stage_start row, already persisted; the caller
|
||||
// publishes it to its bus after the write commits (the daemon's event sink
|
||||
// skips already-persisted rows by their assigned ID).
|
||||
Event event = 2;
|
||||
}
|
||||
|
||||
message FinishStepRequest {
|
||||
// execution_id names the run the step belongs to. The grant check verifies
|
||||
// it against the credential, and the store verifies the step really is that
|
||||
// execution's, so a caller cannot move another run's step by naming it.
|
||||
int64 execution_id = 1;
|
||||
int64 step_id = 2;
|
||||
string from = 3;
|
||||
string to = 4;
|
||||
string detail = 5;
|
||||
int64 tokens_used = 6;
|
||||
}
|
||||
message FinishStepResponse {
|
||||
// event is the transition's stage_finish row, already persisted.
|
||||
Event event = 1;
|
||||
}
|
||||
|
||||
message UpdateRequest { Task task = 1; }
|
||||
message UpdateResponse {}
|
||||
|
||||
|
||||
Reference in new issue
Block a user