feat(eda): workflow inputs, optional repository and agent profiles
Workflow definitions declare inputs, repository (none, optional or required; default required) and profile. A binding assigns each input from a mapped parameter or a constant and may take its repository from a parameter, which must name a configured repo; both are checked when the binding is saved and again at dispatch. Inputs travel as structured task data (tasks.inputs, the task brief, and a JSON block in the agent's mission), never as body text. A task with no repository runs in a per-task scratch workspace with no clone and no publication grant. The agent.run step runs one mission without a repository and finishes the task in the new completed status. Agent profiles live under [containers.profiles.<name>]: an image and an allowlist over the tools archie adds (MCP, scripts, plugins). [containers].image is the default profile; an unconfigured profile parks the task. The workflow is now pinned before the container starts so the profile can choose its image. Also stop two archied tests writing embedded NATS state into the package directory, refuse an empty state_dir for embedded NATS, and drop the NATS store files an earlier lane commit tracked.
This commit is contained in:
118 files changed
+2163
-327
No files matched your search
@@ -458,6 +458,12 @@ target = "127.0.0.1:9090" # REQUIRED. The State Store gRPC address arch
|
|||||||
# max_uptime = "60m" # container lifetime cap
|
# max_uptime = "60m" # container lifetime cap
|
||||||
# volume_ttl = "72h" # max age for persistent repo volumes and Git clone caches
|
# volume_ttl = "72h" # max age for persistent repo volumes and Git clone caches
|
||||||
# pull_policy = "missing" # "missing" or "always"
|
# pull_policy = "missing" # "missing" or "always"
|
||||||
|
#
|
||||||
|
# Agent profiles a workflow selects with `profile: <name>`. A workflow that
|
||||||
|
# names none runs under [containers].image with every tool.
|
||||||
|
# [containers.profiles.network-investigator]
|
||||||
|
# image = "ghcr.io/samcharles93/archie-agent-net:latest" # empty uses [containers].image
|
||||||
|
# tools = ["whois", "dns_lookup"] # allowlist over MCP, script and plugin tools; empty allows all
|
||||||
|
|
||||||
# At least one [[repos]] entry is required before archied will poll
|
# At least one [[repos]] entry is required before archied will poll
|
||||||
# anything -- it is deliberately left commented rather than shipped active:
|
# anything -- it is deliberately left commented rather than shipped active:
|
||||||
|
|||||||
@@ -244,7 +244,7 @@
|
|||||||
"url": "/dev/prds/event-automation/",
|
"url": "/dev/prds/event-automation/",
|
||||||
"title": "Event automation: identified events, typed parameters, purpose-built agents",
|
"title": "Event automation: identified events, typed parameters, purpose-built agents",
|
||||||
"section": "PRDs",
|
"section": "PRDs",
|
||||||
"body": "# Event automation: identified events, typed parameters, purpose-built agents\n\n**Status:** Approved\n**Date:** 2026-09-23\n**Parent:** epic `archie-core-t2db`\n\n## Why this exists\n\nArchie is an event-driven automation system. Any system that can send a\nwebhook (a forge, an error tracker, a firewall, a monitoring stack) should be\nable to start an agent built for that job. The shape this replaces cannot:\n\n- A binding matches only the URL segment an event was posted to, and a source\n may have one binding. Two kinds of event from one sender cannot go to\n different workflows.\n- A mapping is a list of JSON paths written against one example payload.\n Nothing recognises what kind of event arrived.\n- Mapped values reach the task only as `key=value` lines in its body text.\n- Every task needs a repository and runs in a git worktree.\n- Every agent runs the same image with the same forwarded keys.\n\n## The model\n\n```\nsource ─▶ event type ─▶ mapping ─▶ binding ─▶ workflow ─▶ agent profile\n (identified) (params) (+ filter, (inputs, (image, tools,\n repo) profile) secrets, access)\n```\n\n### Sources\n\nA source is the endpoint one sender posts to. Its path is a generated UUIDv7\n(`/webhooks/capture/0199…`), so an endpoint cannot be guessed or collide. The\noperator may replace it with a custom path for a sender that cannot use that\nformat; a custom path must be unique and URL-safe, and is refused otherwise.\nA source owns the signing setting:\n\n- **Signed** (default). Archie generates the secret. Only events with a valid\n `X-Hub-Signature-256` or `X-Signature-256` HMAC are dispatched.\n- **Unsigned.** The operator turns signing off for a sender that cannot sign.\n The source and every event from it are marked unsigned in the inspector,\n the binding list and the task timeline. Turning signing off needs the same\n approval as arming a binding.\n\nUnauthenticated events on a signed source are captured and never\ndispatched.\n\n### Event types\n\nAn event type is a named kind of event from one source, such as\n`github / pull_request.opened` or `firewall / blocked_connection`. It is\ndefined by a match rule:\n\n- header equality (`X-GitHub-Event = pull_request`), and\n- payload conditions (`action = \"opened\"`, or a path being present).\n\nAn event type can be created two ways:\n\n- **From an example payload.** The operator pastes a payload (and optionally\n its headers), and archie converts it to the type's schema. No event needs to\n have arrived.\n- **From captures.** Archie proposes types from what has arrived, as below.\n\nEither way, archie infers the schema, not the operator:\n\n- Each capture gets a structural signature: the set of JSON paths with their\n value types, plus the headers that look like discriminators (`*-Event`,\n `*-Type`, `Content-Type`).\n- Captures from one source that share a signature are grouped. The inspector\n shows each group as a proposed event type, with its inferred schema, its\n number of events and a sample, until the operator names it or dismisses it.\n- The proposed match rule is the set of discriminators that separates the\n group from its siblings. The operator can edit it before saving.\n- A capture that matches no named type is shown as unidentified. It is never\n dispatched.\n\nTwo event types on one source whose rules can match the same event are\nrefused when the second one is saved.\n\n### Mappings\n\nA mapping belongs to an event type. It maps payload paths to named, typed\nparameters, and it is checked against the type's inferred schema, so a path\nthe type does not have is refused when the mapping is saved. Any number of\nbindings may reuse one mapping.\n\nEach mapping shows how many events it has matched, with the time of the most\nrecent, so an operator can see which mappings are live and which have never\nfired.\n\n### Workflows declare inputs\n\nA workflow definition declares its inputs:\n\n```yaml\nid: firewall-investigate\ninputs:\n src_ip: { type: string, required: true }\n severity: { type: string }\nrepository: none # none | optional | required\nprofile: network-investigator\nsteps: ...\n```\n\n- `repository: none` runs the agent in a scratch workspace with no clone.\n `required` gives the agent a worktree of the task's repository.\n- The declared inputs reach the agent as structured data in the task brief\n and in the agent's context. They are not body text.\n- A workflow may declare `outputs`, the structured result it returns when it\n finishes.\n\n### Workflows call workflows\n\nA workflow step can call another workflow. The call passes the callee's\ninputs, and the callee runs as its own run with its own agent profile, so one\nevent can start an investigator that then calls a containment workflow:\n\n```yaml\nsteps:\n - type: workflow.call\n workflow: firewall-contain\n inputs: { src_ip: inputs.src_ip }\n Line truncated
|
"body": "# Event automation: identified events, typed parameters, purpose-built agents\n\n**Status:** Approved\n**Date:** 2026-09-23\n**Parent:** epic `archie-core-t2db`\n\n## Why this exists\n\nArchie is an event-driven automation system. Any system that can send a\nwebhook (a forge, an error tracker, a firewall, a monitoring stack) should be\nable to start an agent built for that job. The shape this replaces cannot:\n\n- A binding matches only the URL segment an event was posted to, and a source\n may have one binding. Two kinds of event from one sender cannot go to\n different workflows.\n- A mapping is a list of JSON paths written against one example payload.\n Nothing recognises what kind of event arrived.\n- Mapped values reach the task only as `key=value` lines in its body text.\n- Every task needs a repository and runs in a git worktree.\n- Every agent runs the same image with the same forwarded keys.\n\n## The model\n\n```\nsource ─▶ event type ─▶ mapping ─▶ binding ─▶ workflow ─▶ agent profile\n (identified) (params) (+ filter, (inputs, (image, tools,\n repo) profile) secrets, access)\n```\n\n### Sources\n\nA source is the endpoint one sender posts to. Its path is a generated UUIDv7\n(`/webhooks/capture/0199…`), so an endpoint cannot be guessed or collide. The\noperator may replace it with a custom path for a sender that cannot use that\nformat; a custom path must be unique and URL-safe, and is refused otherwise.\nA source owns the signing setting:\n\n- **Signed** (default). Archie generates the secret. Only events with a valid\n `X-Hub-Signature-256` or `X-Signature-256` HMAC are dispatched.\n- **Unsigned.** The operator turns signing off for a sender that cannot sign.\n The source and every event from it are marked unsigned in the inspector,\n the binding list and the task timeline. Turning signing off needs the same\n approval as arming a binding.\n\nUnauthenticated events on a signed source are captured and never\ndispatched.\n\n### Event types\n\nAn event type is a named kind of event from one source, such as\n`github / pull_request.opened` or `firewall / blocked_connection`. It is\ndefined by a match rule:\n\n- header equality (`X-GitHub-Event = pull_request`), and\n- payload conditions (`action = \"opened\"`, or a path being present).\n\nAn event type can be created two ways:\n\n- **From an example payload.** The operator pastes a payload (and optionally\n its headers), and archie converts it to the type's schema. No event needs to\n have arrived.\n- **From captures.** Archie proposes types from what has arrived, as below.\n\nEither way, archie infers the schema, not the operator:\n\n- Each capture gets a structural signature: the set of JSON paths with their\n value types, plus the headers that look like discriminators (`*-Event`,\n `*-Type`, `Content-Type`).\n- Captures from one source that share a signature are grouped. The inspector\n shows each group as a proposed event type, with its inferred schema, its\n number of events and a sample, until the operator names it or dismisses it.\n- The proposed match rule is the set of discriminators that separates the\n group from its siblings. The operator can edit it before saving.\n- A capture that matches no named type is shown as unidentified. It is never\n dispatched.\n\nTwo event types on one source whose rules can match the same event are\nrefused when the second one is saved.\n\n### Mappings\n\nA mapping belongs to an event type. It maps payload paths to named, typed\nparameters, and it is checked against the type's inferred schema, so a path\nthe type does not have is refused when the mapping is saved. Any number of\nbindings may reuse one mapping.\n\nEach mapping shows how many events it has matched, with the time of the most\nrecent, so an operator can see which mappings are live and which have never\nfired.\n\n### Workflows declare inputs\n\nA workflow definition declares its inputs:\n\n```yaml\nid: firewall-investigate\ninputs:\n src_ip: { type: string, required: true }\n severity: { type: string }\nrepository: none # none | optional | required\nprofile: network-investigator\nsteps: ...\n```\n\n- `repository: none` runs the agent in a scratch workspace with no clone.\n `required`, the default, gives the agent a worktree of the task's\n repository. `optional` gives it a worktree when the binding names a\n repository and a scratch workspace when it does not.\n- A workflow whose `repository` is not `required` may use only steps that need\n no repository. `agent.run` is one: it runs one agent mission and finishes\n the task as done, with no pull request.\n- The declared inputs reach the agent as structured data in the task brief\n and in the agent's context. They are not body text.\n- A workflow may declare `outputs`, the structured result it returns when it\n finishes.\n\n### Workflows call workflows\n\nA workflow steLine truncated
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"url": "/dev/prds/event-capture-storage/",
|
"url": "/dev/prds/event-capture-storage/",
|
||||||
|
|||||||
@@ -103,7 +103,12 @@ steps: ...
|
|||||||
```
|
```
|
||||||
|
|
||||||
- `repository: none` runs the agent in a scratch workspace with no clone.
|
- `repository: none` runs the agent in a scratch workspace with no clone.
|
||||||
`required` gives the agent a worktree of the task's repository.
|
`required`, the default, gives the agent a worktree of the task's
|
||||||
|
repository. `optional` gives it a worktree when the binding names a
|
||||||
|
repository and a scratch workspace when it does not.
|
||||||
|
- A workflow whose `repository` is not `required` may use only steps that need
|
||||||
|
no repository. `agent.run` is one: it runs one agent mission and finishes
|
||||||
|
the task as done, with no pull request.
|
||||||
- The declared inputs reach the agent as structured data in the task brief
|
- The declared inputs reach the agent as structured data in the task brief
|
||||||
and in the agent's context. They are not body text.
|
and in the agent's context. They are not body text.
|
||||||
- A workflow may declare `outputs`, the structured result it returns when it
|
- A workflow may declare `outputs`, the structured result it returns when it
|
||||||
@@ -142,7 +147,8 @@ A binding connects an event type to one workflow:
|
|||||||
checked against the workflow's declared inputs when the binding is saved;
|
checked against the workflow's declared inputs when the binding is saved;
|
||||||
- a repository, when the workflow's `repository` is not `none`: either fixed
|
- a repository, when the workflow's `repository` is not `none`: either fixed
|
||||||
(`acme/api`) or taken from a parameter (the GitHub webhook's
|
(`acme/api`) or taken from a parameter (the GitHub webhook's
|
||||||
`repository.full_name`).
|
`repository.full_name`). A repository taken from a parameter must be a
|
||||||
|
configured one, or the event does not dispatch.
|
||||||
|
|
||||||
Any number of bindings may use one event type. Approval and re-approval after
|
Any number of bindings may use one event type. Approval and re-approval after
|
||||||
an edit are unchanged. A binding whose workflow input types no longer match is
|
an edit are unchanged. A binding whose workflow input types no longer match is
|
||||||
@@ -153,14 +159,19 @@ refused at dispatch and reported on the binding.
|
|||||||
A profile is a named execution environment that a workflow selects:
|
A profile is a named execution environment that a workflow selects:
|
||||||
|
|
||||||
- the container image;
|
- the container image;
|
||||||
- the tools the agent may call;
|
- an allowlist over the tools Archie adds to the agent: MCP servers,
|
||||||
|
repository scripts and skill plugins. The agent's own file tools are always
|
||||||
|
present;
|
||||||
- the secrets injected into it, by reference to the secret store, never
|
- the secrets injected into it, by reference to the secret store, never
|
||||||
inline (superseded by `docs/prds/orgs-and-access.md`: secrets are granted to
|
inline (superseded by `docs/prds/orgs-and-access.md`: secrets are granted to
|
||||||
identities, and a profile holds no secrets);
|
identities, and a profile holds no secrets);
|
||||||
- its network access and forge access.
|
- its network access and forge access (superseded by
|
||||||
|
`docs/prds/orgs-and-access.md`: permissions are granted to identities).
|
||||||
|
|
||||||
The global `[containers].image` becomes the default profile, so existing
|
Profiles are configured under `[containers.profiles.<name>]`. The global
|
||||||
workflows are unchanged. A secret or permission is granted to an identity, never to a binding, so an
|
`[containers].image` becomes the default profile, so existing workflows are
|
||||||
|
unchanged. A task whose workflow names an unconfigured profile is parked for
|
||||||
|
an operator. A secret or permission is granted to an identity, never to a binding, so an
|
||||||
event can never widen what an agent can do (`docs/prds/orgs-and-access.md`).
|
event can never widen what an agent can do (`docs/prds/orgs-and-access.md`).
|
||||||
|
|
||||||
### Workflows belong to orgs
|
### Workflows belong to orgs
|
||||||
@@ -178,7 +189,7 @@ binding list.
|
|||||||
2. Mappings per event type, and multiple bindings per source.
|
2. Mappings per event type, and multiple bindings per source.
|
||||||
3. Workflow inputs, structured parameters in the task brief, and
|
3. Workflow inputs, structured parameters in the task brief, and
|
||||||
`repository: none | optional | required`.
|
`repository: none | optional | required`.
|
||||||
4. Agent profiles with image, tools, secret references and access.
|
4. Agent profiles with image and tools.
|
||||||
5. Signing as a source setting, with the unsigned option; UUIDv7 source paths
|
5. Signing as a source setting, with the unsigned option; UUIDv7 source paths
|
||||||
with a custom-path override.
|
with a custom-path override.
|
||||||
6. Workflows calling workflows, with declared outputs.
|
6. Workflows calling workflows, with declared outputs.
|
||||||
|
|||||||
@@ -40,6 +40,11 @@ type LoopRunner struct {
|
|||||||
// Limits bounds the size of tool results fed back into the stage. The
|
// Limits bounds the size of tool results fed back into the stage. The
|
||||||
// zero value applies no limits.
|
// zero value applies no limits.
|
||||||
Limits ToolLimits
|
Limits ToolLimits
|
||||||
|
// AllowTools is the agent profile's allowlist over the tools archie adds
|
||||||
|
// (central/MCP, repository scripts and skill plugins). Empty allows them
|
||||||
|
// all. A stage's capture tools are how it returns structured results, so
|
||||||
|
// they are never filtered.
|
||||||
|
AllowTools []string
|
||||||
}
|
}
|
||||||
|
|
||||||
func NewLoopRunner(
|
func NewLoopRunner(
|
||||||
@@ -114,10 +119,10 @@ func (r *LoopRunner) Run(ctx context.Context, workspace string, req Request, rep
|
|||||||
ReadOnly: req.ReadOnly,
|
ReadOnly: req.ReadOnly,
|
||||||
ProtectPaths: protectionMatcher(req.Protection, req.ReadOnly),
|
ProtectPaths: protectionMatcher(req.Protection, req.ReadOnly),
|
||||||
Extra: mergeToolSets(
|
Extra: mergeToolSets(
|
||||||
centralTools,
|
allowedTools(centralTools, r.AllowTools),
|
||||||
captureTools,
|
captureTools,
|
||||||
scriptTools,
|
allowedTools(scriptTools, r.AllowTools),
|
||||||
pluginTools,
|
allowedTools(pluginTools, r.AllowTools),
|
||||||
),
|
),
|
||||||
Logger: r.logger(req),
|
Logger: r.logger(req),
|
||||||
})
|
})
|
||||||
@@ -390,6 +395,20 @@ func scriptToolSet(workspace string) core.ToolSet {
|
|||||||
|
|
||||||
// mergeToolSets combines tool sets into one; later sets win on name
|
// mergeToolSets combines tool sets into one; later sets win on name
|
||||||
// collisions.
|
// collisions.
|
||||||
|
// allowedTools keeps the tools allow names; an empty allow keeps them all.
|
||||||
|
func allowedTools(set core.ToolSet, allow []string) core.ToolSet {
|
||||||
|
if len(allow) == 0 {
|
||||||
|
return set
|
||||||
|
}
|
||||||
|
kept := core.ToolSet{}
|
||||||
|
for name, tool := range set {
|
||||||
|
if slices.Contains(allow, name) {
|
||||||
|
kept[name] = tool
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return kept
|
||||||
|
}
|
||||||
|
|
||||||
func mergeToolSets(sets ...core.ToolSet) core.ToolSet {
|
func mergeToolSets(sets ...core.ToolSet) core.ToolSet {
|
||||||
merged := core.ToolSet{}
|
merged := core.ToolSet{}
|
||||||
for _, set := range sets {
|
for _, set := range sets {
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
|
"slices"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
@@ -352,3 +353,40 @@ func main() {
|
|||||||
"and calls skillscript.Run without forwarding it (issue #45)")
|
"and calls skillscript.Run without forwarding it (issue #45)")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A profile's allowlist removes the tools archie adds that it does not name,
|
||||||
|
// but never a stage's capture tools, which carry its structured result.
|
||||||
|
func TestLoopRunnerAppliesTheProfileAllowlist(t *testing.T) {
|
||||||
|
registry := tools.NewRegistry()
|
||||||
|
for _, name := range []string{"whois", "memory_search"} {
|
||||||
|
if err := registry.Register(tools.ToolEntry{
|
||||||
|
Name: name,
|
||||||
|
Handler: func(context.Context, map[string]any) (any, error) { return "", nil },
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var extra []string
|
||||||
|
runner := &LoopRunner{
|
||||||
|
runtime: runtime.NewRuntime(runtime.Config{}),
|
||||||
|
tools: registry,
|
||||||
|
AllowTools: []string{"whois"},
|
||||||
|
run: func(_ context.Context, cfg agentloop.Config) (agentloop.Result, error) {
|
||||||
|
for name := range cfg.Extra {
|
||||||
|
extra = append(extra, name)
|
||||||
|
}
|
||||||
|
return agentloop.Result{Status: agentloop.StatusPassed}, nil
|
||||||
|
},
|
||||||
|
}
|
||||||
|
_, err := runner.Run(context.Background(), t.TempDir(), Request{
|
||||||
|
Version: ProtocolVersion, TaskID: 1, Attempt: 1, Stage: "build", Model: "provider/model", Mission: "mission",
|
||||||
|
CaptureTools: []CaptureTool{{Name: "report", Description: "d", Parameters: json.RawMessage(`{"type":"object"}`)}},
|
||||||
|
}, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
slices.Sort(extra)
|
||||||
|
if !slices.Equal(extra, []string{"report", "whois"}) {
|
||||||
|
t.Fatalf("agent tools = %v, want the allowed whois and the capture tool", extra)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -198,12 +198,13 @@ func newTaskRunner(providers map[string]agentexec.Provider, log *slog.Logger) ag
|
|||||||
return agentexec.NewLoopRunner(agentexec.NewRuntime(providers), log)
|
return agentexec.NewLoopRunner(agentexec.NewRuntime(providers), log)
|
||||||
}
|
}
|
||||||
|
|
||||||
// applyToolLimits wires the task's carried tool policy into a *LoopRunner's
|
// applyToolLimits wires the task's carried tool policy and its agent
|
||||||
// result cap/spill, so a worker-executed stage enforces the same limits the
|
// profile's tool allowlist into a *LoopRunner: the result cap/spill, so a
|
||||||
// daemon's own chat path applies (config.Config.Tools.Policy, carried
|
// worker-executed stage enforces the same limits the daemon's own chat path
|
||||||
// non-secret via TaskConfig.ToolPolicy). A runner that isn't a *LoopRunner
|
// applies (config.Config.Tools.Policy, carried non-secret via
|
||||||
// (e.g. a test fake) is left untouched.
|
// TaskConfig.ToolPolicy), and the tools the profile allows. A runner that
|
||||||
func applyToolLimits(agent agentexec.Runner, policy config.ToolPolicy) {
|
// isn't a *LoopRunner (e.g. a test fake) is left untouched.
|
||||||
|
func applyToolLimits(agent agentexec.Runner, policy config.ToolPolicy, allow []string) {
|
||||||
runner, ok := agent.(*agentexec.LoopRunner)
|
runner, ok := agent.(*agentexec.LoopRunner)
|
||||||
if !ok {
|
if !ok {
|
||||||
return
|
return
|
||||||
@@ -212,6 +213,7 @@ func applyToolLimits(agent agentexec.Runner, policy config.ToolPolicy) {
|
|||||||
MaxResultChars: policy.MaxResultChars,
|
MaxResultChars: policy.MaxResultChars,
|
||||||
SpillDir: policy.SpillDir,
|
SpillDir: policy.SpillDir,
|
||||||
}
|
}
|
||||||
|
runner.AllowTools = allow
|
||||||
}
|
}
|
||||||
|
|
||||||
// routeTask remains available for routing-only callers. Production execution
|
// routeTask remains available for routing-only callers. Production execution
|
||||||
@@ -273,7 +275,7 @@ func runTask(ctx context.Context, req taskrun.Request, dependencies taskDependen
|
|||||||
if agent == nil {
|
if agent == nil {
|
||||||
return nil, fmt.Errorf("no agent runner configured for task %d", req.Task.ID)
|
return nil, fmt.Errorf("no agent runner configured for task %d", req.Task.ID)
|
||||||
}
|
}
|
||||||
applyToolLimits(agent, req.Cfg.ToolPolicy)
|
applyToolLimits(agent, req.Cfg.ToolPolicy, req.Tools)
|
||||||
agent = persistentRunner{Runner: agent, enabled: req.Repo.PersistentStorage}
|
agent = persistentRunner{Runner: agent, enabled: req.Repo.PersistentStorage}
|
||||||
|
|
||||||
// A workflow run in this process publishes to an in-process *events.Bus
|
// A workflow run in this process publishes to an in-process *events.Bus
|
||||||
|
|||||||
@@ -220,18 +220,21 @@ func TestApplyToolLimitsWiresConfiguredPolicyIntoLoopRunner(t *testing.T) {
|
|||||||
runner := agentexec.NewLoopRunner(agentexec.NewRuntime(nil), slog.New(slog.DiscardHandler))
|
runner := agentexec.NewLoopRunner(agentexec.NewRuntime(nil), slog.New(slog.DiscardHandler))
|
||||||
policy := config.ToolPolicy{MaxResultChars: 12345, SpillDir: "/var/tmp/archie-spill"}
|
policy := config.ToolPolicy{MaxResultChars: 12345, SpillDir: "/var/tmp/archie-spill"}
|
||||||
|
|
||||||
applyToolLimits(runner, policy)
|
applyToolLimits(runner, policy, []string{"whois"})
|
||||||
|
|
||||||
want := agentexec.ToolLimits{MaxResultChars: 12345, SpillDir: "/var/tmp/archie-spill"}
|
want := agentexec.ToolLimits{MaxResultChars: 12345, SpillDir: "/var/tmp/archie-spill"}
|
||||||
if runner.Limits != want {
|
if runner.Limits != want {
|
||||||
t.Fatalf("runner.Limits = %+v, want %+v", runner.Limits, want)
|
t.Fatalf("runner.Limits = %+v, want %+v", runner.Limits, want)
|
||||||
}
|
}
|
||||||
|
if len(runner.AllowTools) != 1 || runner.AllowTools[0] != "whois" {
|
||||||
|
t.Fatalf("runner.AllowTools = %v, want the profile's allowlist", runner.AllowTools)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// TestApplyToolLimitsIgnoresNonLoopRunner confirms a test-fake runner (which
|
// TestApplyToolLimitsIgnoresNonLoopRunner confirms a test-fake runner (which
|
||||||
// is not a *agentexec.LoopRunner) is left untouched rather than panicking.
|
// is not a *agentexec.LoopRunner) is left untouched rather than panicking.
|
||||||
func TestApplyToolLimitsIgnoresNonLoopRunner(t *testing.T) {
|
func TestApplyToolLimitsIgnoresNonLoopRunner(t *testing.T) {
|
||||||
applyToolLimits(panicRunner{t}, config.ToolPolicy{MaxResultChars: 1})
|
applyToolLimits(panicRunner{t}, config.ToolPolicy{MaxResultChars: 1}, nil)
|
||||||
}
|
}
|
||||||
|
|
||||||
type panicRunner struct{ t *testing.T }
|
type panicRunner struct{ t *testing.T }
|
||||||
|
|||||||
@@ -596,6 +596,12 @@ func (b *boot) connectNATS(ctx context.Context) error { //nolint:nestif // embed
|
|||||||
// registers the client close, so the client closes first (cleanups run LIFO).
|
// registers the client close, so the client closes first (cleanups run LIFO).
|
||||||
func (b *boot) startEmbeddedNATS(ctx context.Context) (string, string, error) {
|
func (b *boot) startEmbeddedNATS(ctx context.Context) (string, string, error) {
|
||||||
cfg, log := b.cfg, b.log
|
cfg, log := b.cfg, b.log
|
||||||
|
// An empty state_dir would put the store and its endpoint file in the
|
||||||
|
// process's working directory. Defaults always set it, so empty means a
|
||||||
|
// caller skipped them.
|
||||||
|
if cfg.StateDir == "" {
|
||||||
|
return "", "", errors.New("embedded nats: state_dir is required")
|
||||||
|
}
|
||||||
host := ""
|
host := ""
|
||||||
if b.containerPool != nil {
|
if b.containerPool != nil {
|
||||||
host = b.containerPool.HostGateway()
|
host = b.containerPool.HostGateway()
|
||||||
|
|||||||
@@ -9,7 +9,6 @@ import (
|
|||||||
"log/slog"
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
@@ -118,8 +117,8 @@ func TestConnectEmbeddedNATSUsesManagedWorkerBridge(t *testing.T) {
|
|||||||
|
|
||||||
b := &boot{
|
b := &boot{
|
||||||
cfg: config.Config{
|
cfg: config.Config{
|
||||||
DBPath: filepath.Join(t.TempDir(), "archie.db"),
|
StateDir: t.TempDir(),
|
||||||
NATS: config.NATSConfig{Mode: config.NATSModeEmbedded},
|
NATS: config.NATSConfig{Mode: config.NATSModeEmbedded},
|
||||||
},
|
},
|
||||||
log: slog.New(slog.DiscardHandler),
|
log: slog.New(slog.DiscardHandler),
|
||||||
containerPool: pool,
|
containerPool: pool,
|
||||||
|
|||||||
@@ -1 +0,0 @@
|
|||||||
{"url":"nats://127.0.0.1:34001","token":"UTTAKZ3FQ3FSB7D6XARWKTYMUQ"}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
{"Created":"2026-09-24T04:45:55.407149438Z","name":"ARCHIE_REACTIONS","subjects":["archie.reaction.\u003e"],"retention":"limits","max_consumers":-1,"max_msgs":-1,"max_bytes":-1,"max_age":86400000000000,"max_msgs_per_subject":-1,"max_msg_size":-1,"discard":"old","storage":"file","num_replicas":1,"duplicate_window":120000000000,"compression":"none","allow_direct":false,"mirror_direct":false,"sealed":false,"deny_delete":false,"deny_purge":false,"allow_rollup_hdrs":false,"consumer_limits":{},"allow_msg_ttl":false,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
e5007a6989578acf
|
|
||||||
Binary file not shown.
@@ -1 +0,0 @@
|
|||||||
{"Created":"0001-01-01T00:00:00Z","Name":"","name":"2AiYsxvu","deliver_policy":"all","ack_policy":"explicit","ack_wait":30000000000,"max_deliver":-1,"filter_subject":"archie.reaction.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"inactive_threshold":5000000000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
1a41fdece74230be
|
|
||||||
Binary file not shown.
@@ -1 +0,0 @@
|
|||||||
{"Created":"0001-01-01T00:00:00Z","Name":"","name":"4ETGGNq6","deliver_policy":"all","ack_policy":"explicit","ack_wait":30000000000,"max_deliver":-1,"filter_subject":"archie.reaction.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"inactive_threshold":5000000000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
59023e4aa2a291e6
|
|
||||||
Binary file not shown.
@@ -1 +0,0 @@
|
|||||||
{"Created":"0001-01-01T00:00:00Z","Name":"","name":"8MNb4kc2","deliver_policy":"all","ack_policy":"explicit","ack_wait":30000000000,"max_deliver":-1,"filter_subject":"archie.reaction.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"inactive_threshold":5000000000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
461f1f0b33b1e704
|
|
||||||
Binary file not shown.
@@ -1 +0,0 @@
|
|||||||
{"Created":"2026-09-24T09:46:47.593915791Z","Name":"Bm6IdqYw","name":"Bm6IdqYw","deliver_policy":"all","ack_policy":"explicit","ack_wait":30000000000,"max_deliver":-1,"filter_subject":"archie.reaction.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
8b3c3f8234fa0b40
|
|
||||||
Binary file not shown.
@@ -1 +0,0 @@
|
|||||||
{"Created":"0001-01-01T00:00:00Z","Name":"","name":"QZFaSGZ6","deliver_policy":"all","ack_policy":"explicit","ack_wait":30000000000,"max_deliver":-1,"filter_subject":"archie.reaction.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"inactive_threshold":5000000000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
fb9d97471241c535
|
|
||||||
Binary file not shown.
-1
@@ -1 +0,0 @@
|
|||||||
{"Created":"2026-09-24T09:46:47.587065821Z","Name":"archie-daemon","durable_name":"archie-daemon","name":"archie-daemon","deliver_policy":"all","ack_policy":"explicit","ack_wait":300000000000,"max_deliver":3,"filter_subject":"archie.reaction.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"inactive_threshold":86400000000000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
-1
@@ -1 +0,0 @@
|
|||||||
5eb449f03c31a0f7
|
|
||||||
BIN
Binary file not shown.
@@ -1 +0,0 @@
|
|||||||
{"Created":"2026-09-24T09:46:47.594292859Z","Name":"eq86DXAG","name":"eq86DXAG","deliver_policy":"all","ack_policy":"explicit","ack_wait":30000000000,"max_deliver":-1,"filter_subject":"archie.reaction.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
5ff54a1a5c91cc56
|
|
||||||
Binary file not shown.
@@ -1 +0,0 @@
|
|||||||
{"Created":"0001-01-01T00:00:00Z","Name":"","name":"jRzCdtm6","deliver_policy":"all","ack_policy":"explicit","ack_wait":30000000000,"max_deliver":-1,"filter_subject":"archie.reaction.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"inactive_threshold":5000000000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
abaa43ff23e8df46
|
|
||||||
Binary file not shown.
@@ -1 +0,0 @@
|
|||||||
{"Created":"0001-01-01T00:00:00Z","Name":"","name":"kmNsDMt5","deliver_policy":"all","ack_policy":"explicit","ack_wait":30000000000,"max_deliver":-1,"filter_subject":"archie.reaction.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"inactive_threshold":5000000000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
60d5c7722ea0e3a4
|
|
||||||
Binary file not shown.
@@ -1 +0,0 @@
|
|||||||
{"Created":"0001-01-01T00:00:00Z","Name":"","name":"nYF2oUak","deliver_policy":"all","ack_policy":"explicit","ack_wait":30000000000,"max_deliver":-1,"filter_subject":"archie.reaction.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"inactive_threshold":5000000000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
52d9f8cea485537a
|
|
||||||
Binary file not shown.
@@ -1 +0,0 @@
|
|||||||
{"Created":"0001-01-01T00:00:00Z","Name":"","name":"s3Z87297","deliver_policy":"all","ack_policy":"explicit","ack_wait":30000000000,"max_deliver":-1,"filter_subject":"archie.reaction.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"inactive_threshold":5000000000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
218aa24a9029a5b8
|
|
||||||
Binary file not shown.
@@ -1 +0,0 @@
|
|||||||
{"Created":"0001-01-01T00:00:00Z","Name":"","name":"sg9Gl4th","deliver_policy":"all","ack_policy":"explicit","ack_wait":30000000000,"max_deliver":-1,"filter_subject":"archie.reaction.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"inactive_threshold":5000000000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
4719ba6cad8cfe3c
|
|
||||||
Binary file not shown.
@@ -1 +0,0 @@
|
|||||||
{"Created":"0001-01-01T00:00:00Z","Name":"","name":"vn0YoRC5","deliver_policy":"all","ack_policy":"explicit","ack_wait":30000000000,"max_deliver":-1,"filter_subject":"archie.reaction.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"inactive_threshold":5000000000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
98548a1ca3a96fc3
|
|
||||||
Binary file not shown.
@@ -1 +0,0 @@
|
|||||||
{"Created":"2026-09-24T04:45:55.396149543Z","name":"ARCHIE_TASKS","subjects":["archie.task.\u003e"],"retention":"workqueue","max_consumers":-1,"max_msgs":-1,"max_bytes":-1,"max_age":0,"max_msgs_per_subject":-1,"max_msg_size":-1,"discard":"old","storage":"file","num_replicas":1,"duplicate_window":120000000000,"compression":"none","allow_direct":false,"mirror_direct":false,"sealed":false,"deny_delete":false,"deny_purge":false,"allow_rollup_hdrs":false,"consumer_limits":{},"allow_msg_ttl":false,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
c93aec38b44ea7d3
|
|
||||||
Whitespace-only changes.
@@ -1 +0,0 @@
|
|||||||
{"Created":"2026-09-24T09:46:47.585493669Z","Name":"archie-daemon","durable_name":"archie-daemon","name":"archie-daemon","deliver_policy":"all","ack_policy":"explicit","ack_wait":300000000000,"max_deliver":3,"filter_subject":"archie.task.\u003e","replay_policy":"instant","max_waiting":512,"max_ack_pending":1000,"inactive_threshold":86400000000000,"num_replicas":0,"metadata":{"_nats.req.level":"0"}}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
39a729e190b2fb96
|
|
||||||
Binary file not shown.
@@ -2,7 +2,6 @@ package archied
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"path/filepath"
|
|
||||||
"slices"
|
"slices"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
@@ -29,8 +28,8 @@ import (
|
|||||||
func TestConnectNATSCreatesReactionStream(t *testing.T) {
|
func TestConnectNATSCreatesReactionStream(t *testing.T) {
|
||||||
b := &boot{
|
b := &boot{
|
||||||
cfg: config.Config{
|
cfg: config.Config{
|
||||||
DBPath: filepath.Join(t.TempDir(), "archie.db"),
|
StateDir: t.TempDir(),
|
||||||
NATS: config.NATSConfig{Mode: config.NATSModeEmbedded},
|
NATS: config.NATSConfig{Mode: config.NATSModeEmbedded},
|
||||||
},
|
},
|
||||||
log: slog.New(slog.DiscardHandler),
|
log: slog.New(slog.DiscardHandler),
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -117,6 +117,6 @@ func validateContainers(input []byte) error {
|
|||||||
if strings.TrimSpace(settings.Image) == "" {
|
if strings.TrimSpace(settings.Image) == "" {
|
||||||
return fmt.Errorf("container image is required for autonomous workflow workers")
|
return fmt.Errorf("container image is required for autonomous workflow workers")
|
||||||
}
|
}
|
||||||
return nil
|
return settings.ValidateProfiles()
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -906,6 +906,54 @@ type ContainerConfig struct {
|
|||||||
// gateway; external deployments can set a Compose network explicitly when
|
// gateway; external deployments can set a Compose network explicitly when
|
||||||
// workers must resolve broker service names (e.g. "archie-core_default").
|
// workers must resolve broker service names (e.g. "archie-core_default").
|
||||||
Network string `toml:"network" yaml:"network"`
|
Network string `toml:"network" yaml:"network"`
|
||||||
|
// Profiles are the named agent profiles a workflow selects with
|
||||||
|
// `profile:`. A workflow that names none runs under the default profile:
|
||||||
|
// Image and every tool.
|
||||||
|
Profiles map[string]AgentProfile `toml:"profiles" yaml:"profiles" json:"profiles,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// AgentProfile is a named execution environment for an agent. Secrets and
|
||||||
|
// forge or network access are granted to identities, not profiles
|
||||||
|
// (docs/prds/orgs-and-access.md).
|
||||||
|
type AgentProfile struct {
|
||||||
|
// Image is the container image; empty means [containers].image.
|
||||||
|
Image string `toml:"image" yaml:"image" json:"image,omitempty"`
|
||||||
|
// Tools allowlists the tools archie adds to the agent (MCP servers,
|
||||||
|
// repository scripts and skill plugins) by name. Empty allows them all.
|
||||||
|
// The agent loop's own file tools are always present, read-only when a
|
||||||
|
// step asks for that.
|
||||||
|
Tools []string `toml:"tools" yaml:"tools" json:"tools,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Profile resolves the profile a workflow names: "" is the default profile,
|
||||||
|
// and an unconfigured name is an error.
|
||||||
|
func (c ContainerConfig) Profile(name string) (AgentProfile, error) {
|
||||||
|
if name == "" {
|
||||||
|
return AgentProfile{Image: c.Image}, nil
|
||||||
|
}
|
||||||
|
p, ok := c.Profiles[name]
|
||||||
|
if !ok {
|
||||||
|
return AgentProfile{}, fmt.Errorf("agent profile %q is not configured", name)
|
||||||
|
}
|
||||||
|
if p.Image == "" {
|
||||||
|
p.Image = c.Image
|
||||||
|
}
|
||||||
|
return p, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ValidateProfiles rejects a profile with an empty name or an empty tool name.
|
||||||
|
func (c ContainerConfig) ValidateProfiles() error {
|
||||||
|
for name, p := range c.Profiles {
|
||||||
|
if strings.TrimSpace(name) == "" {
|
||||||
|
return fmt.Errorf("containers.profiles: a profile name must not be empty")
|
||||||
|
}
|
||||||
|
for _, tool := range p.Tools {
|
||||||
|
if strings.TrimSpace(tool) == "" {
|
||||||
|
return fmt.Errorf("containers.profiles.%s.tools: a tool name must not be empty", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Log configures where archied writes its logs.
|
// Log configures where archied writes its logs.
|
||||||
|
|||||||
@@ -0,0 +1,41 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"slices"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestContainerProfileResolution(t *testing.T) {
|
||||||
|
c := ContainerConfig{Image: "archie-agent:1", Profiles: map[string]AgentProfile{
|
||||||
|
"net": {Image: "archie-agent-net:1", Tools: []string{"whois"}},
|
||||||
|
"quiet": {Tools: []string{"notes"}},
|
||||||
|
}}
|
||||||
|
tests := []struct {
|
||||||
|
name, profile, wantImage, wantErr string
|
||||||
|
wantTools []string
|
||||||
|
}{
|
||||||
|
{name: "default profile", profile: "", wantImage: "archie-agent:1"},
|
||||||
|
{name: "named profile", profile: "net", wantImage: "archie-agent-net:1", wantTools: []string{"whois"}},
|
||||||
|
{name: "profile without an image uses the default image", profile: "quiet", wantImage: "archie-agent:1", wantTools: []string{"notes"}},
|
||||||
|
{name: "unconfigured profile", profile: "gone", wantErr: `"gone" is not configured`},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
p, err := c.Profile(tt.profile)
|
||||||
|
if tt.wantErr != "" {
|
||||||
|
if err == nil || !strings.Contains(err.Error(), tt.wantErr) {
|
||||||
|
t.Fatalf("Profile() error = %v, want containing %q", err, tt.wantErr)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil || p.Image != tt.wantImage || !slices.Equal(p.Tools, tt.wantTools) {
|
||||||
|
t.Fatalf("Profile() = %+v, %v; want image %q tools %v", p, err, tt.wantImage, tt.wantTools)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
bad := ContainerConfig{Profiles: map[string]AgentProfile{"x": {Tools: []string{" "}}}}
|
||||||
|
if err := bad.ValidateProfiles(); err == nil {
|
||||||
|
t.Error("ValidateProfiles accepted an empty tool name")
|
||||||
|
}
|
||||||
|
}
|
||||||
+46
-12
@@ -39,6 +39,8 @@ type TaskPayload struct {
|
|||||||
Workflow string `json:"workflow"`
|
Workflow string `json:"workflow"`
|
||||||
Branch string `json:"branch,omitempty"`
|
Branch string `json:"branch,omitempty"`
|
||||||
Plan string `json:"plan,omitempty"`
|
Plan string `json:"plan,omitempty"`
|
||||||
|
// Inputs are the workflow inputs a binding assigned, as structured data.
|
||||||
|
Inputs map[string]any `json:"inputs,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// WriteTaskJSON writes the task payload to <workspace>/.git/task.json.
|
// WriteTaskJSON writes the task payload to <workspace>/.git/task.json.
|
||||||
@@ -88,6 +90,9 @@ type Pool struct {
|
|||||||
|
|
||||||
mu sync.Mutex
|
mu sync.Mutex
|
||||||
active int
|
active int
|
||||||
|
// pulled records the images already made available, so a profile's image
|
||||||
|
// is pulled once rather than on every task. Guarded by mu.
|
||||||
|
pulled map[string]bool
|
||||||
// teardowns holds the one-shot teardown state for each live container,
|
// teardowns holds the one-shot teardown state for each live container,
|
||||||
// keyed by ID. See containerTeardown.
|
// keyed by ID. See containerTeardown.
|
||||||
teardowns map[string]*containerTeardown
|
teardowns map[string]*containerTeardown
|
||||||
@@ -161,7 +166,7 @@ func NewPool(ctx context.Context, cfg Config, log *slog.Logger) (*Pool, error) {
|
|||||||
|
|
||||||
// Pull image if needed.
|
// Pull image if needed.
|
||||||
if cfg.PullPolicy == "always" || cfg.PullPolicy == "missing" {
|
if cfg.PullPolicy == "always" || cfg.PullPolicy == "missing" {
|
||||||
if err := p.pullImage(ctx); err != nil {
|
if err := p.pullImage(ctx, cfg.Image); err != nil {
|
||||||
if err := cli.Close(); err != nil {
|
if err := cli.Close(); err != nil {
|
||||||
log.Warn("docker client close failed during pull error", "err", err)
|
log.Warn("docker client close failed during pull error", "err", err)
|
||||||
}
|
}
|
||||||
@@ -175,12 +180,18 @@ func NewPool(ctx context.Context, cfg Config, log *slog.Logger) (*Pool, error) {
|
|||||||
return p, nil
|
return p, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Acquire creates and starts a container with the given mounts and
|
// Acquire creates and starts a container from image (empty means the
|
||||||
// environment variables. Mounts are provided by the caller (typically
|
// configured image) with the given mounts and environment variables. Mounts
|
||||||
// from a storage.Backend). If MaxUptime is set, the pool schedules a hard
|
// are provided by the caller (typically from a storage.Backend). If MaxUptime
|
||||||
// stop and remove of the container once that lifetime cap elapses,
|
// is set, the pool schedules a hard stop and remove of the container once
|
||||||
// regardless of task state.
|
// that lifetime cap elapses, regardless of task state.
|
||||||
func (p *Pool) Acquire(ctx context.Context, mounts []storage.Mount, env []string) (*Container, error) {
|
func (p *Pool) Acquire(ctx context.Context, image string, mounts []storage.Mount, env []string) (*Container, error) {
|
||||||
|
if image == "" {
|
||||||
|
image = p.cfg.Image
|
||||||
|
}
|
||||||
|
if err := p.ensureImage(ctx, image); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
p.mu.Lock()
|
p.mu.Lock()
|
||||||
if p.cfg.MaxConcurrency > 0 && p.active >= p.cfg.MaxConcurrency {
|
if p.cfg.MaxConcurrency > 0 && p.active >= p.cfg.MaxConcurrency {
|
||||||
p.mu.Unlock()
|
p.mu.Unlock()
|
||||||
@@ -206,7 +217,7 @@ func (p *Pool) Acquire(ctx context.Context, mounts []storage.Mount, env []string
|
|||||||
resp, err := p.cli.ContainerCreate(ctx, client.ContainerCreateOptions{
|
resp, err := p.cli.ContainerCreate(ctx, client.ContainerCreateOptions{
|
||||||
Name: name,
|
Name: name,
|
||||||
Config: &container.Config{
|
Config: &container.Config{
|
||||||
Image: p.cfg.Image,
|
Image: image,
|
||||||
Env: env,
|
Env: env,
|
||||||
Labels: map[string]string{
|
Labels: map[string]string{
|
||||||
"archie-daemon": "true",
|
"archie-daemon": "true",
|
||||||
@@ -466,11 +477,34 @@ func (p *Pool) Close() error {
|
|||||||
|
|
||||||
// ── helpers ──────────────────────────────────────────────────────────
|
// ── helpers ──────────────────────────────────────────────────────────
|
||||||
|
|
||||||
// pullImage pulls the configured image. On "missing" policy, skips if
|
// ensureImage pulls an image other than the configured one the first time a
|
||||||
// the image already exists locally.
|
// task asks for it, under the configured pull policy. The configured image
|
||||||
func (p *Pool) pullImage(ctx context.Context) error {
|
// was already pulled by NewPool.
|
||||||
ref := p.cfg.Image
|
func (p *Pool) ensureImage(ctx context.Context, ref string) error {
|
||||||
|
if ref == p.cfg.Image || (p.cfg.PullPolicy != "always" && p.cfg.PullPolicy != "missing") {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
p.mu.Lock()
|
||||||
|
done := p.pulled[ref]
|
||||||
|
p.mu.Unlock()
|
||||||
|
if done {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := p.pullImage(ctx, ref); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
p.mu.Lock()
|
||||||
|
if p.pulled == nil {
|
||||||
|
p.pulled = map[string]bool{}
|
||||||
|
}
|
||||||
|
p.pulled[ref] = true
|
||||||
|
p.mu.Unlock()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// pullImage pulls ref. On "missing" policy, skips if the image already
|
||||||
|
// exists locally.
|
||||||
|
func (p *Pool) pullImage(ctx context.Context, ref string) error {
|
||||||
if p.cfg.PullPolicy == "missing" {
|
if p.cfg.PullPolicy == "missing" {
|
||||||
_, err := p.cli.ImageInspect(ctx, ref)
|
_, err := p.cli.ImageInspect(ctx, ref)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
|
|||||||
@@ -346,7 +346,7 @@ func TestAcquireEnforcesMaxUptime(t *testing.T) {
|
|||||||
log: discardLogger(),
|
log: discardLogger(),
|
||||||
}
|
}
|
||||||
|
|
||||||
c, err := pool.Acquire(context.Background(), nil, nil)
|
c, err := pool.Acquire(context.Background(), "", nil, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Acquire: %v", err)
|
t.Fatalf("Acquire: %v", err)
|
||||||
}
|
}
|
||||||
@@ -407,7 +407,7 @@ func TestReleaseCancelsTheMaxUptimeTimer(t *testing.T) {
|
|||||||
log: discardLogger(),
|
log: discardLogger(),
|
||||||
}
|
}
|
||||||
|
|
||||||
c, err := pool.Acquire(context.Background(), nil, nil)
|
c, err := pool.Acquire(context.Background(), "", nil, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Acquire: %v", err)
|
t.Fatalf("Acquire: %v", err)
|
||||||
}
|
}
|
||||||
@@ -475,7 +475,7 @@ func TestGracePeriodDoesNotExtendTheMaxUptimeCap(t *testing.T) {
|
|||||||
log: discardLogger(),
|
log: discardLogger(),
|
||||||
}
|
}
|
||||||
|
|
||||||
c, err := pool.Acquire(context.Background(), nil, nil)
|
c, err := pool.Acquire(context.Background(), "", nil, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Acquire: %v", err)
|
t.Fatalf("Acquire: %v", err)
|
||||||
}
|
}
|
||||||
@@ -540,7 +540,7 @@ func TestReleaseAfterTheReaperFiredDoesNotTearDownTwice(t *testing.T) {
|
|||||||
log: discardLogger(),
|
log: discardLogger(),
|
||||||
}
|
}
|
||||||
|
|
||||||
c, err := pool.Acquire(context.Background(), nil, nil)
|
c, err := pool.Acquire(context.Background(), "", nil, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Acquire: %v", err)
|
t.Fatalf("Acquire: %v", err)
|
||||||
}
|
}
|
||||||
@@ -608,7 +608,7 @@ func TestReaperCallbackAfterReleaseDoesNotResurrectTeardown(t *testing.T) {
|
|||||||
log: discardLogger(),
|
log: discardLogger(),
|
||||||
}
|
}
|
||||||
|
|
||||||
c, err := pool.Acquire(context.Background(), nil, nil)
|
c, err := pool.Acquire(context.Background(), "", nil, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Acquire: %v", err)
|
t.Fatalf("Acquire: %v", err)
|
||||||
}
|
}
|
||||||
@@ -675,7 +675,7 @@ func TestPoolActiveReportsInFlightContainers(t *testing.T) {
|
|||||||
var held []*Container
|
var held []*Container
|
||||||
acquire := func(t *testing.T) {
|
acquire := func(t *testing.T) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
c, err := pool.Acquire(context.Background(), nil, nil)
|
c, err := pool.Acquire(context.Background(), "", nil, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Acquire: %v", err)
|
t.Fatalf("Acquire: %v", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1318,8 +1318,11 @@ type Task struct {
|
|||||||
// separately from retry_count, so an operator's manual retries no
|
// separately from retry_count, so an operator's manual retries no
|
||||||
// longer draw down the review-remediation budget (and vice versa).
|
// longer draw down the review-remediation budget (and vice versa).
|
||||||
RemediationRounds int32 `protobuf:"varint,33,opt,name=remediation_rounds,json=remediationRounds,proto3" json:"remediation_rounds,omitempty"`
|
RemediationRounds int32 `protobuf:"varint,33,opt,name=remediation_rounds,json=remediationRounds,proto3" json:"remediation_rounds,omitempty"`
|
||||||
unknownFields protoimpl.UnknownFields
|
// inputs_json is the JSON object of workflow inputs a binding dispatch
|
||||||
sizeCache protoimpl.SizeCache
|
// resolved (task.EncodeInputs); empty for a task no binding started.
|
||||||
|
InputsJson string `protobuf:"bytes,34,opt,name=inputs_json,json=inputsJson,proto3" json:"inputs_json,omitempty"`
|
||||||
|
unknownFields protoimpl.UnknownFields
|
||||||
|
sizeCache protoimpl.SizeCache
|
||||||
}
|
}
|
||||||
|
|
||||||
func (x *Task) Reset() {
|
func (x *Task) Reset() {
|
||||||
@@ -1583,6 +1586,13 @@ func (x *Task) GetRemediationRounds() int32 {
|
|||||||
return 0
|
return 0
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (x *Task) GetInputsJson() string {
|
||||||
|
if x != nil {
|
||||||
|
return x.InputsJson
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
// Event mirrors internal/events.Event. Data is carried as a JSON object
|
// Event mirrors internal/events.Event. Data is carried as a JSON object
|
||||||
// string (matching the store's own on-disk representation) rather than
|
// string (matching the store's own on-disk representation) rather than
|
||||||
// google.protobuf.Struct, to stay a direct mirror of the existing column.
|
// google.protobuf.Struct, to stay a direct mirror of the existing column.
|
||||||
@@ -2320,10 +2330,15 @@ type Binding struct {
|
|||||||
// Unused: signing moved to Source. Kept so the field number is never reused.
|
// Unused: signing moved to Source. Kept so the field number is never reused.
|
||||||
//
|
//
|
||||||
// Deprecated: Marked as deprecated in state/v1/state.proto.
|
// Deprecated: Marked as deprecated in state/v1/state.proto.
|
||||||
Secret string `protobuf:"bytes,10,opt,name=secret,proto3" json:"secret,omitempty"`
|
Secret string `protobuf:"bytes,10,opt,name=secret,proto3" json:"secret,omitempty"`
|
||||||
CreatedAt *timestamppb.Timestamp `protobuf:"bytes,11,opt,name=created_at,json=createdAt,proto3" json:"created_at,omitempty"`
|
CreatedAt *timestamppb.Timestamp `protobuf:"bytes,11,opt,name=created_at,json=createdAt,proto3" json:"created_at,omitempty"`
|
||||||
UpdatedAt *timestamppb.Timestamp `protobuf:"bytes,12,opt,name=updated_at,json=updatedAt,proto3" json:"updated_at,omitempty"`
|
UpdatedAt *timestamppb.Timestamp `protobuf:"bytes,12,opt,name=updated_at,json=updatedAt,proto3" json:"updated_at,omitempty"`
|
||||||
Filter string `protobuf:"bytes,13,opt,name=filter,proto3" json:"filter,omitempty"`
|
Filter string `protobuf:"bytes,13,opt,name=filter,proto3" json:"filter,omitempty"`
|
||||||
|
// inputs_json assigns workflow inputs, a JSON object of
|
||||||
|
// {"param": name} or {"value": constant} per input.
|
||||||
|
InputsJson string `protobuf:"bytes,14,opt,name=inputs_json,json=inputsJson,proto3" json:"inputs_json,omitempty"`
|
||||||
|
// repo_param names the mapped parameter holding "owner/name".
|
||||||
|
RepoParam string `protobuf:"bytes,15,opt,name=repo_param,json=repoParam,proto3" json:"repo_param,omitempty"`
|
||||||
unknownFields protoimpl.UnknownFields
|
unknownFields protoimpl.UnknownFields
|
||||||
sizeCache protoimpl.SizeCache
|
sizeCache protoimpl.SizeCache
|
||||||
}
|
}
|
||||||
@@ -2450,6 +2465,20 @@ func (x *Binding) GetFilter() string {
|
|||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (x *Binding) GetInputsJson() string {
|
||||||
|
if x != nil {
|
||||||
|
return x.InputsJson
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func (x *Binding) GetRepoParam() string {
|
||||||
|
if x != nil {
|
||||||
|
return x.RepoParam
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
// Source mirrors internal/domain/source.Source.
|
// Source mirrors internal/domain/source.Source.
|
||||||
type Source struct {
|
type Source struct {
|
||||||
state protoimpl.MessageState `protogen:"open.v1"`
|
state protoimpl.MessageState `protogen:"open.v1"`
|
||||||
@@ -9113,6 +9142,7 @@ type EnqueueBindingTaskRequest struct {
|
|||||||
Identity string `protobuf:"bytes,6,opt,name=identity,proto3" json:"identity,omitempty"`
|
Identity string `protobuf:"bytes,6,opt,name=identity,proto3" json:"identity,omitempty"`
|
||||||
BindingId string `protobuf:"bytes,7,opt,name=binding_id,json=bindingId,proto3" json:"binding_id,omitempty"`
|
BindingId string `protobuf:"bytes,7,opt,name=binding_id,json=bindingId,proto3" json:"binding_id,omitempty"`
|
||||||
BindingVersion int64 `protobuf:"varint,8,opt,name=binding_version,json=bindingVersion,proto3" json:"binding_version,omitempty"`
|
BindingVersion int64 `protobuf:"varint,8,opt,name=binding_version,json=bindingVersion,proto3" json:"binding_version,omitempty"`
|
||||||
|
InputsJson string `protobuf:"bytes,9,opt,name=inputs_json,json=inputsJson,proto3" json:"inputs_json,omitempty"`
|
||||||
unknownFields protoimpl.UnknownFields
|
unknownFields protoimpl.UnknownFields
|
||||||
sizeCache protoimpl.SizeCache
|
sizeCache protoimpl.SizeCache
|
||||||
}
|
}
|
||||||
@@ -9203,6 +9233,13 @@ func (x *EnqueueBindingTaskRequest) GetBindingVersion() int64 {
|
|||||||
return 0
|
return 0
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (x *EnqueueBindingTaskRequest) GetInputsJson() string {
|
||||||
|
if x != nil {
|
||||||
|
return x.InputsJson
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
type EnqueueBindingTaskResponse struct {
|
type EnqueueBindingTaskResponse struct {
|
||||||
state protoimpl.MessageState `protogen:"open.v1"`
|
state protoimpl.MessageState `protogen:"open.v1"`
|
||||||
Task *Task `protobuf:"bytes,1,opt,name=task,proto3" json:"task,omitempty"`
|
Task *Task `protobuf:"bytes,1,opt,name=task,proto3" json:"task,omitempty"`
|
||||||
@@ -9327,7 +9364,7 @@ const file_state_v1_state_proto_rawDesc = "" +
|
|||||||
"\asubject\x18\x03 \x01(\tR\asubject\x12-\n" +
|
"\asubject\x18\x03 \x01(\tR\asubject\x12-\n" +
|
||||||
"\x05audit\x18\x04 \x01(\v2\x17.state.v1.IdentityAuditR\x05audit\"M\n" +
|
"\x05audit\x18\x04 \x01(\v2\x17.state.v1.IdentityAuditR\x05audit\"M\n" +
|
||||||
"\x1bBindIdentitySubjectResponse\x12.\n" +
|
"\x1bBindIdentitySubjectResponse\x12.\n" +
|
||||||
"\bidentity\x18\x01 \x01(\v2\x12.state.v1.IdentityR\bidentity\"\xd9\b\n" +
|
"\bidentity\x18\x01 \x01(\v2\x12.state.v1.IdentityR\bidentity\"\xfa\b\n" +
|
||||||
"\x04Task\x12\x0e\n" +
|
"\x04Task\x12\x0e\n" +
|
||||||
"\x02id\x18\x01 \x01(\x03R\x02id\x12\x14\n" +
|
"\x02id\x18\x01 \x01(\x03R\x02id\x12\x14\n" +
|
||||||
"\x05owner\x18\x02 \x01(\tR\x05owner\x12\x12\n" +
|
"\x05owner\x18\x02 \x01(\tR\x05owner\x12\x12\n" +
|
||||||
@@ -9371,7 +9408,9 @@ const file_state_v1_state_proto_rawDesc = "" +
|
|||||||
"\rreview_cursor\x18\x1f \x01(\x03R\freviewCursor\x12\x1d\n" +
|
"\rreview_cursor\x18\x1f \x01(\x03R\freviewCursor\x12\x1d\n" +
|
||||||
"\n" +
|
"\n" +
|
||||||
"park_class\x18 \x01(\tR\tparkClass\x12-\n" +
|
"park_class\x18 \x01(\tR\tparkClass\x12-\n" +
|
||||||
"\x12remediation_rounds\x18! \x01(\x05R\x11remediationRounds\"\xf8\x02\n" +
|
"\x12remediation_rounds\x18! \x01(\x05R\x11remediationRounds\x12\x1f\n" +
|
||||||
|
"\vinputs_json\x18\" \x01(\tR\n" +
|
||||||
|
"inputsJson\"\xf8\x02\n" +
|
||||||
"\x05Event\x12\x0e\n" +
|
"\x05Event\x12\x0e\n" +
|
||||||
"\x02id\x18\x01 \x01(\x03R\x02id\x12*\n" +
|
"\x02id\x18\x01 \x01(\x03R\x02id\x12*\n" +
|
||||||
"\x02at\x18\x02 \x01(\v2\x1a.google.protobuf.TimestampR\x02at\x12\x12\n" +
|
"\x02at\x18\x02 \x01(\v2\x1a.google.protobuf.TimestampR\x02at\x12\x12\n" +
|
||||||
@@ -9445,7 +9484,7 @@ const file_state_v1_state_proto_rawDesc = "" +
|
|||||||
"matchCount\x12B\n" +
|
"matchCount\x12B\n" +
|
||||||
"\x0flast_matched_at\x18\t \x01(\v2\x1a.google.protobuf.TimestampR\rlastMatchedAt\"(\n" +
|
"\x0flast_matched_at\x18\t \x01(\v2\x1a.google.protobuf.TimestampR\rlastMatchedAt\"(\n" +
|
||||||
"\x0eBindingMatcher\x12\x16\n" +
|
"\x0eBindingMatcher\x12\x16\n" +
|
||||||
"\x06source\x18\x01 \x01(\tR\x06source\"\xa2\x03\n" +
|
"\x06source\x18\x01 \x01(\tR\x06source\"\xe2\x03\n" +
|
||||||
"\aBinding\x12\x0e\n" +
|
"\aBinding\x12\x0e\n" +
|
||||||
"\x02id\x18\x01 \x01(\tR\x02id\x12\x12\n" +
|
"\x02id\x18\x01 \x01(\tR\x02id\x12\x12\n" +
|
||||||
"\x04name\x18\x02 \x01(\tR\x04name\x122\n" +
|
"\x04name\x18\x02 \x01(\tR\x04name\x122\n" +
|
||||||
@@ -9463,7 +9502,11 @@ const file_state_v1_state_proto_rawDesc = "" +
|
|||||||
"created_at\x18\v \x01(\v2\x1a.google.protobuf.TimestampR\tcreatedAt\x129\n" +
|
"created_at\x18\v \x01(\v2\x1a.google.protobuf.TimestampR\tcreatedAt\x129\n" +
|
||||||
"\n" +
|
"\n" +
|
||||||
"updated_at\x18\f \x01(\v2\x1a.google.protobuf.TimestampR\tupdatedAt\x12\x16\n" +
|
"updated_at\x18\f \x01(\v2\x1a.google.protobuf.TimestampR\tupdatedAt\x12\x16\n" +
|
||||||
"\x06filter\x18\r \x01(\tR\x06filter\"\xc4\x01\n" +
|
"\x06filter\x18\r \x01(\tR\x06filter\x12\x1f\n" +
|
||||||
|
"\vinputs_json\x18\x0e \x01(\tR\n" +
|
||||||
|
"inputsJson\x12\x1d\n" +
|
||||||
|
"\n" +
|
||||||
|
"repo_param\x18\x0f \x01(\tR\trepoParam\"\xc4\x01\n" +
|
||||||
"\x06Source\x12\x12\n" +
|
"\x06Source\x12\x12\n" +
|
||||||
"\x04path\x18\x01 \x01(\tR\x04path\x12\x18\n" +
|
"\x04path\x18\x01 \x01(\tR\x04path\x12\x18\n" +
|
||||||
"\asigning\x18\x02 \x01(\tR\asigning\x12\x16\n" +
|
"\asigning\x18\x02 \x01(\tR\asigning\x12\x16\n" +
|
||||||
@@ -9831,7 +9874,7 @@ const file_state_v1_state_proto_rawDesc = "" +
|
|||||||
"\asources\x18\x01 \x03(\tR\asources\x12\x14\n" +
|
"\asources\x18\x01 \x03(\tR\asources\x12\x14\n" +
|
||||||
"\x05limit\x18\x02 \x01(\x03R\x05limit\"W\n" +
|
"\x05limit\x18\x02 \x01(\x03R\x05limit\"W\n" +
|
||||||
"\"StreamUndispatchedCapturesResponse\x121\n" +
|
"\"StreamUndispatchedCapturesResponse\x121\n" +
|
||||||
"\acapture\x18\x01 \x01(\v2\x17.state.v1.CapturedEventR\acapture\"\xef\x01\n" +
|
"\acapture\x18\x01 \x01(\v2\x17.state.v1.CapturedEventR\acapture\"\x90\x02\n" +
|
||||||
"\x19EnqueueBindingTaskRequest\x12\x14\n" +
|
"\x19EnqueueBindingTaskRequest\x12\x14\n" +
|
||||||
"\x05owner\x18\x01 \x01(\tR\x05owner\x12\x12\n" +
|
"\x05owner\x18\x01 \x01(\tR\x05owner\x12\x12\n" +
|
||||||
"\x04repo\x18\x02 \x01(\tR\x04repo\x12\x14\n" +
|
"\x04repo\x18\x02 \x01(\tR\x04repo\x12\x14\n" +
|
||||||
@@ -9841,7 +9884,9 @@ const file_state_v1_state_proto_rawDesc = "" +
|
|||||||
"\bidentity\x18\x06 \x01(\tR\bidentity\x12\x1d\n" +
|
"\bidentity\x18\x06 \x01(\tR\bidentity\x12\x1d\n" +
|
||||||
"\n" +
|
"\n" +
|
||||||
"binding_id\x18\a \x01(\tR\tbindingId\x12'\n" +
|
"binding_id\x18\a \x01(\tR\tbindingId\x12'\n" +
|
||||||
"\x0fbinding_version\x18\b \x01(\x03R\x0ebindingVersion\"@\n" +
|
"\x0fbinding_version\x18\b \x01(\x03R\x0ebindingVersion\x12\x1f\n" +
|
||||||
|
"\vinputs_json\x18\t \x01(\tR\n" +
|
||||||
|
"inputsJson\"@\n" +
|
||||||
"\x1aEnqueueBindingTaskResponse\x12\"\n" +
|
"\x1aEnqueueBindingTaskResponse\x12\"\n" +
|
||||||
"\x04task\x18\x01 \x01(\v2\x0e.state.v1.TaskR\x04task2\x903\n" +
|
"\x04task\x18\x01 \x01(\v2\x0e.state.v1.TaskR\x04task2\x903\n" +
|
||||||
"\x11StateStoreService\x12\\\n" +
|
"\x11StateStoreService\x12\\\n" +
|
||||||
|
|||||||
@@ -0,0 +1,84 @@
|
|||||||
|
package daemon
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
|
||||||
|
"github.com/samcharles93/archie-core/internal/domain/binding"
|
||||||
|
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||||
|
workflowtask "github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||||
|
)
|
||||||
|
|
||||||
|
// bindingTarget is what one binding dispatch starts: the repository the task
|
||||||
|
// works on (empty for none) and the workflow inputs it carries.
|
||||||
|
type bindingTarget struct {
|
||||||
|
owner, repo string
|
||||||
|
inputs map[string]any
|
||||||
|
// declared is whether the workflow declares inputs; such a task's body is
|
||||||
|
// provenance only, since its data travels as inputs.
|
||||||
|
declared bool
|
||||||
|
}
|
||||||
|
|
||||||
|
// activeWorkflows returns the definitions a dispatch cycle checks bindings
|
||||||
|
// against, the same set a task is later pinned from.
|
||||||
|
func (d *Daemon) activeWorkflows(ctx context.Context) (workflow.WorkflowDefinitionCollection, error) {
|
||||||
|
if d.WorkflowDefinitions == nil {
|
||||||
|
return workflow.ShippedDefinitions(), nil
|
||||||
|
}
|
||||||
|
collection, _, err := d.WorkflowDefinitions.WorkflowDefinitions(ctx)
|
||||||
|
return collection, err
|
||||||
|
}
|
||||||
|
|
||||||
|
// resolveBindingTarget checks a matched binding against the workflow it
|
||||||
|
// targets and resolves the task's repository and inputs. A non-empty reason
|
||||||
|
// means the binding does not dispatch and the reason is recorded on it; ok
|
||||||
|
// false with no reason means resolveBindingRepo already logged why.
|
||||||
|
func (d *Daemon) resolveBindingTarget(b binding.Binding, values map[string]any, workflows workflow.WorkflowDefinitionCollection) (target bindingTarget, reason string, ok bool) {
|
||||||
|
entry, found := workflows.DefinitionByID(b.Workflow)
|
||||||
|
if !found {
|
||||||
|
return bindingTarget{}, fmt.Sprintf("workflow %q is not defined", b.Workflow), false
|
||||||
|
}
|
||||||
|
iface, err := workflowtask.ParseWorkflowInterface(entry.YAML)
|
||||||
|
if err != nil {
|
||||||
|
return bindingTarget{}, fmt.Sprintf("workflow %q: %v", b.Workflow, err), false
|
||||||
|
}
|
||||||
|
target.inputs = b.ResolveInputs(values)
|
||||||
|
target.declared = len(iface.Inputs) > 0
|
||||||
|
if err := iface.CheckInputs(target.inputs); err != nil {
|
||||||
|
return bindingTarget{}, "inputs do not match the workflow: " + err.Error(), false
|
||||||
|
}
|
||||||
|
switch mode := iface.RepositoryMode(); {
|
||||||
|
case mode == workflowtask.RepositoryNone:
|
||||||
|
return target, "", true
|
||||||
|
case b.RepoParam != "":
|
||||||
|
owner, repo, _, err := b.RepositoryFrom(values)
|
||||||
|
if err != nil {
|
||||||
|
return bindingTarget{}, err.Error(), false
|
||||||
|
}
|
||||||
|
if !d.repoConfigured(owner, repo) {
|
||||||
|
return bindingTarget{}, fmt.Sprintf("repository %s/%s from the event is not a configured repository", owner, repo), false
|
||||||
|
}
|
||||||
|
target.owner, target.repo = owner, repo
|
||||||
|
return target, "", true
|
||||||
|
case mode == workflowtask.RepositoryOptional && b.Owner == "":
|
||||||
|
return target, "", true
|
||||||
|
}
|
||||||
|
owner, repo, resolved := d.resolveBindingRepo(b)
|
||||||
|
if !resolved {
|
||||||
|
return bindingTarget{}, "", false
|
||||||
|
}
|
||||||
|
target.owner, target.repo = owner, repo
|
||||||
|
return target, "", true
|
||||||
|
}
|
||||||
|
|
||||||
|
// repoConfigured reports whether owner/repo is one of the configured repos. A
|
||||||
|
// repository named by an event must be one, so a payload cannot point an
|
||||||
|
// agent at an arbitrary repository.
|
||||||
|
func (d *Daemon) repoConfigured(owner, repo string) bool {
|
||||||
|
for _, r := range d.Cfg.Get().Repos {
|
||||||
|
if r.Owner == owner && r.Name == repo {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
@@ -81,7 +81,7 @@ func TestConnectedNATSEndpointReachesManagedContainerCreate(t *testing.T) {
|
|||||||
Token: token,
|
Token: token,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
if _, err := pool.Acquire(t.Context(), nil, d.containerEnv(nil, "")); err != nil {
|
if _, err := pool.Acquire(t.Context(), "", nil, d.containerEnv(nil, "")); err != nil {
|
||||||
t.Fatalf("Pool.Acquire = %v", err)
|
t.Fatalf("Pool.Acquire = %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+134
-54
@@ -647,6 +647,11 @@ func (d *Daemon) dispatchBindings(ctx context.Context) {
|
|||||||
d.Log.Warn("list undispatched captures failed", "error", err)
|
d.Log.Warn("list undispatched captures failed", "error", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
workflows, err := d.activeWorkflows(ctx)
|
||||||
|
if err != nil {
|
||||||
|
d.Log.Warn("binding dispatch: workflow definitions unavailable", "error", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
for _, c := range captures {
|
for _, c := range captures {
|
||||||
armed, err := d.BindingDispatcher.ArmedBindingsForSource(ctx, c.Source)
|
armed, err := d.BindingDispatcher.ArmedBindingsForSource(ctx, c.Source)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -657,7 +662,7 @@ func (d *Daemon) dispatchBindings(ctx context.Context) {
|
|||||||
if !b.Matcher.Matches(c.Source, c.Dispatchable()) {
|
if !b.Matcher.Matches(c.Source, c.Dispatchable()) {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
d.dispatchOneBinding(ctx, b, c)
|
d.dispatchOneBinding(ctx, b, c, workflows)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -684,12 +689,13 @@ func (d *Daemon) bindingMapping(ctx context.Context, b binding.Binding) *mapping
|
|||||||
// when its mapping belongs to the capture's event type. The mapping resolves
|
// when its mapping belongs to the capture's event type. The mapping resolves
|
||||||
// the payload (a required-field failure records binding_dispatch_failure and
|
// the payload (a required-field failure records binding_dispatch_failure and
|
||||||
// stops), counts the match, and the binding's filter must admit the resolved
|
// stops), counts the match, and the binding's filter must admit the resolved
|
||||||
// parameters. The dispatch is then claimed in the binding_dispatches ledger
|
// parameters. The workflow it targets must accept the binding's inputs and
|
||||||
|
// repository (resolveBindingTarget). The dispatch is then claimed in the binding_dispatches ledger
|
||||||
// before the task is enqueued: a capture stays listed until every binding for
|
// before the task is enqueued: a capture stays listed until every binding for
|
||||||
// its event type has dispatched it, so the claim is what stops a binding that
|
// its event type has dispatched it, so the claim is what stops a binding that
|
||||||
// already fired from firing again on a later cycle. A failed enqueue after the
|
// already fired from firing again on a later cycle. A failed enqueue after the
|
||||||
// claim loses that dispatch, which is the at-most-once side of the trade.
|
// claim loses that dispatch, which is the at-most-once side of the trade.
|
||||||
func (d *Daemon) dispatchOneBinding(ctx context.Context, b binding.Binding, c storecontract.CapturedEvent) {
|
func (d *Daemon) dispatchOneBinding(ctx context.Context, b binding.Binding, c storecontract.CapturedEvent, workflows workflow.WorkflowDefinitionCollection) {
|
||||||
m := d.bindingMapping(ctx, b)
|
m := d.bindingMapping(ctx, b)
|
||||||
if m == nil || m.EventTypeID != c.EventType {
|
if m == nil || m.EventTypeID != c.EventType {
|
||||||
return
|
return
|
||||||
@@ -713,9 +719,12 @@ func (d *Daemon) dispatchOneBinding(ctx context.Context, b binding.Binding, c st
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
owner, repo, ok := d.resolveBindingRepo(b)
|
target, reason, ok := d.resolveBindingTarget(b, values, workflows)
|
||||||
|
if reason != "" {
|
||||||
|
d.recordDispatchFailure(ctx, b, c, reason, nil)
|
||||||
|
}
|
||||||
if !ok {
|
if !ok {
|
||||||
return // resolveBindingRepo already logged a warning
|
return
|
||||||
}
|
}
|
||||||
if err := d.BindingDispatcher.RecordDispatch(ctx, b.ID, int64(b.Version), c.ID, 0); err != nil {
|
if err := d.BindingDispatcher.RecordDispatch(ctx, b.ID, int64(b.Version), c.ID, 0); err != nil {
|
||||||
if !errors.Is(err, storecontract.ErrAlreadyDispatched) {
|
if !errors.Is(err, storecontract.ErrAlreadyDispatched) {
|
||||||
@@ -725,20 +734,29 @@ func (d *Daemon) dispatchOneBinding(ctx context.Context, b binding.Binding, c st
|
|||||||
}
|
}
|
||||||
title := fmt.Sprintf("binding %s/%d from %s", b.Name, b.Version, c.Source)
|
title := fmt.Sprintf("binding %s/%d from %s", b.Name, b.Version, c.Source)
|
||||||
body := renderBindingBody(values, c)
|
body := renderBindingBody(values, c)
|
||||||
task, err := d.BindingTaskCreator.EnqueueBindingTask(ctx, owner, repo, title, body, b.Workflow, "", b.ID, b.Version)
|
if target.declared {
|
||||||
|
body = fmt.Sprintf("Started by binding %q from capture %q; the workflow's inputs travel with the task.", b.Name, c.ID)
|
||||||
|
}
|
||||||
|
task, err := d.BindingTaskCreator.EnqueueBindingTask(ctx, target.owner, target.repo, title, body, b.Workflow, "", b.ID, b.Version, target.inputs)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
d.Log.Warn("binding dispatch: enqueue", "binding", b.ID, "capture", c.ID, "error", err)
|
d.Log.Warn("binding dispatch: enqueue", "binding", b.ID, "capture", c.ID, "error", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if c.Unsigned {
|
if c.Unsigned {
|
||||||
if _, err := d.Store.InsertEvent(ctx, events.Event{
|
d.markUnsignedStart(ctx, task.ID, b, c)
|
||||||
TaskID: task.ID,
|
}
|
||||||
Kind: events.KindUnsignedEvent,
|
}
|
||||||
Detail: fmt.Sprintf("started by an unsigned event from source %q", c.Source),
|
|
||||||
Data: map[string]any{"source": c.Source, "capture_id": c.ID, "binding_id": b.ID},
|
// markUnsignedStart records on a task's timeline that an unsigned event
|
||||||
}); err != nil {
|
// started it.
|
||||||
d.Log.Warn("binding dispatch: unsigned marker", "task", task.ID, "error", err)
|
func (d *Daemon) markUnsignedStart(ctx context.Context, taskID int64, b binding.Binding, c storecontract.CapturedEvent) {
|
||||||
}
|
if _, err := d.Store.InsertEvent(ctx, events.Event{
|
||||||
|
TaskID: taskID,
|
||||||
|
Kind: events.KindUnsignedEvent,
|
||||||
|
Detail: fmt.Sprintf("started by an unsigned event from source %q", c.Source),
|
||||||
|
Data: map[string]any{"source": c.Source, "capture_id": c.ID, "binding_id": b.ID},
|
||||||
|
}); err != nil {
|
||||||
|
d.Log.Warn("binding dispatch: unsigned marker", "task", taskID, "error", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1215,7 +1233,7 @@ func (d *Daemon) process(ctx context.Context, task *workflow.Task) {
|
|||||||
|
|
||||||
trees := d.treesFor(task)
|
trees := d.treesFor(task)
|
||||||
repo, ok := d.repoFor(task)
|
repo, ok := d.repoFor(task)
|
||||||
if !ok {
|
if !ok && task.HasRepository() {
|
||||||
d.parkRunningTask(ctx, task.ID, "repo no longer in config", taskstate.ParkTerminal)
|
d.parkRunningTask(ctx, task.ID, "repo no longer in config", taskstate.ParkTerminal)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -1224,31 +1242,26 @@ func (d *Daemon) process(ctx context.Context, task *workflow.Task) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
workDir := trees.Dir(task.Owner, task.Repo, task.IssueNumber)
|
|
||||||
d.Log.Info("processing task", "repo", repo.FullName(), "issue", task.IssueNumber, "attempt", task.Attempt)
|
d.Log.Info("processing task", "repo", repo.FullName(), "issue", task.IssueNumber, "attempt", task.Attempt)
|
||||||
|
workDir, ok := d.prepareWorkspace(ctx, task, trees, repo)
|
||||||
// Clone the worktree before Docker mount setup. The container
|
if !ok {
|
||||||
// binds /data/worktree to workDir on the host -- the directory
|
|
||||||
// must exist before Acquire is called.
|
|
||||||
var branch string
|
|
||||||
var err error
|
|
||||||
// Every task gets an independent full clone. The former
|
|
||||||
// PreparePersistent path shared objects with a per-repo bare cache;
|
|
||||||
// go-git has no --dissociate, so a shared cache would stay a live
|
|
||||||
// dependency of each worktree and expiring one would corrupt running
|
|
||||||
// tasks. repo.PersistentStorage still governs the container volume.
|
|
||||||
_, branch, err = trees.Prepare(ctx, task.Owner, task.Repo, repo.BaseBranch(), task.IssueNumber, task.Title, task.Body, task.Labels)
|
|
||||||
if err != nil {
|
|
||||||
d.Log.Error("worktree prepare failed", "err", err)
|
|
||||||
d.parkRunningTask(ctx, task.ID, "worktree prepare failed: "+err.Error(), taskstate.ParkTransient)
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
task.Branch = branch
|
if !task.HasRepository() {
|
||||||
if err := d.Store.Update(ctx, task); err != nil {
|
defer func() {
|
||||||
d.Log.Warn("task branch not persisted", "task", task.ID, "err", err)
|
if err := trees.RemoveScratch(task.ID); err != nil {
|
||||||
|
d.Log.Warn("scratch workspace cleanup failed", "task", task.ID, "err", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
}
|
}
|
||||||
|
|
||||||
ctr, revokeStateStoreGrant, ok := d.acquireTaskContainer(ctx, task, repo, workDir)
|
// The workflow is pinned before the container starts, because the
|
||||||
|
// profile it names decides the container's image.
|
||||||
|
profile, ok := d.pinTaskProfile(ctx, task)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctr, revokeStateStoreGrant, ok := d.acquireTaskContainer(ctx, task, repo, workDir, profile.Image)
|
||||||
if !ok {
|
if !ok {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -1263,7 +1276,7 @@ func (d *Daemon) process(ctx context.Context, task *workflow.Task) {
|
|||||||
// worktreerpc clients to that identity, and the daemon registered one
|
// worktreerpc clients to that identity, and the daemon registered one
|
||||||
// server pair per identity (plus the root pair), so a container-mode
|
// server pair per identity (plus the root pair), so a container-mode
|
||||||
// task is always served by its own forge client and worktree manager.
|
// task is always served by its own forge client and worktree manager.
|
||||||
d.runViaAgent(ctx, task, repo)
|
d.runViaAgent(ctx, task, repo, profile)
|
||||||
|
|
||||||
// Teardown storage after workflow completes. The Docker backend is a
|
// Teardown storage after workflow completes. The Docker backend is a
|
||||||
// no-op; future backends (temp volumes, NFS leases) use this hook.
|
// no-op; future backends (temp volumes, NFS leases) use this hook.
|
||||||
@@ -1280,8 +1293,40 @@ func (d *Daemon) process(ctx context.Context, task *workflow.Task) {
|
|||||||
d.cleanupTerminalTaskWorktree(ctx, task, trees)
|
d.cleanupTerminalTaskWorktree(ctx, task, trees)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// prepareWorkspace makes the directory the container binds as its
|
||||||
|
// workspace, which must exist before Acquire: a fresh clone of the task's
|
||||||
|
// repository, or an empty scratch directory for a task with none. It parks
|
||||||
|
// the task and reports false when it cannot.
|
||||||
|
func (d *Daemon) prepareWorkspace(ctx context.Context, task *workflow.Task, trees *worktree.Manager, repo config.Repo) (string, bool) {
|
||||||
|
if !task.HasRepository() {
|
||||||
|
dir, err := trees.PrepareScratch(task.ID)
|
||||||
|
if err != nil {
|
||||||
|
d.Log.Error("scratch workspace prepare failed", "err", err)
|
||||||
|
d.parkRunningTask(ctx, task.ID, "scratch workspace prepare failed: "+err.Error(), taskstate.ParkTransient)
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return dir, true
|
||||||
|
}
|
||||||
|
// Every task gets an independent full clone. The former
|
||||||
|
// PreparePersistent path shared objects with a per-repo bare cache;
|
||||||
|
// go-git has no --dissociate, so a shared cache would stay a live
|
||||||
|
// dependency of each worktree and expiring one would corrupt running
|
||||||
|
// tasks. repo.PersistentStorage still governs the container volume.
|
||||||
|
dir, branch, err := trees.Prepare(ctx, task.Owner, task.Repo, repo.BaseBranch(), task.IssueNumber, task.Title, task.Body, task.Labels)
|
||||||
|
if err != nil {
|
||||||
|
d.Log.Error("worktree prepare failed", "err", err)
|
||||||
|
d.parkRunningTask(ctx, task.ID, "worktree prepare failed: "+err.Error(), taskstate.ParkTransient)
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
task.Branch = branch
|
||||||
|
if err := d.Store.Update(ctx, task); err != nil {
|
||||||
|
d.Log.Warn("task branch not persisted", "task", task.ID, "err", err)
|
||||||
|
}
|
||||||
|
return dir, true
|
||||||
|
}
|
||||||
|
|
||||||
func (d *Daemon) cleanupTerminalTaskWorktree(ctx context.Context, task *workflow.Task, trees *worktree.Manager) {
|
func (d *Daemon) cleanupTerminalTaskWorktree(ctx context.Context, task *workflow.Task, trees *worktree.Manager) {
|
||||||
if d.Store == nil || trees == nil || task == nil {
|
if d.Store == nil || trees == nil || task == nil || !task.HasRepository() {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
cleanupCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
|
cleanupCtx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
|
||||||
@@ -1331,6 +1376,7 @@ func (d *Daemon) acquireTaskContainer(
|
|||||||
task *workflow.Task,
|
task *workflow.Task,
|
||||||
repo config.Repo,
|
repo config.Repo,
|
||||||
workDir string,
|
workDir string,
|
||||||
|
image string,
|
||||||
) (*container.Container, func(), bool) {
|
) (*container.Container, func(), bool) {
|
||||||
park := func(reason string, err error) {
|
park := func(reason string, err error) {
|
||||||
d.Log.Error(reason, "err", err)
|
d.Log.Error(reason, "err", err)
|
||||||
@@ -1343,6 +1389,7 @@ func (d *Daemon) acquireTaskContainer(
|
|||||||
Number: task.IssueNumber, Title: task.Title, Body: task.Body,
|
Number: task.IssueNumber, Title: task.Title, Body: task.Body,
|
||||||
Labels: strings.Split(task.Labels, ","),
|
Labels: strings.Split(task.Labels, ","),
|
||||||
Workflow: task.Workflow, Branch: task.Branch, Plan: task.Plan,
|
Workflow: task.Workflow, Branch: task.Branch, Plan: task.Plan,
|
||||||
|
Inputs: task.Inputs,
|
||||||
}); err != nil {
|
}); err != nil {
|
||||||
park("task.json write failed", err)
|
park("task.json write failed", err)
|
||||||
return nil, nil, false
|
return nil, nil, false
|
||||||
@@ -1383,7 +1430,7 @@ func (d *Daemon) acquireTaskContainer(
|
|||||||
return nil, nil, false
|
return nil, nil, false
|
||||||
}
|
}
|
||||||
|
|
||||||
ctr, err := d.ContainerPool.Acquire(ctx, mounts, d.containerEnv(task, stateStoreToken))
|
ctr, err := d.ContainerPool.Acquire(ctx, image, mounts, d.containerEnv(task, stateStoreToken))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
revokeStateStoreGrant()
|
revokeStateStoreGrant()
|
||||||
// Roll back the storage we just set up: the mounts were created
|
// Roll back the storage we just set up: the mounts were created
|
||||||
@@ -1497,28 +1544,15 @@ func (d *Daemon) recordPark(ctx context.Context, taskID int64, reason string) {
|
|||||||
// itself when nothing else could have: the request never reached (or was
|
// itself when nothing else could have: the request never reached (or was
|
||||||
// never answered by) an archie-agent, or archie-agent failed before its
|
// never answered by) an archie-agent, or archie-agent failed before its
|
||||||
// own workflow.Run got a chance to record an outcome.
|
// own workflow.Run got a chance to record an outcome.
|
||||||
func (d *Daemon) runViaAgent(ctx context.Context, task *workflow.Task, repo config.Repo) {
|
func (d *Daemon) runViaAgent(ctx context.Context, task *workflow.Task, repo config.Repo, profile config.AgentProfile) {
|
||||||
if d.WorktreeGrants == nil {
|
grant, revoke, ok := d.publicationGrant(ctx, task)
|
||||||
const reason = "worktree publication grants are unavailable"
|
if !ok {
|
||||||
d.Log.Error(reason, "task", task.ID)
|
|
||||||
d.parkRunningTask(ctx, task.ID, reason, taskstate.ParkTransient)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
grant, revoke, err := d.WorktreeGrants.Issue(task)
|
|
||||||
if err != nil {
|
|
||||||
d.Log.Error("worktree publication grant failed", "task", task.ID, "err", err)
|
|
||||||
d.parkRunningTask(ctx, task.ID, "worktree publication grant failed: "+err.Error(), taskstate.ParkTransient)
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
defer revoke()
|
defer revoke()
|
||||||
cfg := d.configFor(task)
|
cfg := d.configFor(task)
|
||||||
taskCfg := cfg.ForTask()
|
taskCfg := cfg.ForTask()
|
||||||
d.captureAttemptConfig(ctx, task, taskCfg)
|
d.captureAttemptConfig(ctx, task, taskCfg)
|
||||||
if err := d.pinWorkflowDefinition(ctx, task); err != nil {
|
|
||||||
d.Log.Error("pin workflow definition failed", "task", task.ID, "err", err)
|
|
||||||
d.parkRunningTask(ctx, task.ID, "pin workflow definition: "+err.Error(), taskstate.ParkTransient)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
req := taskrun.Request{
|
req := taskrun.Request{
|
||||||
Task: task,
|
Task: task,
|
||||||
Repo: repo,
|
Repo: repo,
|
||||||
@@ -1529,6 +1563,7 @@ func (d *Daemon) runViaAgent(ctx context.Context, task *workflow.Task, repo conf
|
|||||||
KindWorkflows: d.KindWorkflows,
|
KindWorkflows: d.KindWorkflows,
|
||||||
LabelWorkflows: d.LabelWorkflows,
|
LabelWorkflows: d.LabelWorkflows,
|
||||||
WorkflowDefinition: task.WorkflowDefinitionYAML,
|
WorkflowDefinition: task.WorkflowDefinitionYAML,
|
||||||
|
Tools: profile.Tools,
|
||||||
}
|
}
|
||||||
data, err := json.Marshal(req)
|
data, err := json.Marshal(req)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -1567,6 +1602,48 @@ func (d *Daemon) runViaAgent(ctx context.Context, task *workflow.Task, repo conf
|
|||||||
d.Log.Info("taskrun complete", "task", task.ID, "status", resp.Status)
|
d.Log.Info("taskrun complete", "task", task.ID, "status", resp.Status)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// pinTaskProfile pins the task's workflow definition and resolves the agent
|
||||||
|
// profile it names, parking the task when either fails. An unconfigured
|
||||||
|
// profile needs an operator: the fix is configuration, then a retry.
|
||||||
|
func (d *Daemon) pinTaskProfile(ctx context.Context, task *workflow.Task) (config.AgentProfile, bool) {
|
||||||
|
if err := d.pinWorkflowDefinition(ctx, task); err != nil {
|
||||||
|
d.Log.Error("pin workflow definition failed", "task", task.ID, "err", err)
|
||||||
|
d.parkRunningTask(ctx, task.ID, "pin workflow definition: "+err.Error(), taskstate.ParkTransient)
|
||||||
|
return config.AgentProfile{}, false
|
||||||
|
}
|
||||||
|
iface, err := workflowtask.ParseWorkflowInterface(task.WorkflowDefinitionYAML)
|
||||||
|
if err == nil {
|
||||||
|
var profile config.AgentProfile
|
||||||
|
if profile, err = d.configFor(task).Containers.Profile(iface.Profile); err == nil {
|
||||||
|
return profile, true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
d.Log.Error("agent profile unavailable", "task", task.ID, "err", err)
|
||||||
|
d.parkRunningTask(ctx, task.ID, "workflow "+task.Workflow+": "+err.Error(), taskstate.ParkNeedsHuman)
|
||||||
|
return config.AgentProfile{}, false
|
||||||
|
}
|
||||||
|
|
||||||
|
// publicationGrant issues the capability to publish the task's branch. A task
|
||||||
|
// with no repository has nothing to publish and gets none.
|
||||||
|
func (d *Daemon) publicationGrant(ctx context.Context, task *workflow.Task) (string, func(), bool) {
|
||||||
|
if !task.HasRepository() {
|
||||||
|
return "", func() {}, true
|
||||||
|
}
|
||||||
|
if d.WorktreeGrants == nil {
|
||||||
|
const reason = "worktree publication grants are unavailable"
|
||||||
|
d.Log.Error(reason, "task", task.ID)
|
||||||
|
d.parkRunningTask(ctx, task.ID, reason, taskstate.ParkTransient)
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
grant, revoke, err := d.WorktreeGrants.Issue(task)
|
||||||
|
if err != nil {
|
||||||
|
d.Log.Error("worktree publication grant failed", "task", task.ID, "err", err)
|
||||||
|
d.parkRunningTask(ctx, task.ID, "worktree publication grant failed: "+err.Error(), taskstate.ParkTransient)
|
||||||
|
return "", nil, false
|
||||||
|
}
|
||||||
|
return grant, revoke, true
|
||||||
|
}
|
||||||
|
|
||||||
func (d *Daemon) pinWorkflowDefinition(ctx context.Context, task *workflow.Task) error {
|
func (d *Daemon) pinWorkflowDefinition(ctx context.Context, task *workflow.Task) error {
|
||||||
if task.WorkflowDefinitionYAML != "" {
|
if task.WorkflowDefinitionYAML != "" {
|
||||||
if workflow.DigestDefinition(task.WorkflowDefinitionYAML) != task.WorkflowDefinitionDigest {
|
if workflow.DigestDefinition(task.WorkflowDefinitionYAML) != task.WorkflowDefinitionDigest {
|
||||||
@@ -1970,6 +2047,9 @@ func (d *Daemon) repoFor(t *workflow.Task) (config.Repo, bool) {
|
|||||||
// policy comes from the identity's own config, not the root daemon's.
|
// policy comes from the identity's own config, not the root daemon's.
|
||||||
// Unknown repos default to the safe, serialized behavior.
|
// Unknown repos default to the safe, serialized behavior.
|
||||||
func (d *Daemon) allowConcurrentForTask(task *workflow.Task) bool {
|
func (d *Daemon) allowConcurrentForTask(task *workflow.Task) bool {
|
||||||
|
if !task.HasRepository() {
|
||||||
|
return true
|
||||||
|
}
|
||||||
repo, ok := d.repoFor(task)
|
repo, ok := d.repoFor(task)
|
||||||
return ok && repo.AllowConcurrent
|
return ok && repo.AllowConcurrent
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -624,7 +624,7 @@ func TestRunViaAgentParksOnRequestFailure(t *testing.T) {
|
|||||||
// No responder registered on the taskrun subject, ever -- the retry
|
// No responder registered on the taskrun subject, ever -- the retry
|
||||||
// window must exhaust and runViaAgent must park rather than leave the
|
// window must exhaust and runViaAgent must park rather than leave the
|
||||||
// task stuck running.
|
// task stuck running.
|
||||||
d.runViaAgent(ctx, task, config.Repo{Owner: "acme", Name: "widget"})
|
runPinnedViaAgent(ctx, d, task, config.Repo{Owner: "acme", Name: "widget"})
|
||||||
|
|
||||||
got, err := s.TaskByIssue(ctx, "acme", "widget", 1)
|
got, err := s.TaskByIssue(ctx, "acme", "widget", 1)
|
||||||
if err != nil || got == nil {
|
if err != nil || got == nil {
|
||||||
@@ -684,7 +684,7 @@ func TestRunViaAgentRetriesUntilResponderAppears(t *testing.T) {
|
|||||||
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
||||||
}()
|
}()
|
||||||
|
|
||||||
d.runViaAgent(ctx, task, config.Repo{Owner: "acme", Name: "widget"})
|
runPinnedViaAgent(ctx, d, task, config.Repo{Owner: "acme", Name: "widget"})
|
||||||
|
|
||||||
got, err := s.TaskByIssue(ctx, "acme", "widget", 6)
|
got, err := s.TaskByIssue(ctx, "acme", "widget", 6)
|
||||||
if err != nil || got == nil {
|
if err != nil || got == nil {
|
||||||
@@ -716,7 +716,7 @@ func TestRunViaAgentDoesNotRetryOnContextCancellation(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
start := time.Now()
|
start := time.Now()
|
||||||
d.runViaAgent(ctx, task, config.Repo{Owner: "acme", Name: "widget"})
|
runPinnedViaAgent(ctx, d, task, config.Repo{Owner: "acme", Name: "widget"})
|
||||||
if elapsed := time.Since(start); elapsed > 200*time.Millisecond {
|
if elapsed := time.Since(start); elapsed > 200*time.Millisecond {
|
||||||
t.Fatalf("runViaAgent took %s with an already-cancelled context, want it to return immediately without retrying", elapsed)
|
t.Fatalf("runViaAgent took %s with an already-cancelled context, want it to return immediately without retrying", elapsed)
|
||||||
}
|
}
|
||||||
@@ -743,7 +743,7 @@ func TestRunViaAgentParksOnRunError(t *testing.T) {
|
|||||||
}
|
}
|
||||||
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
||||||
|
|
||||||
d.runViaAgent(ctx, task, config.Repo{Owner: "acme", Name: "widget"})
|
runPinnedViaAgent(ctx, d, task, config.Repo{Owner: "acme", Name: "widget"})
|
||||||
|
|
||||||
got, err := s.TaskByIssue(ctx, "acme", "widget", 2)
|
got, err := s.TaskByIssue(ctx, "acme", "widget", 2)
|
||||||
if err != nil || got == nil {
|
if err != nil || got == nil {
|
||||||
@@ -788,7 +788,7 @@ func TestRunViaAgentSendsExpectedRequest(t *testing.T) {
|
|||||||
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
||||||
|
|
||||||
repo := config.Repo{Owner: "acme", Name: "widget", Base: "main"}
|
repo := config.Repo{Owner: "acme", Name: "widget", Base: "main"}
|
||||||
d.runViaAgent(ctx, task, repo)
|
runPinnedViaAgent(ctx, d, task, repo)
|
||||||
|
|
||||||
select {
|
select {
|
||||||
case req := <-received:
|
case req := <-received:
|
||||||
@@ -857,7 +857,7 @@ func TestRunViaAgentSendsMCPServers(t *testing.T) {
|
|||||||
}
|
}
|
||||||
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
||||||
|
|
||||||
d.runViaAgent(ctx, task, config.Repo{Owner: "acme", Name: "widget", Base: "main"})
|
runPinnedViaAgent(ctx, d, task, config.Repo{Owner: "acme", Name: "widget", Base: "main"})
|
||||||
|
|
||||||
select {
|
select {
|
||||||
case req := <-received:
|
case req := <-received:
|
||||||
@@ -908,7 +908,7 @@ func TestRunViaAgentSendsRoutingBindings(t *testing.T) {
|
|||||||
}
|
}
|
||||||
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
||||||
|
|
||||||
d.runViaAgent(ctx, task, config.Repo{Owner: "acme", Name: "widget", Base: "main"})
|
runPinnedViaAgent(ctx, d, task, config.Repo{Owner: "acme", Name: "widget", Base: "main"})
|
||||||
|
|
||||||
select {
|
select {
|
||||||
case p := <-received:
|
case p := <-received:
|
||||||
@@ -957,7 +957,7 @@ func TestRunViaAgentObservesReportedAgentVersion(t *testing.T) {
|
|||||||
}
|
}
|
||||||
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
||||||
|
|
||||||
d.runViaAgent(ctx, task, config.Repo{Owner: "acme", Name: "widget"})
|
runPinnedViaAgent(ctx, d, task, config.Repo{Owner: "acme", Name: "widget"})
|
||||||
|
|
||||||
version, installType, ok := status.Snapshot()
|
version, installType, ok := status.Snapshot()
|
||||||
if !ok || version != "1.9.11" || installType != "container" {
|
if !ok || version != "1.9.11" || installType != "container" {
|
||||||
@@ -992,7 +992,7 @@ func TestRunViaAgentIgnoresEmptyAgentVersion(t *testing.T) {
|
|||||||
}
|
}
|
||||||
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
||||||
|
|
||||||
d.runViaAgent(ctx, task, config.Repo{Owner: "acme", Name: "widget"})
|
runPinnedViaAgent(ctx, d, task, config.Repo{Owner: "acme", Name: "widget"})
|
||||||
|
|
||||||
if _, _, ok := status.Snapshot(); ok {
|
if _, _, ok := status.Snapshot(); ok {
|
||||||
t.Fatal("Snapshot() reports observed after an empty AgentVersion response")
|
t.Fatal("Snapshot() reports observed after an empty AgentVersion response")
|
||||||
@@ -2031,7 +2031,7 @@ func TestDispatchCapturesTheAttemptsEffectiveConfig(t *testing.T) {
|
|||||||
}
|
}
|
||||||
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
||||||
|
|
||||||
d.runViaAgent(ctx, task, config.Repo{Owner: "acme", Name: "widget", Base: "main"})
|
runPinnedViaAgent(ctx, d, task, config.Repo{Owner: "acme", Name: "widget", Base: "main"})
|
||||||
|
|
||||||
timeline, err := s.TaskEvents(ctx, task.ID)
|
timeline, err := s.TaskEvents(ctx, task.ID)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -2173,7 +2173,7 @@ func TestCaptureAttemptConfigFailureDoesNotStopTheDispatch(t *testing.T) {
|
|||||||
}
|
}
|
||||||
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
||||||
|
|
||||||
d.runViaAgent(ctx, task, config.Repo{Owner: "acme", Name: "widget", Base: "main"})
|
runPinnedViaAgent(ctx, d, task, config.Repo{Owner: "acme", Name: "widget", Base: "main"})
|
||||||
|
|
||||||
select {
|
select {
|
||||||
case <-dispatched:
|
case <-dispatched:
|
||||||
@@ -2285,3 +2285,68 @@ func TestConfigForIdentityInheritsAnUnsetDiffCap(t *testing.T) {
|
|||||||
t.Fatalf("DiffCap() = %d, want 0: an identity may switch its own cap off explicitly", off.DiffCap())
|
t.Fatalf("DiffCap() = %d, want 0: an identity may switch its own cap off explicitly", off.DiffCap())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A workflow's profile decides the tools the agent gets: its allowlist is
|
||||||
|
// carried in the taskrun request, and a profile that is not configured parks
|
||||||
|
// the task for an operator instead of running it under the default.
|
||||||
|
func TestRunViaAgentCarriesTheWorkflowProfile(t *testing.T) {
|
||||||
|
d, s, busClient := daemonWithNATS(t)
|
||||||
|
cfg := d.Cfg.Get()
|
||||||
|
cfg.Containers.Profiles = map[string]config.AgentProfile{"net": {Image: "agent-net:1", Tools: []string{"whois"}}}
|
||||||
|
d.Cfg.Set(cfg)
|
||||||
|
d.WorkflowDefinitions = &workflowDefinitionsStub{collection: workflow.WorkflowDefinitionCollection{Definitions: []workflow.WorkflowDefinitionEntry{
|
||||||
|
{ID: "netflow", YAML: "id: netflow\nprofile: net\nsteps:\n - type: implement.prepare\n"},
|
||||||
|
{ID: "missing", YAML: "id: missing\nprofile: gone\nsteps:\n - type: implement.prepare\n"},
|
||||||
|
}}, version: 1}
|
||||||
|
ctx := context.Background()
|
||||||
|
|
||||||
|
claim := func(number int, wf string) *workflow.Task {
|
||||||
|
t.Helper()
|
||||||
|
if _, err := s.EnqueueIssue(ctx, "acme", "widget", number, "t", "b", "", ""); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
task, err := s.ClaimNext(ctx)
|
||||||
|
if err != nil || task == nil {
|
||||||
|
t.Fatalf("claim: (%v, %v)", task, err)
|
||||||
|
}
|
||||||
|
task.Workflow = wf
|
||||||
|
return task
|
||||||
|
}
|
||||||
|
|
||||||
|
task := claim(11, "netflow")
|
||||||
|
received := make(chan taskrun.Request, 1)
|
||||||
|
sub, err := mustCoreConn(t, busClient).Subscribe(agentnats.SubjectForTask(task.ID), func(msg *natsio.Msg) {
|
||||||
|
var req taskrun.Request
|
||||||
|
_ = json.Unmarshal(msg.Data, &req)
|
||||||
|
received <- req
|
||||||
|
data, _ := json.Marshal(taskrun.Response{Status: workflow.StatusPROpen})
|
||||||
|
_ = msg.Respond(data)
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("subscribe: %v", err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { _ = sub.Unsubscribe() })
|
||||||
|
|
||||||
|
profile, ok := d.pinTaskProfile(ctx, task)
|
||||||
|
if !ok || profile.Image != "agent-net:1" {
|
||||||
|
t.Fatalf("pinTaskProfile() = %+v, %v; want the net profile", profile, ok)
|
||||||
|
}
|
||||||
|
d.runViaAgent(ctx, task, config.Repo{Owner: "acme", Name: "widget", Base: "main"}, profile)
|
||||||
|
select {
|
||||||
|
case req := <-received:
|
||||||
|
if len(req.Tools) != 1 || req.Tools[0] != "whois" {
|
||||||
|
t.Fatalf("request Tools = %v, want the profile's allowlist", req.Tools)
|
||||||
|
}
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("archied did not publish a taskrun request")
|
||||||
|
}
|
||||||
|
|
||||||
|
parked := claim(12, "missing")
|
||||||
|
if _, ok := d.pinTaskProfile(ctx, parked); ok {
|
||||||
|
t.Fatal("a workflow naming an unconfigured profile was not refused")
|
||||||
|
}
|
||||||
|
got, err := s.TaskByID(ctx, parked.ID)
|
||||||
|
if err != nil || got.Status != workflow.StatusParked || !strings.Contains(got.ParkReason, `"gone" is not configured`) {
|
||||||
|
t.Fatalf("task = %+v, %v; want parked naming the profile", got, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
package daemon
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/samcharles93/archie-core/internal/config"
|
||||||
|
"github.com/samcharles93/archie-core/internal/domain/binding"
|
||||||
|
"github.com/samcharles93/archie-core/internal/domain/mapping"
|
||||||
|
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
investigateYAML = "id: investigate\nrepository: none\ninputs:\n src_ip: {type: string, required: true}\n severity: {type: string}\nsteps:\n - type: agent.run\n settings:\n mission: look\n"
|
||||||
|
reviewYAML = "id: review\nrepository: required\nsteps:\n - type: implement.prepare\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
func eventDefinitions() *workflowDefinitionsStub {
|
||||||
|
return &workflowDefinitionsStub{collection: workflow.WorkflowDefinitionCollection{Definitions: []workflow.WorkflowDefinitionEntry{
|
||||||
|
{ID: "investigate", YAML: investigateYAML},
|
||||||
|
{ID: "review", YAML: reviewYAML},
|
||||||
|
}}, version: 1}
|
||||||
|
}
|
||||||
|
|
||||||
|
// armBinding saves and approves b on source, returning its id.
|
||||||
|
func armBinding(t *testing.T, s *dispatchStores, b binding.Binding) string {
|
||||||
|
t.Helper()
|
||||||
|
id, err := s.EdaStore.InsertBinding(t.Context(), b)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("InsertBinding: %v", err)
|
||||||
|
}
|
||||||
|
if err := s.EdaStore.ApproveBinding(t.Context(), id); err != nil {
|
||||||
|
t.Fatalf("ApproveBinding: %v", err)
|
||||||
|
}
|
||||||
|
return id
|
||||||
|
}
|
||||||
|
|
||||||
|
func dispatchFailures(t *testing.T, s *dispatchStores) []string {
|
||||||
|
t.Helper()
|
||||||
|
events, err := s.EventsSince(t.Context(), "", 100)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("EventsSince: %v", err)
|
||||||
|
}
|
||||||
|
var out []string
|
||||||
|
for _, e := range events {
|
||||||
|
if e.Kind == "binding_dispatch_failure" {
|
||||||
|
out = append(out, e.Detail)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDispatchBindingTargetsTheWorkflowInterface(t *testing.T) {
|
||||||
|
fields := []mapping.Field{
|
||||||
|
{Name: "ip", Path: "ip", Type: mapping.TypeString},
|
||||||
|
{Name: "sev", Path: "sev", Type: mapping.TypeString},
|
||||||
|
{Name: "full", Path: "repository.full_name", Type: mapping.TypeString},
|
||||||
|
}
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
binding binding.Binding
|
||||||
|
body string
|
||||||
|
wantOwner string
|
||||||
|
wantRepo string
|
||||||
|
wantInputs map[string]any
|
||||||
|
wantFailure string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
name: "no-repository workflow gets structured inputs and no repo",
|
||||||
|
binding: binding.Binding{Workflow: "investigate", Inputs: map[string]binding.InputSource{
|
||||||
|
"src_ip": {Param: "ip"}, "severity": {Value: "high"},
|
||||||
|
}},
|
||||||
|
body: `{"ip":"10.0.0.9"}`,
|
||||||
|
wantInputs: map[string]any{"src_ip": "10.0.0.9", "severity": "high"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "a required input that did not resolve is refused",
|
||||||
|
binding: binding.Binding{Workflow: "investigate", Inputs: map[string]binding.InputSource{"src_ip": {Param: "ip"}}},
|
||||||
|
body: `{"sev":"low"}`,
|
||||||
|
wantFailure: `input "src_ip" is required`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "repository from the payload",
|
||||||
|
binding: binding.Binding{Workflow: "review", RepoParam: "full"},
|
||||||
|
body: `{"repository":{"full_name":"acme/api"}}`,
|
||||||
|
wantOwner: "acme", wantRepo: "api",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "repository from the payload must be configured",
|
||||||
|
binding: binding.Binding{Workflow: "review", RepoParam: "full"},
|
||||||
|
body: `{"repository":{"full_name":"evil/elsewhere"}}`,
|
||||||
|
wantFailure: "evil/elsewhere from the event is not a configured repository",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: "undefined workflow is refused",
|
||||||
|
binding: binding.Binding{Workflow: "gone"},
|
||||||
|
body: `{}`,
|
||||||
|
wantFailure: `workflow "gone" is not defined`,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
s := openDispatchTestStore(t)
|
||||||
|
mappingID := seedMapping(t, s, "fw", fields...)
|
||||||
|
b := tt.binding
|
||||||
|
b.Name, b.Matcher, b.MappingID = "b", binding.Matcher{Source: "fw"}, mappingID
|
||||||
|
armBinding(t, s, b)
|
||||||
|
seedCapture(t, s, "fw", true, tt.body)
|
||||||
|
|
||||||
|
d := newDispatchDaemonWithRepos(t, s, []config.Repo{{Owner: "acme", Name: "api"}, {Owner: "acme", Name: "web"}})
|
||||||
|
d.WorkflowDefinitions = eventDefinitions()
|
||||||
|
d.dispatchBindings(t.Context())
|
||||||
|
|
||||||
|
tasks, err := s.Tasks(t.Context(), 10)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("Tasks: %v", err)
|
||||||
|
}
|
||||||
|
failures := dispatchFailures(t, s)
|
||||||
|
if tt.wantFailure != "" {
|
||||||
|
if len(tasks) != 0 {
|
||||||
|
t.Fatalf("dispatched %d task(s), want none", len(tasks))
|
||||||
|
}
|
||||||
|
if len(failures) == 0 || !strings.Contains(failures[0], tt.wantFailure) {
|
||||||
|
t.Fatalf("failures = %q, want one containing %q", failures, tt.wantFailure)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if len(tasks) != 1 {
|
||||||
|
t.Fatalf("dispatched %d task(s), want 1 (failures %q)", len(tasks), failures)
|
||||||
|
}
|
||||||
|
got, err := s.TaskByID(t.Context(), tasks[0].ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("TaskByID: %v", err)
|
||||||
|
}
|
||||||
|
if got.Owner != tt.wantOwner || got.Repo != tt.wantRepo {
|
||||||
|
t.Errorf("task repo = %q/%q, want %q/%q", got.Owner, got.Repo, tt.wantOwner, tt.wantRepo)
|
||||||
|
}
|
||||||
|
if len(got.Inputs) != len(tt.wantInputs) {
|
||||||
|
t.Fatalf("inputs = %v, want %v", got.Inputs, tt.wantInputs)
|
||||||
|
}
|
||||||
|
for k, v := range tt.wantInputs {
|
||||||
|
if got.Inputs[k] != v {
|
||||||
|
t.Errorf("input %s = %v, want %v", k, got.Inputs[k], v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(tt.wantInputs) > 0 && strings.Contains(got.Body, "10.0.0.9") {
|
||||||
|
t.Errorf("body %q repeats the inputs as text", got.Body)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,46 @@
|
|||||||
|
package daemon
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"log/slog"
|
||||||
|
"os"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/samcharles93/archie-core/internal/config"
|
||||||
|
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||||
|
"github.com/samcharles93/archie-core/internal/worktree"
|
||||||
|
)
|
||||||
|
|
||||||
|
// A task with no repository gets an empty scratch workspace, no publication
|
||||||
|
// grant, and is not serialized behind other repository-less tasks.
|
||||||
|
func TestNoRepositoryTaskRunsInScratch(t *testing.T) {
|
||||||
|
d := &Daemon{Cfg: config.NewHolder(config.Config{}), Log: slog.New(slog.DiscardHandler)}
|
||||||
|
task := &workflow.Task{ID: 12}
|
||||||
|
trees := &worktree.Manager{WorkDir: t.TempDir()}
|
||||||
|
|
||||||
|
dir, ok := d.prepareWorkspace(t.Context(), task, trees, config.Repo{})
|
||||||
|
if !ok || dir != trees.ScratchDir(12) {
|
||||||
|
t.Fatalf("prepareWorkspace() = %q, %v; want the task's scratch dir", dir, ok)
|
||||||
|
}
|
||||||
|
if info, err := os.Stat(dir); err != nil || !info.IsDir() {
|
||||||
|
t.Fatalf("scratch dir not created: %v", err)
|
||||||
|
}
|
||||||
|
grant, revoke, ok := d.publicationGrant(t.Context(), task)
|
||||||
|
if !ok || grant != "" || revoke == nil {
|
||||||
|
t.Fatalf("publicationGrant() = %q, %v; want no grant and a no-op revoke", grant, ok)
|
||||||
|
}
|
||||||
|
revoke()
|
||||||
|
if !d.allowConcurrentForTask(task) {
|
||||||
|
t.Error("repository-less tasks are serialized on an empty repo key")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// runPinnedViaAgent pins the task's workflow and profile, then hands it to the
|
||||||
|
// agent, the order process uses.
|
||||||
|
func runPinnedViaAgent(ctx context.Context, d *Daemon, task *workflow.Task, repo config.Repo) {
|
||||||
|
profile, ok := d.pinTaskProfile(ctx, task)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
d.runViaAgent(ctx, task, repo, profile)
|
||||||
|
}
|
||||||
@@ -180,7 +180,7 @@ func TestCancelledTaskTransitionsToParked(t *testing.T) {
|
|||||||
d.TaskRunReadyTimeout = 100 * time.Millisecond
|
d.TaskRunReadyTimeout = 100 * time.Millisecond
|
||||||
d.TaskRunRetryBackoff = 10 * time.Millisecond
|
d.TaskRunRetryBackoff = 10 * time.Millisecond
|
||||||
|
|
||||||
d.runViaAgent(ctx, task, config.Repo{Owner: "acme", Name: "widget"})
|
runPinnedViaAgent(ctx, d, task, config.Repo{Owner: "acme", Name: "widget"})
|
||||||
|
|
||||||
got, err := s.TaskByID(context.Background(), task.ID)
|
got, err := s.TaskByID(context.Background(), task.ID)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -519,7 +519,7 @@ func TestAcquireTaskContainerFailureParksTaskOnCancelledContext(t *testing.T) {
|
|||||||
d.Storage = &errStorage{setupErr: tt.setupErr}
|
d.Storage = &errStorage{setupErr: tt.setupErr}
|
||||||
}
|
}
|
||||||
|
|
||||||
ctr, revoke, ok := d.acquireTaskContainer(ctx, task, config.Repo{Owner: "acme", Name: "widget"}, workDir)
|
ctr, revoke, ok := d.acquireTaskContainer(ctx, task, config.Repo{Owner: "acme", Name: "widget"}, workDir, "")
|
||||||
if ok || ctr != nil || revoke != nil {
|
if ok || ctr != nil || revoke != nil {
|
||||||
t.Fatalf("acquireTaskContainer ok = %v, ctr = %v, revoke==nil:%v; want false, nil, true", ok, ctr, revoke == nil)
|
t.Fatalf("acquireTaskContainer ok = %v, ctr = %v, revoke==nil:%v; want false, nil, true", ok, ctr, revoke == nil)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -56,12 +56,20 @@ type Binding struct {
|
|||||||
// behaviour that predates this field. Setting only one is invalid
|
// behaviour that predates this field. Setting only one is invalid
|
||||||
// (Validate rejects it): a pin is a complete owner/repo pair or not a
|
// (Validate rejects it): a pin is a complete owner/repo pair or not a
|
||||||
// pin at all, never a half-guess.
|
// pin at all, never a half-guess.
|
||||||
Owner string `json:"owner,omitempty"`
|
Owner string `json:"owner,omitempty"`
|
||||||
Repo string `json:"repo,omitempty"`
|
Repo string `json:"repo,omitempty"`
|
||||||
Version int `json:"version"`
|
// RepoParam names the mapped parameter holding "owner/name" when the
|
||||||
Status Status `json:"status"`
|
// repository comes from the event instead of a fixed Owner/Repo pin. The
|
||||||
CreatedAt time.Time `json:"created_at"`
|
// named repository must still be a configured one.
|
||||||
UpdatedAt time.Time `json:"updated_at"`
|
RepoParam string `json:"repo_param,omitempty"`
|
||||||
|
// Inputs assigns the workflow's declared inputs, each from a mapped
|
||||||
|
// parameter or a constant; CheckWorkflow checks them when the binding is
|
||||||
|
// saved.
|
||||||
|
Inputs map[string]InputSource `json:"inputs,omitempty"`
|
||||||
|
Version int `json:"version"`
|
||||||
|
Status Status `json:"status"`
|
||||||
|
CreatedAt time.Time `json:"created_at"`
|
||||||
|
UpdatedAt time.Time `json:"updated_at"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// Validate checks a binding is well-formed before it is persisted: a
|
// Validate checks a binding is well-formed before it is persisted: a
|
||||||
@@ -84,6 +92,14 @@ func (b Binding) Validate() error {
|
|||||||
if (b.Owner == "") != (b.Repo == "") {
|
if (b.Owner == "") != (b.Repo == "") {
|
||||||
return fmt.Errorf("binding: owner and repo must both be set or both be empty")
|
return fmt.Errorf("binding: owner and repo must both be set or both be empty")
|
||||||
}
|
}
|
||||||
|
if b.Owner != "" && b.RepoParam != "" {
|
||||||
|
return fmt.Errorf("binding: a fixed owner/repo and a repository parameter cannot both be set")
|
||||||
|
}
|
||||||
|
for _, name := range sortedKeys(b.Inputs) {
|
||||||
|
if err := b.Inputs[name].validate(name); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,124 @@
|
|||||||
|
package binding
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"github.com/samcharles93/archie-core/internal/domain/mapping"
|
||||||
|
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||||
|
)
|
||||||
|
|
||||||
|
// InputSource assigns one workflow input: either the value of a mapped
|
||||||
|
// parameter (Param) or a constant (Value), never both.
|
||||||
|
type InputSource struct {
|
||||||
|
Param string `json:"param,omitempty"`
|
||||||
|
Value any `json:"value,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s InputSource) validate(name string) error {
|
||||||
|
if (s.Param == "") == (s.Value == nil) {
|
||||||
|
return fmt.Errorf("binding: input %q must take either a parameter or a constant", name)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// CheckWorkflow checks the binding against the workflow it targets and the
|
||||||
|
// mapping it reads, as the binding is saved: every assigned input is declared,
|
||||||
|
// every required input is assigned, a parameter exists and its type feeds the
|
||||||
|
// input's, a constant has the input's type, and the repository agrees with the
|
||||||
|
// workflow's repository mode.
|
||||||
|
func (b Binding) CheckWorkflow(w task.WorkflowInterface, fields []mapping.Field) error {
|
||||||
|
byName := make(map[string]mapping.Field, len(fields))
|
||||||
|
for _, f := range fields {
|
||||||
|
byName[f.Name] = f
|
||||||
|
}
|
||||||
|
for _, name := range sortedKeys(b.Inputs) {
|
||||||
|
src := b.Inputs[name]
|
||||||
|
spec, ok := w.Inputs[name]
|
||||||
|
if !ok {
|
||||||
|
return fmt.Errorf("binding: input %q is not declared by workflow %q", name, b.Workflow)
|
||||||
|
}
|
||||||
|
if src.Param != "" {
|
||||||
|
f, ok := byName[src.Param]
|
||||||
|
if !ok {
|
||||||
|
return fmt.Errorf("binding: input %q reads parameter %q, which the mapping does not have", name, src.Param)
|
||||||
|
}
|
||||||
|
if !task.TypeAccepts(spec.Type, string(f.Type)) {
|
||||||
|
return fmt.Errorf("binding: input %q is %s, but parameter %q is %s", name, spec.Type, src.Param, f.Type)
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if got := task.ValueType(src.Value); !task.TypeAccepts(spec.Type, got) {
|
||||||
|
return fmt.Errorf("binding: input %q is %s, but its constant is %s", name, spec.Type, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, name := range sortedKeys(w.Inputs) {
|
||||||
|
if _, ok := b.Inputs[name]; w.Inputs[name].Required && !ok {
|
||||||
|
return fmt.Errorf("binding: workflow %q requires input %q", b.Workflow, name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return b.checkRepository(w.RepositoryMode(), byName)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (b Binding) checkRepository(mode task.RepositoryMode, fields map[string]mapping.Field) error {
|
||||||
|
if mode == task.RepositoryNone && (b.Owner != "" || b.RepoParam != "") {
|
||||||
|
return fmt.Errorf("binding: workflow %q runs without a repository, so the binding cannot name one", b.Workflow)
|
||||||
|
}
|
||||||
|
if b.RepoParam == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
f, ok := fields[b.RepoParam]
|
||||||
|
if !ok {
|
||||||
|
return fmt.Errorf("binding: repository parameter %q is not in the mapping", b.RepoParam)
|
||||||
|
}
|
||||||
|
if f.Type != mapping.TypeString {
|
||||||
|
return fmt.Errorf("binding: repository parameter %q is %s, want string", b.RepoParam, f.Type)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ResolveInputs builds the workflow input values from the mapping's resolved
|
||||||
|
// parameters and the binding's constants. A parameter that did not resolve is
|
||||||
|
// left out, so the workflow's required-input check reports it.
|
||||||
|
func (b Binding) ResolveInputs(values map[string]any) map[string]any {
|
||||||
|
if len(b.Inputs) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
out := make(map[string]any, len(b.Inputs))
|
||||||
|
for name, src := range b.Inputs {
|
||||||
|
if src.Param == "" {
|
||||||
|
out[name] = src.Value
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if v, ok := values[src.Param]; ok {
|
||||||
|
out[name] = v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// RepositoryFrom returns the owner and repository a dispatch targets when the
|
||||||
|
// binding reads it from a parameter holding "owner/name". ok is false when the
|
||||||
|
// binding has no repository parameter; err reports a value that is not an
|
||||||
|
// owner/name pair.
|
||||||
|
func (b Binding) RepositoryFrom(values map[string]any) (owner, repo string, ok bool, err error) {
|
||||||
|
if b.RepoParam == "" {
|
||||||
|
return "", "", false, nil
|
||||||
|
}
|
||||||
|
full, _ := values[b.RepoParam].(string)
|
||||||
|
owner, repo, found := strings.Cut(full, "/")
|
||||||
|
if !found || owner == "" || repo == "" || strings.Contains(repo, "/") {
|
||||||
|
return "", "", true, fmt.Errorf("binding: repository parameter %q is %q, want owner/name", b.RepoParam, full)
|
||||||
|
}
|
||||||
|
return owner, repo, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func sortedKeys[V any](m map[string]V) []string {
|
||||||
|
keys := make([]string, 0, len(m))
|
||||||
|
for k := range m {
|
||||||
|
keys = append(keys, k)
|
||||||
|
}
|
||||||
|
sort.Strings(keys)
|
||||||
|
return keys
|
||||||
|
}
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
package binding
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/samcharles93/archie-core/internal/domain/mapping"
|
||||||
|
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestCheckWorkflow(t *testing.T) {
|
||||||
|
fields := []mapping.Field{
|
||||||
|
{Name: "ip", Type: mapping.TypeString},
|
||||||
|
{Name: "count", Type: mapping.TypeNumber},
|
||||||
|
{Name: "repo", Type: mapping.TypeString},
|
||||||
|
{Name: "blob", Type: mapping.TypeAny},
|
||||||
|
}
|
||||||
|
investigate := task.WorkflowInterface{
|
||||||
|
Repository: task.RepositoryNone,
|
||||||
|
Inputs: map[string]task.InputSpec{
|
||||||
|
"src_ip": {Type: "string", Required: true},
|
||||||
|
"severity": {Type: "number"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
review := task.WorkflowInterface{Repository: task.RepositoryRequired}
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
b Binding
|
||||||
|
w task.WorkflowInterface
|
||||||
|
wantErr string
|
||||||
|
}{
|
||||||
|
{name: "param and constant", w: investigate, b: Binding{Inputs: map[string]InputSource{
|
||||||
|
"src_ip": {Param: "ip"}, "severity": {Value: 3.0},
|
||||||
|
}}},
|
||||||
|
{name: "any parameter feeds a typed input", w: investigate, b: Binding{Inputs: map[string]InputSource{"src_ip": {Param: "blob"}}}},
|
||||||
|
{name: "required input unassigned", w: investigate, b: Binding{}, wantErr: `requires input "src_ip"`},
|
||||||
|
{name: "undeclared input", w: investigate, b: Binding{Inputs: map[string]InputSource{
|
||||||
|
"src_ip": {Param: "ip"}, "port": {Value: 22.0},
|
||||||
|
}}, wantErr: `"port" is not declared`},
|
||||||
|
{name: "unknown parameter", w: investigate, b: Binding{Inputs: map[string]InputSource{"src_ip": {Param: "addr"}}}, wantErr: `parameter "addr", which the mapping does not have`},
|
||||||
|
{name: "parameter of the wrong type", w: investigate, b: Binding{Inputs: map[string]InputSource{"src_ip": {Param: "count"}}}, wantErr: `"src_ip" is string, but parameter "count" is number`},
|
||||||
|
{name: "constant of the wrong type", w: investigate, b: Binding{Inputs: map[string]InputSource{"src_ip": {Value: 4.0}}}, wantErr: "its constant is number"},
|
||||||
|
{name: "no-repository workflow with a pin", w: investigate, b: Binding{Owner: "acme", Repo: "api", Inputs: map[string]InputSource{"src_ip": {Param: "ip"}}}, wantErr: "runs without a repository"},
|
||||||
|
{name: "repository from a string parameter", w: review, b: Binding{RepoParam: "repo"}},
|
||||||
|
{name: "repository parameter missing", w: review, b: Binding{RepoParam: "full_name"}, wantErr: `"full_name" is not in the mapping`},
|
||||||
|
{name: "repository parameter not a string", w: review, b: Binding{RepoParam: "count"}, wantErr: "is number, want string"},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
err := tt.b.CheckWorkflow(tt.w, fields)
|
||||||
|
if tt.wantErr == "" {
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("CheckWorkflow() = %v", err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err == nil || !strings.Contains(err.Error(), tt.wantErr) {
|
||||||
|
t.Fatalf("CheckWorkflow() = %v, want containing %q", err, tt.wantErr)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestValidateInputsAndRepository(t *testing.T) {
|
||||||
|
base := Binding{Name: "n", Matcher: Matcher{Source: "s"}, MappingID: "m", Workflow: "w"}
|
||||||
|
both := base
|
||||||
|
both.Inputs = map[string]InputSource{"a": {Param: "p", Value: "x"}}
|
||||||
|
neither := base
|
||||||
|
neither.Inputs = map[string]InputSource{"a": {}}
|
||||||
|
pinned := base
|
||||||
|
pinned.Owner, pinned.Repo, pinned.RepoParam = "acme", "api", "repo"
|
||||||
|
for name, b := range map[string]Binding{"both": both, "neither": neither, "pin and parameter": pinned} {
|
||||||
|
if err := b.Validate(); err == nil {
|
||||||
|
t.Errorf("%s: Validate() = nil, want an error", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveInputsAndRepository(t *testing.T) {
|
||||||
|
b := Binding{RepoParam: "full", Inputs: map[string]InputSource{
|
||||||
|
"src_ip": {Param: "ip"}, "severity": {Value: "high"}, "missing": {Param: "absent"},
|
||||||
|
}}
|
||||||
|
values := map[string]any{"ip": "10.0.0.1", "full": "acme/api", "n": json.Number("1")}
|
||||||
|
got := b.ResolveInputs(values)
|
||||||
|
if got["src_ip"] != "10.0.0.1" || got["severity"] != "high" {
|
||||||
|
t.Errorf("ResolveInputs() = %v", got)
|
||||||
|
}
|
||||||
|
if _, ok := got["missing"]; ok {
|
||||||
|
t.Error("an unresolved parameter was passed as an input")
|
||||||
|
}
|
||||||
|
owner, repo, ok, err := b.RepositoryFrom(values)
|
||||||
|
if err != nil || !ok || owner != "acme" || repo != "api" {
|
||||||
|
t.Errorf("RepositoryFrom() = %q, %q, %v, %v", owner, repo, ok, err)
|
||||||
|
}
|
||||||
|
for _, bad := range []string{"acme", "acme/", "/api", "a/b/c"} {
|
||||||
|
if _, _, _, err := b.RepositoryFrom(map[string]any{"full": bad}); err == nil {
|
||||||
|
t.Errorf("RepositoryFrom(%q) accepted a value that is not owner/name", bad)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -251,7 +251,7 @@ type BindingDispatcher interface {
|
|||||||
// keeps the lifecycle surface narrow (8 methods, the interfacebloat limit)
|
// keeps the lifecycle surface narrow (8 methods, the interfacebloat limit)
|
||||||
// and keeps the binding-specific shape on the binding interfaces.
|
// and keeps the binding-specific shape on the binding interfaces.
|
||||||
type BindingTaskCreator interface {
|
type BindingTaskCreator interface {
|
||||||
EnqueueBindingTask(ctx context.Context, owner, repo, title, body, wf, identity, bindingID string, bindingVersion int) (*task.Task, error)
|
EnqueueBindingTask(ctx context.Context, owner, repo, title, body, wf, identity, bindingID string, bindingVersion int, inputs map[string]any) (*task.Task, error)
|
||||||
}
|
}
|
||||||
|
|
||||||
// PlaybookDispatcher is the idempotency-ledger surface for side-effecting
|
// PlaybookDispatcher is the idempotency-ledger surface for side-effecting
|
||||||
|
|||||||
@@ -132,7 +132,7 @@ func (a AgentStage) buildRequest(tc *TaskContext, modelRef string) agentexec.Req
|
|||||||
Workflow: tc.Task.Workflow,
|
Workflow: tc.Task.Workflow,
|
||||||
Model: modelRef,
|
Model: modelRef,
|
||||||
ContextWindow: modelContextBudget(tc.Cfg, modelRef),
|
ContextWindow: modelContextBudget(tc.Cfg, modelRef),
|
||||||
Mission: missionWithSkill(tc, a.Mission(tc)),
|
Mission: missionWithInputs(tc.Task, missionWithSkill(tc, a.Mission(tc))),
|
||||||
ExtraRules: a.buildExtraRules(tc),
|
ExtraRules: a.buildExtraRules(tc),
|
||||||
ReadOnly: a.ReadOnly,
|
ReadOnly: a.ReadOnly,
|
||||||
Budget: budget,
|
Budget: budget,
|
||||||
|
|||||||
@@ -0,0 +1,83 @@
|
|||||||
|
package workflow
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gopkg.in/yaml.v3"
|
||||||
|
|
||||||
|
"github.com/samcharles93/archie-core/internal/agentexec"
|
||||||
|
)
|
||||||
|
|
||||||
|
// AgentRunStepName is the step type that runs one agent mission and finishes
|
||||||
|
// the workflow with its summary. It needs no repository: the agent works in
|
||||||
|
// whatever workspace the task has, a worktree or a scratch directory.
|
||||||
|
const AgentRunStepName = "agent.run"
|
||||||
|
|
||||||
|
// agentRunSettings are the agent.run step's settings.
|
||||||
|
type agentRunSettings struct {
|
||||||
|
// Mission is what the agent is asked to do. The workflow's inputs are
|
||||||
|
// appended to it as structured data.
|
||||||
|
Mission string `yaml:"mission"`
|
||||||
|
// Role selects the model (cfg.Models[Role]); empty means builder.
|
||||||
|
Role string `yaml:"role"`
|
||||||
|
// ReadOnly restricts the agent to read-only tools.
|
||||||
|
ReadOnly bool `yaml:"read_only"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// AgentRunStepType contributes the agent.run step type.
|
||||||
|
func AgentRunStepType() StepType {
|
||||||
|
return StepType{Name: AgentRunStepName, Factory: newAgentRunStage}
|
||||||
|
}
|
||||||
|
|
||||||
|
func newAgentRunStage(settings yaml.Node) (Stage, error) {
|
||||||
|
var s agentRunSettings
|
||||||
|
if settings.Kind == 0 {
|
||||||
|
return Stage{}, fmt.Errorf("%s: settings.mission is required", AgentRunStepName)
|
||||||
|
}
|
||||||
|
if err := settings.Decode(&s); err != nil {
|
||||||
|
return Stage{}, fmt.Errorf("%s: %w", AgentRunStepName, err)
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(s.Mission) == "" {
|
||||||
|
return Stage{}, fmt.Errorf("%s: settings.mission is required", AgentRunStepName)
|
||||||
|
}
|
||||||
|
role := s.Role
|
||||||
|
if role == "" {
|
||||||
|
role = "builder"
|
||||||
|
}
|
||||||
|
agent := AgentStage{
|
||||||
|
Name: AgentRunStepName,
|
||||||
|
Role: role,
|
||||||
|
ReadOnly: s.ReadOnly,
|
||||||
|
Mission: func(*TaskContext) string { return s.Mission },
|
||||||
|
OnResult: func(tc *TaskContext, res agentexec.Result) error {
|
||||||
|
tc.Outcome = Outcome{Status: StatusCompleted, Detail: res.Summary}
|
||||||
|
return nil
|
||||||
|
},
|
||||||
|
}.Stage()
|
||||||
|
return Stage{Name: AgentRunStepName, Run: func(ctx context.Context, tc *TaskContext) error {
|
||||||
|
if tc.Dir == "" {
|
||||||
|
dir, branch, err := tc.Trees.Prepare(ctx, tc.Task.Owner, tc.Task.Repo, tc.Repo.BaseBranch(), tc.Task.IssueNumber, tc.Task.Title, tc.Task.Body, tc.Task.Labels)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("%s: workspace: %w", AgentRunStepName, err)
|
||||||
|
}
|
||||||
|
tc.Dir, tc.Branch = dir, branch
|
||||||
|
}
|
||||||
|
return agent.Run(ctx, tc)
|
||||||
|
}}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// missionWithInputs appends the task's workflow inputs to a mission as a JSON
|
||||||
|
// block, so the agent reads them as structured data rather than prose.
|
||||||
|
func missionWithInputs(t *Task, mission string) string {
|
||||||
|
if t == nil || len(t.Inputs) == 0 {
|
||||||
|
return mission
|
||||||
|
}
|
||||||
|
data, err := json.MarshalIndent(t.Inputs, "", " ")
|
||||||
|
if err != nil {
|
||||||
|
return mission
|
||||||
|
}
|
||||||
|
return mission + "\n\nWorkflow inputs (JSON):\n```json\n" + string(data) + "\n```"
|
||||||
|
}
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
package workflow
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"log/slog"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"github.com/samcharles93/archie-core/internal/agentexec"
|
||||||
|
"github.com/samcharles93/archie-core/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
func eventRegistry() StepRegistry {
|
||||||
|
registry := BuiltinStepRegistry()
|
||||||
|
registry[AgentRunStepName] = newAgentRunStage
|
||||||
|
return registry
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestParseDefinitionRepositoryMode(t *testing.T) {
|
||||||
|
for _, test := range []struct{ name, yaml, want string }{
|
||||||
|
{"none with a repo step", "id: w\nrepository: none\nsteps:\n - type: implement.prepare\n", `"implement.prepare" needs a repository`},
|
||||||
|
{"optional with a repo step", "id: w\nrepository: optional\nsteps:\n - type: implement.prepare\n", "needs a repository"},
|
||||||
|
{"agent.run without a mission", "id: w\nrepository: none\nsteps:\n - type: agent.run\n", "settings.mission is required"},
|
||||||
|
{"unknown input type", "id: w\ninputs:\n ip: {type: ipv4}\nsteps:\n - type: implement.prepare\n", `type "ipv4"`},
|
||||||
|
} {
|
||||||
|
t.Run(test.name, func(t *testing.T) {
|
||||||
|
_, err := ParseDefinition(test.yaml, eventRegistry())
|
||||||
|
if err == nil || !strings.Contains(err.Error(), test.want) {
|
||||||
|
t.Fatalf("ParseDefinition() error = %v, want containing %q", err, test.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
definition, err := ParseDefinition("id: w\nrepository: none\nprofile: net\ninputs:\n src_ip: {type: string, required: true}\nsteps:\n - type: agent.run\n settings:\n mission: investigate\n", eventRegistry())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("ParseDefinition(repo-free) = %v", err)
|
||||||
|
}
|
||||||
|
if definition.Profile != "net" || !definition.Inputs["src_ip"].Required {
|
||||||
|
t.Fatalf("interface = %+v, want profile net and a required src_ip", definition.WorkflowInterface)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// agent.run works in whatever workspace the task has, hands the agent its
|
||||||
|
// inputs as a JSON block, and completes the task with the agent's summary.
|
||||||
|
func TestAgentRunCompletesWithInputs(t *testing.T) {
|
||||||
|
wf, err := ParseAndCompile("id: w\nrepository: none\nsteps:\n - type: agent.run\n settings:\n mission: investigate the address\n read_only: true\n", eventRegistry())
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
runner := &fakeAgentRunner{result: agentexec.Result{Status: agentexec.StatusPassed, Summary: "benign scanner"}}
|
||||||
|
store := &recordingStore{}
|
||||||
|
tc := &TaskContext{
|
||||||
|
Task: &Task{ID: 3, Attempt: 1, Inputs: map[string]any{"src_ip": "10.0.0.9"}},
|
||||||
|
Agent: runner, Store: store, Trees: &fakeTrees{dir: "/scratch/3"},
|
||||||
|
Cfg: config.Config{Models: map[string]string{"builder": "p/m"}},
|
||||||
|
Log: slog.New(slog.DiscardHandler),
|
||||||
|
}
|
||||||
|
Run(context.Background(), wf, tc)
|
||||||
|
|
||||||
|
if runner.workspace != "/scratch/3" {
|
||||||
|
t.Errorf("workspace = %q, want the task's scratch workspace", runner.workspace)
|
||||||
|
}
|
||||||
|
if !runner.request.ReadOnly {
|
||||||
|
t.Error("read_only setting did not reach the agent request")
|
||||||
|
}
|
||||||
|
if !strings.Contains(runner.request.Mission, "investigate the address") || !strings.Contains(runner.request.Mission, `"src_ip": "10.0.0.9"`) {
|
||||||
|
t.Errorf("mission = %q, want the mission and the inputs as JSON", runner.request.Mission)
|
||||||
|
}
|
||||||
|
if tc.Outcome.Status != StatusCompleted || tc.Outcome.Detail != "benign scanner" {
|
||||||
|
t.Errorf("outcome = %+v, want completed with the summary", tc.Outcome)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -18,8 +18,16 @@ import (
|
|||||||
type YAMLDefinition struct {
|
type YAMLDefinition struct {
|
||||||
ID string `yaml:"id" json:"id"`
|
ID string `yaml:"id" json:"id"`
|
||||||
Steps []StepRecord `yaml:"steps" json:"steps"`
|
Steps []StepRecord `yaml:"steps" json:"steps"`
|
||||||
|
// WorkflowInterface declares the workflow's inputs, repository mode and
|
||||||
|
// agent profile.
|
||||||
|
task.WorkflowInterface `yaml:",inline" json:",inline"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// repoFreeStepTypes are the step types that run without a repository. A
|
||||||
|
// workflow whose repository is not required may use only these, since it may
|
||||||
|
// run with no worktree at all.
|
||||||
|
var repoFreeStepTypes = map[string]bool{AgentRunStepName: true}
|
||||||
|
|
||||||
// StepRecord selects one registered step type and supplies its typed settings.
|
// StepRecord selects one registered step type and supplies its typed settings.
|
||||||
type StepRecord struct {
|
type StepRecord struct {
|
||||||
Type string `yaml:"type" json:"type"`
|
Type string `yaml:"type" json:"type"`
|
||||||
@@ -45,11 +53,18 @@ func ParseDefinition(src string, registry StepRegistry) (YAMLDefinition, error)
|
|||||||
if len(definition.Steps) == 0 {
|
if len(definition.Steps) == 0 {
|
||||||
return YAMLDefinition{}, fmt.Errorf("workflow %q has no steps", definition.ID)
|
return YAMLDefinition{}, fmt.Errorf("workflow %q has no steps", definition.ID)
|
||||||
}
|
}
|
||||||
|
if err := definition.Validate(); err != nil {
|
||||||
|
return YAMLDefinition{}, fmt.Errorf("workflow %q: %w", definition.ID, err)
|
||||||
|
}
|
||||||
|
mode := definition.RepositoryMode()
|
||||||
for i, step := range definition.Steps {
|
for i, step := range definition.Steps {
|
||||||
factory, ok := registry[step.Type]
|
factory, ok := registry[step.Type]
|
||||||
if !ok {
|
if !ok {
|
||||||
return YAMLDefinition{}, fmt.Errorf("workflow %q step %d: unknown type %q", definition.ID, i+1, step.Type)
|
return YAMLDefinition{}, fmt.Errorf("workflow %q step %d: unknown type %q", definition.ID, i+1, step.Type)
|
||||||
}
|
}
|
||||||
|
if mode != task.RepositoryRequired && !repoFreeStepTypes[step.Type] {
|
||||||
|
return YAMLDefinition{}, fmt.Errorf("workflow %q step %d: %q needs a repository, but the workflow's repository is %s", definition.ID, i+1, step.Type, mode)
|
||||||
|
}
|
||||||
if _, err := factory(step.Settings); err != nil {
|
if _, err := factory(step.Settings); err != nil {
|
||||||
return YAMLDefinition{}, fmt.Errorf("workflow %q step %q settings: %w", definition.ID, step.Type, err)
|
return YAMLDefinition{}, fmt.Errorf("workflow %q step %q settings: %w", definition.ID, step.Type, err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -476,7 +476,7 @@ type fakeTrees struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (f *fakeTrees) Prepare(context.Context, string, string, string, int, string, string, string) (string, string, error) {
|
func (f *fakeTrees) Prepare(context.Context, string, string, string, int, string, string, string) (string, string, error) {
|
||||||
return "", "", nil
|
return f.dir, "", nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (f *fakeTrees) CommitAll(context.Context, string, string) (bool, error) {
|
func (f *fakeTrees) CommitAll(context.Context, string, string) (bool, error) {
|
||||||
|
|||||||
@@ -0,0 +1,161 @@
|
|||||||
|
package task
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"regexp"
|
||||||
|
"slices"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"gopkg.in/yaml.v3"
|
||||||
|
)
|
||||||
|
|
||||||
|
// RepositoryMode says whether a workflow works on a repository.
|
||||||
|
type RepositoryMode string
|
||||||
|
|
||||||
|
const (
|
||||||
|
// RepositoryRequired gives the agent a worktree of the task's repository.
|
||||||
|
// It is the default, so a workflow that does not declare one keeps working
|
||||||
|
// on a repository.
|
||||||
|
RepositoryRequired RepositoryMode = "required"
|
||||||
|
// RepositoryOptional runs with a worktree when the binding names a
|
||||||
|
// repository, and in a scratch workspace when it does not.
|
||||||
|
RepositoryOptional RepositoryMode = "optional"
|
||||||
|
// RepositoryNone runs in a scratch workspace with no clone.
|
||||||
|
RepositoryNone RepositoryMode = "none"
|
||||||
|
)
|
||||||
|
|
||||||
|
// InputSpec declares one workflow input. Type is one of the mapping field
|
||||||
|
// types, so a mapped parameter and the input it feeds are checked alike.
|
||||||
|
type InputSpec struct {
|
||||||
|
Type string `yaml:"type" json:"type"`
|
||||||
|
Required bool `yaml:"required,omitempty" json:"required,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// WorkflowInterface is what a workflow declares to whatever starts it: the
|
||||||
|
// inputs it takes, whether it works on a repository, and the agent profile it
|
||||||
|
// runs under. It lives beside the definition entry so a process that does not
|
||||||
|
// link the engine can check a binding against it.
|
||||||
|
type WorkflowInterface struct {
|
||||||
|
Inputs map[string]InputSpec `yaml:"inputs,omitempty" json:"inputs,omitempty"`
|
||||||
|
Repository RepositoryMode `yaml:"repository,omitempty" json:"repository,omitempty"`
|
||||||
|
Profile string `yaml:"profile,omitempty" json:"profile,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
var (
|
||||||
|
inputName = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
|
||||||
|
inputTypes = []string{"string", "number", "bool", "object", "array", "any"}
|
||||||
|
)
|
||||||
|
|
||||||
|
// ParseWorkflowInterface reads the interface a workflow YAML declares,
|
||||||
|
// ignoring its steps.
|
||||||
|
func ParseWorkflowInterface(src string) (WorkflowInterface, error) {
|
||||||
|
var w WorkflowInterface
|
||||||
|
if err := yaml.Unmarshal([]byte(src), &w); err != nil {
|
||||||
|
return WorkflowInterface{}, fmt.Errorf("parse workflow YAML: %w", err)
|
||||||
|
}
|
||||||
|
return w, w.Validate()
|
||||||
|
}
|
||||||
|
|
||||||
|
// Validate rejects an unknown repository mode, a malformed input name and an
|
||||||
|
// unknown input type.
|
||||||
|
func (w WorkflowInterface) Validate() error {
|
||||||
|
switch w.Repository {
|
||||||
|
case "", RepositoryRequired, RepositoryOptional, RepositoryNone:
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("workflow repository %q must be none, optional or required", w.Repository)
|
||||||
|
}
|
||||||
|
for name, spec := range w.Inputs {
|
||||||
|
if !inputName.MatchString(name) {
|
||||||
|
return fmt.Errorf("workflow input %q must be a letter or underscore followed by letters, digits or underscores", name)
|
||||||
|
}
|
||||||
|
if !slices.Contains(inputTypes, spec.Type) {
|
||||||
|
return fmt.Errorf("workflow input %q type %q must be one of %s", name, spec.Type, strings.Join(inputTypes, ", "))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// RepositoryMode returns the declared mode, defaulting to required.
|
||||||
|
func (w WorkflowInterface) RepositoryMode() RepositoryMode {
|
||||||
|
if w.Repository == "" {
|
||||||
|
return RepositoryRequired
|
||||||
|
}
|
||||||
|
return w.Repository
|
||||||
|
}
|
||||||
|
|
||||||
|
// CheckInputs rejects values that do not satisfy the declared inputs: a
|
||||||
|
// required input that is missing or null, a value for an undeclared input, or
|
||||||
|
// a value whose type is not the declared one.
|
||||||
|
func (w WorkflowInterface) CheckInputs(values map[string]any) error {
|
||||||
|
for name, spec := range w.Inputs {
|
||||||
|
v, ok := values[name]
|
||||||
|
if spec.Required && (!ok || v == nil) {
|
||||||
|
return fmt.Errorf("input %q is required", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for name, v := range values {
|
||||||
|
spec, ok := w.Inputs[name]
|
||||||
|
if !ok {
|
||||||
|
return fmt.Errorf("input %q is not declared by the workflow", name)
|
||||||
|
}
|
||||||
|
if got := ValueType(v); v != nil && !TypeAccepts(spec.Type, got) {
|
||||||
|
return fmt.Errorf("input %q is %s, want %s", name, got, spec.Type)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// TypeAccepts reports whether a value or parameter of type got may feed an
|
||||||
|
// input declared as want.
|
||||||
|
func TypeAccepts(want, got string) bool {
|
||||||
|
return want == "any" || got == "any" || want == got
|
||||||
|
}
|
||||||
|
|
||||||
|
// ValueType names the JSON type of a decoded value, in the mapping field type
|
||||||
|
// vocabulary.
|
||||||
|
func ValueType(v any) string {
|
||||||
|
switch v.(type) {
|
||||||
|
case nil:
|
||||||
|
return "null"
|
||||||
|
case string:
|
||||||
|
return "string"
|
||||||
|
case bool:
|
||||||
|
return "bool"
|
||||||
|
case json.Number, float64, float32, int, int32, int64:
|
||||||
|
return "number"
|
||||||
|
case map[string]any:
|
||||||
|
return "object"
|
||||||
|
case []any:
|
||||||
|
return "array"
|
||||||
|
default:
|
||||||
|
return fmt.Sprintf("%T", v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// EncodeInputs is the stored and wire form of a task's inputs: a JSON
|
||||||
|
// object, or empty for none.
|
||||||
|
func EncodeInputs(inputs map[string]any) (string, error) {
|
||||||
|
if len(inputs) == 0 {
|
||||||
|
return "", nil
|
||||||
|
}
|
||||||
|
data, err := json.Marshal(inputs)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("encode task inputs: %w", err)
|
||||||
|
}
|
||||||
|
return string(data), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// DecodeInputs reverses EncodeInputs, keeping numbers exact.
|
||||||
|
func DecodeInputs(s string) (map[string]any, error) {
|
||||||
|
if s == "" {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
decoder := json.NewDecoder(strings.NewReader(s))
|
||||||
|
decoder.UseNumber()
|
||||||
|
var inputs map[string]any
|
||||||
|
if err := decoder.Decode(&inputs); err != nil {
|
||||||
|
return nil, fmt.Errorf("decode task inputs: %w", err)
|
||||||
|
}
|
||||||
|
return inputs, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
package task
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestParseWorkflowInterface(t *testing.T) {
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
src string
|
||||||
|
wantErr string
|
||||||
|
wantMode RepositoryMode
|
||||||
|
}{
|
||||||
|
{name: "undeclared repository defaults to required", src: "id: a\nsteps: []\n", wantMode: RepositoryRequired},
|
||||||
|
{name: "none", src: "id: a\nrepository: none\n", wantMode: RepositoryNone},
|
||||||
|
{name: "unknown mode", src: "id: a\nrepository: sometimes\n", wantErr: "must be none, optional or required"},
|
||||||
|
{name: "unknown input type", src: "id: a\ninputs:\n ip: {type: ipv4}\n", wantErr: `type "ipv4"`},
|
||||||
|
{name: "malformed input name", src: "id: a\ninputs:\n src-ip: {type: string}\n", wantErr: `input "src-ip"`},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
w, err := ParseWorkflowInterface(tt.src)
|
||||||
|
if tt.wantErr != "" {
|
||||||
|
if err == nil || !strings.Contains(err.Error(), tt.wantErr) {
|
||||||
|
t.Fatalf("err = %v, want containing %q", err, tt.wantErr)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got := w.RepositoryMode(); got != tt.wantMode {
|
||||||
|
t.Errorf("RepositoryMode() = %q, want %q", got, tt.wantMode)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCheckInputs(t *testing.T) {
|
||||||
|
w := WorkflowInterface{Inputs: map[string]InputSpec{
|
||||||
|
"src_ip": {Type: "string", Required: true},
|
||||||
|
"severity": {Type: "number"},
|
||||||
|
"extra": {Type: "any"},
|
||||||
|
}}
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
values map[string]any
|
||||||
|
wantErr string
|
||||||
|
}{
|
||||||
|
{name: "required and typed", values: map[string]any{"src_ip": "10.0.0.1", "severity": json.Number("3")}},
|
||||||
|
{name: "optional null is absent", values: map[string]any{"src_ip": "10.0.0.1", "severity": nil}},
|
||||||
|
{name: "any takes an object", values: map[string]any{"src_ip": "x", "extra": map[string]any{"a": 1.0}}},
|
||||||
|
{name: "missing required", values: map[string]any{"severity": 1.0}, wantErr: `"src_ip" is required`},
|
||||||
|
{name: "null required", values: map[string]any{"src_ip": nil}, wantErr: `"src_ip" is required`},
|
||||||
|
{name: "undeclared", values: map[string]any{"src_ip": "x", "port": 22.0}, wantErr: `"port" is not declared`},
|
||||||
|
{name: "wrong type", values: map[string]any{"src_ip": "x", "severity": "high"}, wantErr: `"severity" is string, want number`},
|
||||||
|
}
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
err := w.CheckInputs(tt.values)
|
||||||
|
if tt.wantErr == "" {
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("CheckInputs() = %v", err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err == nil || !strings.Contains(err.Error(), tt.wantErr) {
|
||||||
|
t.Fatalf("CheckInputs() = %v, want containing %q", err, tt.wantErr)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -30,6 +30,7 @@ const (
|
|||||||
StatusDead = taskstate.Dead
|
StatusDead = taskstate.Dead
|
||||||
StatusRejected = taskstate.Rejected
|
StatusRejected = taskstate.Rejected
|
||||||
StatusClosedWontDo = taskstate.Declined
|
StatusClosedWontDo = taskstate.Declined
|
||||||
|
StatusCompleted = taskstate.Completed
|
||||||
)
|
)
|
||||||
|
|
||||||
// Task is the task-execution record the workflow domain operates on.
|
// Task is the task-execution record the workflow domain operates on.
|
||||||
@@ -91,6 +92,10 @@ type Task struct {
|
|||||||
// later edits to the binding cannot silently rewrite history.
|
// later edits to the binding cannot silently rewrite history.
|
||||||
BindingID string `json:"binding_id"`
|
BindingID string `json:"binding_id"`
|
||||||
BindingVersion int `json:"binding_version"`
|
BindingVersion int `json:"binding_version"`
|
||||||
|
// Inputs are the workflow inputs the binding assigned, checked against
|
||||||
|
// the workflow's declared inputs at dispatch. They reach the agent as
|
||||||
|
// structured data, never as body text.
|
||||||
|
Inputs map[string]any `json:"inputs,omitempty"`
|
||||||
// ReviewPayload is the JSON-encoded review unit (the forge review's
|
// ReviewPayload is the JSON-encoded review unit (the forge review's
|
||||||
// actionable comments) the remediate workflow's current run must
|
// actionable comments) the remediate workflow's current run must
|
||||||
// address. The daemon's reaction consumer injects it before queuing a
|
// address. The daemon's reaction consumer injects it before queuing a
|
||||||
@@ -153,4 +158,16 @@ type Definition struct {
|
|||||||
Name string `json:"name"`
|
Name string `json:"name"`
|
||||||
Origin string `json:"origin"`
|
Origin string `json:"origin"`
|
||||||
Enabled bool `json:"enabled"`
|
Enabled bool `json:"enabled"`
|
||||||
|
// Inputs and Repository are what the workflow declares, so a binding
|
||||||
|
// editor can offer the inputs to assign and whether to name a repository.
|
||||||
|
Inputs map[string]InputSpec `json:"inputs,omitempty"`
|
||||||
|
Repository RepositoryMode `json:"repository,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// HasRepository reports whether the task works on a repository. A task a
|
||||||
|
// binding started for a workflow with repository none (or optional, with no
|
||||||
|
// repository named) has neither owner nor repo and runs in a scratch
|
||||||
|
// workspace.
|
||||||
|
func (t Task) HasRepository() bool {
|
||||||
|
return t.Owner != "" || t.Repo != ""
|
||||||
}
|
}
|
||||||
@@ -30,6 +30,7 @@ const (
|
|||||||
StatusDead = task.StatusDead
|
StatusDead = task.StatusDead
|
||||||
StatusRejected = task.StatusRejected
|
StatusRejected = task.StatusRejected
|
||||||
StatusClosedWontDo = task.StatusClosedWontDo
|
StatusClosedWontDo = task.StatusClosedWontDo
|
||||||
|
StatusCompleted = task.StatusCompleted
|
||||||
)
|
)
|
||||||
|
|
||||||
// Task.Source values.
|
// Task.Source values.
|
||||||
|
|||||||
@@ -152,7 +152,7 @@ func TestSubscribeTasksOwnsWireErrors(t *testing.T) {
|
|||||||
{name: "missing task", payload: []byte(`{}`), want: "validate taskrun request: task is required"},
|
{name: "missing task", payload: []byte(`{}`), want: "validate taskrun request: task is required"},
|
||||||
{name: "null task", payload: []byte(`{"task":null}`), want: "validate taskrun request: task is required"},
|
{name: "null task", payload: []byte(`{"task":null}`), want: "validate taskrun request: task is required"},
|
||||||
{name: "nonpositive task ID", payload: []byte(`{"task":{"id":0}}`), want: "validate taskrun request: task ID must be positive, got 0"},
|
{name: "nonpositive task ID", payload: []byte(`{"task":{"id":0}}`), want: "validate taskrun request: task ID must be positive, got 0"},
|
||||||
{name: "missing worktree grant", payload: []byte(`{"task":{"id":9}}`), want: "validate taskrun request: worktree grant is required"},
|
{name: "missing worktree grant", payload: []byte(`{"task":{"id":9,"owner":"acme","repo":"api"}}`), want: "validate taskrun request: worktree grant is required"},
|
||||||
{name: "missing workflow definition", payload: []byte(`{"task":{"id":9},"worktree_grant":"grant"}`), want: "validate taskrun request: workflow definition is required"},
|
{name: "missing workflow definition", payload: []byte(`{"task":{"id":9},"worktree_grant":"grant"}`), want: "validate taskrun request: workflow definition is required"},
|
||||||
{name: "payload subject mismatch", payload: []byte(`{"task":{"id":8},"worktree_grant":"grant","workflow_definition":"id: implement\nsteps: []\n"}`), want: "validate taskrun request: task ID 8 does not match subject task ID 9"},
|
{name: "payload subject mismatch", payload: []byte(`{"task":{"id":8},"worktree_grant":"grant","workflow_definition":"id: implement\nsteps: []\n"}`), want: "validate taskrun request: task ID 8 does not match subject task ID 9"},
|
||||||
{name: "handler", payload: []byte(`{"task":{"id":9},"worktree_grant":"grant","workflow_definition":"id: implement\nsteps: []\n"}`), want: "run failed", wantHandlerCalls: 1},
|
{name: "handler", payload: []byte(`{"task":{"id":9},"worktree_grant":"grant","workflow_definition":"id: implement\nsteps: []\n"}`), want: "run failed", wantHandlerCalls: 1},
|
||||||
|
|||||||
@@ -523,6 +523,9 @@ func validateContainers(cfg *config.Config) error {
|
|||||||
if cfg.Containers.VolumeTTL < 0 {
|
if cfg.Containers.VolumeTTL < 0 {
|
||||||
return fmt.Errorf("%w: containers.volume_ttl must not be negative", ErrInvalidInput)
|
return fmt.Errorf("%w: containers.volume_ttl must not be negative", ErrInvalidInput)
|
||||||
}
|
}
|
||||||
|
if err := cfg.Containers.ValidateProfiles(); err != nil {
|
||||||
|
return fmt.Errorf("%w: %w", ErrInvalidInput, err)
|
||||||
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -28,12 +28,17 @@ var droppedColumns = map[string]bool{
|
|||||||
// imported capture has no event type, so it stays unidentified and never
|
// imported capture has no event type, so it stays unidentified and never
|
||||||
// dispatches, and no legacy source could take unsigned events. An imported
|
// dispatches, and no legacy source could take unsigned events. An imported
|
||||||
// mapping has no event type, so no binding using it dispatches until it is
|
// mapping has no event type, so no binding using it dispatches until it is
|
||||||
// given one, and an imported binding has no filter.
|
// given one, and an imported binding has no filter. Nor does an imported
|
||||||
|
// binding assign workflow inputs or read its repository from a parameter,
|
||||||
|
// and no legacy task carries inputs.
|
||||||
var newColumns = map[string]bool{
|
var newColumns = map[string]bool{
|
||||||
"captures.event_type": true,
|
"captures.event_type": true,
|
||||||
"mappings.event_type": true,
|
"mappings.event_type": true,
|
||||||
"bindings.filter": true,
|
"bindings.filter": true,
|
||||||
"captures.unsigned": true,
|
"captures.unsigned": true,
|
||||||
|
"bindings.inputs": true,
|
||||||
|
"bindings.repo_param": true,
|
||||||
|
"tasks.inputs": true,
|
||||||
}
|
}
|
||||||
|
|
||||||
// targetKeys override a table's legacy key where the target has no column
|
// targetKeys override a table's legacy key where the target has no column
|
||||||
|
|||||||
@@ -301,7 +301,11 @@ func (s *EDA) RecordMappingMatch(ctx context.Context, mappingID, captureID strin
|
|||||||
|
|
||||||
// bindingValue takes GetBindingRow; the list queries' rows share its fields
|
// bindingValue takes GetBindingRow; the list queries' rows share its fields
|
||||||
// and convert to it.
|
// and convert to it.
|
||||||
func bindingValue(r postgresdb.GetBindingRow) binding.Binding {
|
func bindingValue(r postgresdb.GetBindingRow) (binding.Binding, error) {
|
||||||
|
var inputs map[string]binding.InputSource
|
||||||
|
if err := json.Unmarshal([]byte(r.Inputs), &inputs); err != nil {
|
||||||
|
return binding.Binding{}, fmt.Errorf("edastore: decode binding %s inputs: %w", r.ID, err)
|
||||||
|
}
|
||||||
return binding.Binding{
|
return binding.Binding{
|
||||||
ID: r.ID,
|
ID: r.ID,
|
||||||
Name: r.Name,
|
Name: r.Name,
|
||||||
@@ -311,27 +315,47 @@ func bindingValue(r postgresdb.GetBindingRow) binding.Binding {
|
|||||||
Workflow: r.Workflow,
|
Workflow: r.Workflow,
|
||||||
Owner: r.Owner,
|
Owner: r.Owner,
|
||||||
Repo: r.Repo,
|
Repo: r.Repo,
|
||||||
|
RepoParam: r.RepoParam,
|
||||||
|
Inputs: inputs,
|
||||||
Version: int(r.Version),
|
Version: int(r.Version),
|
||||||
Status: binding.Status(r.Status),
|
Status: binding.Status(r.Status),
|
||||||
CreatedAt: r.CreatedAt,
|
CreatedAt: r.CreatedAt,
|
||||||
UpdatedAt: r.UpdatedAt,
|
UpdatedAt: r.UpdatedAt,
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// bindingInputs encodes a binding's input assignments for storage.
|
||||||
|
func bindingInputs(b binding.Binding) (string, error) {
|
||||||
|
if len(b.Inputs) == 0 {
|
||||||
|
return "{}", nil
|
||||||
}
|
}
|
||||||
|
data, err := json.Marshal(b.Inputs)
|
||||||
|
if err != nil {
|
||||||
|
return "", fmt.Errorf("edastore: encode binding inputs: %w", err)
|
||||||
|
}
|
||||||
|
return string(data), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// InsertBinding stores a new binding as pending_approval. Any number of
|
// InsertBinding stores a new binding as pending_approval. Any number of
|
||||||
// bindings may share a source.
|
// bindings may share a source.
|
||||||
func (s *EDA) InsertBinding(ctx context.Context, b binding.Binding) (string, error) {
|
func (s *EDA) InsertBinding(ctx context.Context, b binding.Binding) (string, error) {
|
||||||
id := newRecordID()
|
id := newRecordID()
|
||||||
err := s.q.InsertBinding(ctx, postgresdb.InsertBindingParams{
|
inputs, err := bindingInputs(b)
|
||||||
ID: id,
|
if err != nil {
|
||||||
Name: b.Name,
|
return "", err
|
||||||
Source: b.Matcher.Source,
|
}
|
||||||
Mapping: b.MappingID,
|
err = s.q.InsertBinding(ctx, postgresdb.InsertBindingParams{
|
||||||
Filter: b.Filter,
|
ID: id,
|
||||||
Workflow: b.Workflow,
|
Name: b.Name,
|
||||||
Owner: b.Owner,
|
Source: b.Matcher.Source,
|
||||||
Repo: b.Repo,
|
Mapping: b.MappingID,
|
||||||
Status: string(binding.StatusPendingApproval),
|
Filter: b.Filter,
|
||||||
|
Workflow: b.Workflow,
|
||||||
|
Owner: b.Owner,
|
||||||
|
Repo: b.Repo,
|
||||||
|
Status: string(binding.StatusPendingApproval),
|
||||||
|
Inputs: inputs,
|
||||||
|
RepoParam: b.RepoParam,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return "", fmt.Errorf("edastore: insert binding: %w", err)
|
return "", fmt.Errorf("edastore: insert binding: %w", err)
|
||||||
@@ -350,7 +374,10 @@ func (s *EDA) GetBinding(ctx context.Context, id string) (*binding.Binding, erro
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
b := bindingValue(r)
|
b, err := bindingValue(r)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
return &b, nil
|
return &b, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -361,7 +388,11 @@ func (s *EDA) ListBindings(ctx context.Context) ([]binding.Binding, error) {
|
|||||||
}
|
}
|
||||||
out := make([]binding.Binding, 0, len(rows))
|
out := make([]binding.Binding, 0, len(rows))
|
||||||
for _, r := range rows {
|
for _, r := range rows {
|
||||||
out = append(out, bindingValue(postgresdb.GetBindingRow(r)))
|
b, err := bindingValue(postgresdb.GetBindingRow(r))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out = append(out, b)
|
||||||
}
|
}
|
||||||
return out, nil
|
return out, nil
|
||||||
}
|
}
|
||||||
@@ -374,7 +405,11 @@ func (s *EDA) ArmedBindingsForSource(ctx context.Context, source string) ([]bind
|
|||||||
}
|
}
|
||||||
out := make([]binding.Binding, 0, len(rows))
|
out := make([]binding.Binding, 0, len(rows))
|
||||||
for _, r := range rows {
|
for _, r := range rows {
|
||||||
out = append(out, bindingValue(postgresdb.GetBindingRow(r)))
|
b, err := bindingValue(postgresdb.GetBindingRow(r))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out = append(out, b)
|
||||||
}
|
}
|
||||||
return out, nil
|
return out, nil
|
||||||
}
|
}
|
||||||
@@ -382,16 +417,22 @@ func (s *EDA) ArmedBindingsForSource(ctx context.Context, source string) ([]bind
|
|||||||
// UpdateBinding rewrites a binding's editable fields, bumps its version and
|
// UpdateBinding rewrites a binding's editable fields, bumps its version and
|
||||||
// drops it back to pending_approval.
|
// drops it back to pending_approval.
|
||||||
func (s *EDA) UpdateBinding(ctx context.Context, b binding.Binding) error {
|
func (s *EDA) UpdateBinding(ctx context.Context, b binding.Binding) error {
|
||||||
|
inputs, err := bindingInputs(b)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
n, err := s.q.UpdateBinding(ctx, postgresdb.UpdateBindingParams{
|
n, err := s.q.UpdateBinding(ctx, postgresdb.UpdateBindingParams{
|
||||||
ID: b.ID,
|
ID: b.ID,
|
||||||
Name: b.Name,
|
Name: b.Name,
|
||||||
Source: b.Matcher.Source,
|
Source: b.Matcher.Source,
|
||||||
Mapping: b.MappingID,
|
Mapping: b.MappingID,
|
||||||
Filter: b.Filter,
|
Filter: b.Filter,
|
||||||
Workflow: b.Workflow,
|
Workflow: b.Workflow,
|
||||||
Owner: b.Owner,
|
Owner: b.Owner,
|
||||||
Repo: b.Repo,
|
Repo: b.Repo,
|
||||||
Status: string(binding.StatusPendingApproval),
|
Status: string(binding.StatusPendingApproval),
|
||||||
|
Inputs: inputs,
|
||||||
|
RepoParam: b.RepoParam,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
|
|||||||
@@ -0,0 +1,21 @@
|
|||||||
|
-- Workflow inputs and bindings that assign them
|
||||||
|
-- (docs/prds/event-automation.md, "Workflows declare inputs" and "Bindings").
|
||||||
|
--
|
||||||
|
-- bindings.inputs is a JSON object assigning each workflow input either a
|
||||||
|
-- mapped parameter ({"param": "src_ip"}) or a constant ({"value": 3}).
|
||||||
|
-- bindings.repo_param names the mapped parameter holding "owner/name" when the
|
||||||
|
-- repository comes from the event rather than a fixed owner/repo.
|
||||||
|
-- tasks.inputs is the JSON object of input values a binding dispatch resolved;
|
||||||
|
-- empty for a task no binding started.
|
||||||
|
|
||||||
|
-- +goose Up
|
||||||
|
|
||||||
|
ALTER TABLE bindings ADD COLUMN inputs text NOT NULL DEFAULT '{}';
|
||||||
|
ALTER TABLE bindings ADD COLUMN repo_param text NOT NULL DEFAULT '';
|
||||||
|
ALTER TABLE tasks ADD COLUMN inputs text NOT NULL DEFAULT '';
|
||||||
|
|
||||||
|
-- +goose Down
|
||||||
|
|
||||||
|
ALTER TABLE tasks DROP COLUMN inputs;
|
||||||
|
ALTER TABLE bindings DROP COLUMN repo_param;
|
||||||
|
ALTER TABLE bindings DROP COLUMN inputs;
|
||||||
@@ -11,7 +11,7 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
const armedBindingsForSource = `-- name: ArmedBindingsForSource :many
|
const armedBindingsForSource = `-- name: ArmedBindingsForSource :many
|
||||||
SELECT id, name, source, mapping, workflow, owner, repo, version, status, created_at, updated_at, filter
|
SELECT id, name, source, mapping, workflow, owner, repo, version, status, created_at, updated_at, filter, inputs, repo_param
|
||||||
FROM bindings WHERE source = $1 AND status = 'armed' ORDER BY created_at DESC
|
FROM bindings WHERE source = $1 AND status = 'armed' ORDER BY created_at DESC
|
||||||
`
|
`
|
||||||
|
|
||||||
@@ -28,6 +28,8 @@ type ArmedBindingsForSourceRow struct {
|
|||||||
CreatedAt time.Time
|
CreatedAt time.Time
|
||||||
UpdatedAt time.Time
|
UpdatedAt time.Time
|
||||||
Filter string
|
Filter string
|
||||||
|
Inputs string
|
||||||
|
RepoParam string
|
||||||
}
|
}
|
||||||
|
|
||||||
func (q *Queries) ArmedBindingsForSource(ctx context.Context, source string) ([]ArmedBindingsForSourceRow, error) {
|
func (q *Queries) ArmedBindingsForSource(ctx context.Context, source string) ([]ArmedBindingsForSourceRow, error) {
|
||||||
@@ -52,6 +54,8 @@ func (q *Queries) ArmedBindingsForSource(ctx context.Context, source string) ([]
|
|||||||
&i.CreatedAt,
|
&i.CreatedAt,
|
||||||
&i.UpdatedAt,
|
&i.UpdatedAt,
|
||||||
&i.Filter,
|
&i.Filter,
|
||||||
|
&i.Inputs,
|
||||||
|
&i.RepoParam,
|
||||||
); err != nil {
|
); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -171,7 +175,7 @@ func (q *Queries) EventTypesForSource(ctx context.Context, source string) ([]Eve
|
|||||||
}
|
}
|
||||||
|
|
||||||
const getBinding = `-- name: GetBinding :one
|
const getBinding = `-- name: GetBinding :one
|
||||||
SELECT id, name, source, mapping, workflow, owner, repo, version, status, created_at, updated_at, filter
|
SELECT id, name, source, mapping, workflow, owner, repo, version, status, created_at, updated_at, filter, inputs, repo_param
|
||||||
FROM bindings WHERE id = $1
|
FROM bindings WHERE id = $1
|
||||||
`
|
`
|
||||||
|
|
||||||
@@ -188,6 +192,8 @@ type GetBindingRow struct {
|
|||||||
CreatedAt time.Time
|
CreatedAt time.Time
|
||||||
UpdatedAt time.Time
|
UpdatedAt time.Time
|
||||||
Filter string
|
Filter string
|
||||||
|
Inputs string
|
||||||
|
RepoParam string
|
||||||
}
|
}
|
||||||
|
|
||||||
func (q *Queries) GetBinding(ctx context.Context, id string) (GetBindingRow, error) {
|
func (q *Queries) GetBinding(ctx context.Context, id string) (GetBindingRow, error) {
|
||||||
@@ -206,6 +212,8 @@ func (q *Queries) GetBinding(ctx context.Context, id string) (GetBindingRow, err
|
|||||||
&i.CreatedAt,
|
&i.CreatedAt,
|
||||||
&i.UpdatedAt,
|
&i.UpdatedAt,
|
||||||
&i.Filter,
|
&i.Filter,
|
||||||
|
&i.Inputs,
|
||||||
|
&i.RepoParam,
|
||||||
)
|
)
|
||||||
return i, err
|
return i, err
|
||||||
}
|
}
|
||||||
@@ -278,20 +286,22 @@ func (q *Queries) GetSource(ctx context.Context, path string) (Source, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const insertBinding = `-- name: InsertBinding :exec
|
const insertBinding = `-- name: InsertBinding :exec
|
||||||
INSERT INTO bindings (id, name, source, mapping, filter, workflow, owner, repo, version, status)
|
INSERT INTO bindings (id, name, source, mapping, filter, workflow, owner, repo, version, status, inputs, repo_param)
|
||||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, 1, $9)
|
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, 1, $9, $10, $11)
|
||||||
`
|
`
|
||||||
|
|
||||||
type InsertBindingParams struct {
|
type InsertBindingParams struct {
|
||||||
ID string
|
ID string
|
||||||
Name string
|
Name string
|
||||||
Source string
|
Source string
|
||||||
Mapping string
|
Mapping string
|
||||||
Filter string
|
Filter string
|
||||||
Workflow string
|
Workflow string
|
||||||
Owner string
|
Owner string
|
||||||
Repo string
|
Repo string
|
||||||
Status string
|
Status string
|
||||||
|
Inputs string
|
||||||
|
RepoParam string
|
||||||
}
|
}
|
||||||
|
|
||||||
func (q *Queries) InsertBinding(ctx context.Context, arg InsertBindingParams) error {
|
func (q *Queries) InsertBinding(ctx context.Context, arg InsertBindingParams) error {
|
||||||
@@ -305,6 +315,8 @@ func (q *Queries) InsertBinding(ctx context.Context, arg InsertBindingParams) er
|
|||||||
arg.Owner,
|
arg.Owner,
|
||||||
arg.Repo,
|
arg.Repo,
|
||||||
arg.Status,
|
arg.Status,
|
||||||
|
arg.Inputs,
|
||||||
|
arg.RepoParam,
|
||||||
)
|
)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -496,7 +508,7 @@ func (q *Queries) InsertToolCall(ctx context.Context, arg InsertToolCallParams)
|
|||||||
}
|
}
|
||||||
|
|
||||||
const listBindings = `-- name: ListBindings :many
|
const listBindings = `-- name: ListBindings :many
|
||||||
SELECT id, name, source, mapping, workflow, owner, repo, version, status, created_at, updated_at, filter
|
SELECT id, name, source, mapping, workflow, owner, repo, version, status, created_at, updated_at, filter, inputs, repo_param
|
||||||
FROM bindings ORDER BY created_at DESC
|
FROM bindings ORDER BY created_at DESC
|
||||||
`
|
`
|
||||||
|
|
||||||
@@ -513,6 +525,8 @@ type ListBindingsRow struct {
|
|||||||
CreatedAt time.Time
|
CreatedAt time.Time
|
||||||
UpdatedAt time.Time
|
UpdatedAt time.Time
|
||||||
Filter string
|
Filter string
|
||||||
|
Inputs string
|
||||||
|
RepoParam string
|
||||||
}
|
}
|
||||||
|
|
||||||
func (q *Queries) ListBindings(ctx context.Context) ([]ListBindingsRow, error) {
|
func (q *Queries) ListBindings(ctx context.Context) ([]ListBindingsRow, error) {
|
||||||
@@ -537,6 +551,8 @@ func (q *Queries) ListBindings(ctx context.Context) ([]ListBindingsRow, error) {
|
|||||||
&i.CreatedAt,
|
&i.CreatedAt,
|
||||||
&i.UpdatedAt,
|
&i.UpdatedAt,
|
||||||
&i.Filter,
|
&i.Filter,
|
||||||
|
&i.Inputs,
|
||||||
|
&i.RepoParam,
|
||||||
); err != nil {
|
); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -861,20 +877,22 @@ func (q *Queries) TaskToolCalls(ctx context.Context, taskID int64) ([]TaskToolCa
|
|||||||
const updateBinding = `-- name: UpdateBinding :execrows
|
const updateBinding = `-- name: UpdateBinding :execrows
|
||||||
UPDATE bindings
|
UPDATE bindings
|
||||||
SET name = $2, source = $3, mapping = $4, filter = $5, workflow = $6, owner = $7, repo = $8,
|
SET name = $2, source = $3, mapping = $4, filter = $5, workflow = $6, owner = $7, repo = $8,
|
||||||
version = version + 1, status = $9, updated_at = now()
|
version = version + 1, status = $9, inputs = $10, repo_param = $11, updated_at = now()
|
||||||
WHERE id = $1
|
WHERE id = $1
|
||||||
`
|
`
|
||||||
|
|
||||||
type UpdateBindingParams struct {
|
type UpdateBindingParams struct {
|
||||||
ID string
|
ID string
|
||||||
Name string
|
Name string
|
||||||
Source string
|
Source string
|
||||||
Mapping string
|
Mapping string
|
||||||
Filter string
|
Filter string
|
||||||
Workflow string
|
Workflow string
|
||||||
Owner string
|
Owner string
|
||||||
Repo string
|
Repo string
|
||||||
Status string
|
Status string
|
||||||
|
Inputs string
|
||||||
|
RepoParam string
|
||||||
}
|
}
|
||||||
|
|
||||||
func (q *Queries) UpdateBinding(ctx context.Context, arg UpdateBindingParams) (int64, error) {
|
func (q *Queries) UpdateBinding(ctx context.Context, arg UpdateBindingParams) (int64, error) {
|
||||||
@@ -888,6 +906,8 @@ func (q *Queries) UpdateBinding(ctx context.Context, arg UpdateBindingParams) (i
|
|||||||
arg.Owner,
|
arg.Owner,
|
||||||
arg.Repo,
|
arg.Repo,
|
||||||
arg.Status,
|
arg.Status,
|
||||||
|
arg.Inputs,
|
||||||
|
arg.RepoParam,
|
||||||
)
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return 0, err
|
return 0, err
|
||||||
|
|||||||
@@ -30,6 +30,8 @@ type Binding struct {
|
|||||||
CreatedAt time.Time
|
CreatedAt time.Time
|
||||||
UpdatedAt time.Time
|
UpdatedAt time.Time
|
||||||
Filter string
|
Filter string
|
||||||
|
Inputs string
|
||||||
|
RepoParam string
|
||||||
}
|
}
|
||||||
|
|
||||||
type BindingDispatch struct {
|
type BindingDispatch struct {
|
||||||
@@ -251,6 +253,7 @@ type Task struct {
|
|||||||
CreatedAt time.Time
|
CreatedAt time.Time
|
||||||
UpdatedAt time.Time
|
UpdatedAt time.Time
|
||||||
ReviewCursor int64
|
ReviewCursor int64
|
||||||
|
Inputs string
|
||||||
}
|
}
|
||||||
|
|
||||||
type ToolCall struct {
|
type ToolCall struct {
|
||||||
|
|||||||
@@ -48,7 +48,7 @@ func (q *Queries) BeginRemediationTask(ctx context.Context, arg BeginRemediation
|
|||||||
const claimByIssue = `-- name: ClaimByIssue :one
|
const claimByIssue = `-- name: ClaimByIssue :one
|
||||||
UPDATE tasks SET status = 'running', attempt = attempt + 1, updated_at = now()
|
UPDATE tasks SET status = 'running', attempt = attempt + 1, updated_at = now()
|
||||||
WHERE owner = $1 AND repo = $2 AND issue_number = $3 AND status = 'queued'
|
WHERE owner = $1 AND repo = $2 AND issue_number = $3 AND status = 'queued'
|
||||||
RETURNING id, owner, repo, issue_number, title, body, labels, status, workflow, stage, branch, plan, notes, pr_number, tokens_used, iterations, attempt, park_reason, watch_comment_id, park_class, remediation_rounds, retry_count, source, identity, binding_id, binding_version, review_payload, workflow_definition_version, workflow_definition_digest, workflow_definition_yaml, created_at, updated_at, review_cursor
|
RETURNING id, owner, repo, issue_number, title, body, labels, status, workflow, stage, branch, plan, notes, pr_number, tokens_used, iterations, attempt, park_reason, watch_comment_id, park_class, remediation_rounds, retry_count, source, identity, binding_id, binding_version, review_payload, workflow_definition_version, workflow_definition_digest, workflow_definition_yaml, created_at, updated_at, review_cursor, inputs
|
||||||
`
|
`
|
||||||
|
|
||||||
type ClaimByIssueParams struct {
|
type ClaimByIssueParams struct {
|
||||||
@@ -94,6 +94,7 @@ func (q *Queries) ClaimByIssue(ctx context.Context, arg ClaimByIssueParams) (Tas
|
|||||||
&i.CreatedAt,
|
&i.CreatedAt,
|
||||||
&i.UpdatedAt,
|
&i.UpdatedAt,
|
||||||
&i.ReviewCursor,
|
&i.ReviewCursor,
|
||||||
|
&i.Inputs,
|
||||||
)
|
)
|
||||||
return i, err
|
return i, err
|
||||||
}
|
}
|
||||||
@@ -108,7 +109,7 @@ WHERE id = (
|
|||||||
FOR UPDATE SKIP LOCKED
|
FOR UPDATE SKIP LOCKED
|
||||||
LIMIT 1
|
LIMIT 1
|
||||||
)
|
)
|
||||||
RETURNING id, owner, repo, issue_number, title, body, labels, status, workflow, stage, branch, plan, notes, pr_number, tokens_used, iterations, attempt, park_reason, watch_comment_id, park_class, remediation_rounds, retry_count, source, identity, binding_id, binding_version, review_payload, workflow_definition_version, workflow_definition_digest, workflow_definition_yaml, created_at, updated_at, review_cursor
|
RETURNING id, owner, repo, issue_number, title, body, labels, status, workflow, stage, branch, plan, notes, pr_number, tokens_used, iterations, attempt, park_reason, watch_comment_id, park_class, remediation_rounds, retry_count, source, identity, binding_id, binding_version, review_payload, workflow_definition_version, workflow_definition_digest, workflow_definition_yaml, created_at, updated_at, review_cursor, inputs
|
||||||
`
|
`
|
||||||
|
|
||||||
// FOR UPDATE SKIP LOCKED replaces the SQLite single-writer assumption: two
|
// FOR UPDATE SKIP LOCKED replaces the SQLite single-writer assumption: two
|
||||||
@@ -151,6 +152,7 @@ func (q *Queries) ClaimNextTask(ctx context.Context) (Task, error) {
|
|||||||
&i.CreatedAt,
|
&i.CreatedAt,
|
||||||
&i.UpdatedAt,
|
&i.UpdatedAt,
|
||||||
&i.ReviewCursor,
|
&i.ReviewCursor,
|
||||||
|
&i.Inputs,
|
||||||
)
|
)
|
||||||
return i, err
|
return i, err
|
||||||
}
|
}
|
||||||
@@ -241,7 +243,7 @@ VALUES (
|
|||||||
), $3) + 1,
|
), $3) + 1,
|
||||||
$4, $5, 'chat', $6, 'chat', $7
|
$4, $5, 'chat', $6, 'chat', $7
|
||||||
)
|
)
|
||||||
RETURNING id, owner, repo, issue_number, title, body, labels, status, workflow, stage, branch, plan, notes, pr_number, tokens_used, iterations, attempt, park_reason, watch_comment_id, park_class, remediation_rounds, retry_count, source, identity, binding_id, binding_version, review_payload, workflow_definition_version, workflow_definition_digest, workflow_definition_yaml, created_at, updated_at, review_cursor
|
RETURNING id, owner, repo, issue_number, title, body, labels, status, workflow, stage, branch, plan, notes, pr_number, tokens_used, iterations, attempt, park_reason, watch_comment_id, park_class, remediation_rounds, retry_count, source, identity, binding_id, binding_version, review_payload, workflow_definition_version, workflow_definition_digest, workflow_definition_yaml, created_at, updated_at, review_cursor, inputs
|
||||||
`
|
`
|
||||||
|
|
||||||
type InsertChatTaskParams struct {
|
type InsertChatTaskParams struct {
|
||||||
@@ -303,6 +305,7 @@ func (q *Queries) InsertChatTask(ctx context.Context, arg InsertChatTaskParams)
|
|||||||
&i.CreatedAt,
|
&i.CreatedAt,
|
||||||
&i.UpdatedAt,
|
&i.UpdatedAt,
|
||||||
&i.ReviewCursor,
|
&i.ReviewCursor,
|
||||||
|
&i.Inputs,
|
||||||
)
|
)
|
||||||
return i, err
|
return i, err
|
||||||
}
|
}
|
||||||
@@ -556,22 +559,28 @@ func (q *Queries) SetReviewCursors(ctx context.Context, arg SetReviewCursorsPara
|
|||||||
}
|
}
|
||||||
|
|
||||||
const stampTaskBinding = `-- name: StampTaskBinding :exec
|
const stampTaskBinding = `-- name: StampTaskBinding :exec
|
||||||
UPDATE tasks SET binding_id = $2, binding_version = $3 WHERE id = $1
|
UPDATE tasks SET binding_id = $2, binding_version = $3, inputs = $4 WHERE id = $1
|
||||||
`
|
`
|
||||||
|
|
||||||
type StampTaskBindingParams struct {
|
type StampTaskBindingParams struct {
|
||||||
ID int64
|
ID int64
|
||||||
BindingID string
|
BindingID string
|
||||||
BindingVersion int64
|
BindingVersion int64
|
||||||
|
Inputs string
|
||||||
}
|
}
|
||||||
|
|
||||||
func (q *Queries) StampTaskBinding(ctx context.Context, arg StampTaskBindingParams) error {
|
func (q *Queries) StampTaskBinding(ctx context.Context, arg StampTaskBindingParams) error {
|
||||||
_, err := q.db.Exec(ctx, stampTaskBinding, arg.ID, arg.BindingID, arg.BindingVersion)
|
_, err := q.db.Exec(ctx, stampTaskBinding,
|
||||||
|
arg.ID,
|
||||||
|
arg.BindingID,
|
||||||
|
arg.BindingVersion,
|
||||||
|
arg.Inputs,
|
||||||
|
)
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|
||||||
const taskByID = `-- name: TaskByID :one
|
const taskByID = `-- name: TaskByID :one
|
||||||
SELECT id, owner, repo, issue_number, title, body, labels, status, workflow, stage, branch, plan, notes, pr_number, tokens_used, iterations, attempt, park_reason, watch_comment_id, park_class, remediation_rounds, retry_count, source, identity, binding_id, binding_version, review_payload, workflow_definition_version, workflow_definition_digest, workflow_definition_yaml, created_at, updated_at, review_cursor FROM tasks WHERE id = $1
|
SELECT id, owner, repo, issue_number, title, body, labels, status, workflow, stage, branch, plan, notes, pr_number, tokens_used, iterations, attempt, park_reason, watch_comment_id, park_class, remediation_rounds, retry_count, source, identity, binding_id, binding_version, review_payload, workflow_definition_version, workflow_definition_digest, workflow_definition_yaml, created_at, updated_at, review_cursor, inputs FROM tasks WHERE id = $1
|
||||||
`
|
`
|
||||||
|
|
||||||
func (q *Queries) TaskByID(ctx context.Context, id int64) (Task, error) {
|
func (q *Queries) TaskByID(ctx context.Context, id int64) (Task, error) {
|
||||||
@@ -611,12 +620,13 @@ func (q *Queries) TaskByID(ctx context.Context, id int64) (Task, error) {
|
|||||||
&i.CreatedAt,
|
&i.CreatedAt,
|
||||||
&i.UpdatedAt,
|
&i.UpdatedAt,
|
||||||
&i.ReviewCursor,
|
&i.ReviewCursor,
|
||||||
|
&i.Inputs,
|
||||||
)
|
)
|
||||||
return i, err
|
return i, err
|
||||||
}
|
}
|
||||||
|
|
||||||
const taskByIssue = `-- name: TaskByIssue :one
|
const taskByIssue = `-- name: TaskByIssue :one
|
||||||
SELECT id, owner, repo, issue_number, title, body, labels, status, workflow, stage, branch, plan, notes, pr_number, tokens_used, iterations, attempt, park_reason, watch_comment_id, park_class, remediation_rounds, retry_count, source, identity, binding_id, binding_version, review_payload, workflow_definition_version, workflow_definition_digest, workflow_definition_yaml, created_at, updated_at, review_cursor FROM tasks WHERE owner = $1 AND repo = $2 AND issue_number = $3
|
SELECT id, owner, repo, issue_number, title, body, labels, status, workflow, stage, branch, plan, notes, pr_number, tokens_used, iterations, attempt, park_reason, watch_comment_id, park_class, remediation_rounds, retry_count, source, identity, binding_id, binding_version, review_payload, workflow_definition_version, workflow_definition_digest, workflow_definition_yaml, created_at, updated_at, review_cursor, inputs FROM tasks WHERE owner = $1 AND repo = $2 AND issue_number = $3
|
||||||
`
|
`
|
||||||
|
|
||||||
type TaskByIssueParams struct {
|
type TaskByIssueParams struct {
|
||||||
@@ -662,12 +672,13 @@ func (q *Queries) TaskByIssue(ctx context.Context, arg TaskByIssueParams) (Task,
|
|||||||
&i.CreatedAt,
|
&i.CreatedAt,
|
||||||
&i.UpdatedAt,
|
&i.UpdatedAt,
|
||||||
&i.ReviewCursor,
|
&i.ReviewCursor,
|
||||||
|
&i.Inputs,
|
||||||
)
|
)
|
||||||
return i, err
|
return i, err
|
||||||
}
|
}
|
||||||
|
|
||||||
const taskByPR = `-- name: TaskByPR :one
|
const taskByPR = `-- name: TaskByPR :one
|
||||||
SELECT id, owner, repo, issue_number, title, body, labels, status, workflow, stage, branch, plan, notes, pr_number, tokens_used, iterations, attempt, park_reason, watch_comment_id, park_class, remediation_rounds, retry_count, source, identity, binding_id, binding_version, review_payload, workflow_definition_version, workflow_definition_digest, workflow_definition_yaml, created_at, updated_at, review_cursor FROM tasks WHERE owner = $1 AND repo = $2 AND pr_number = $3 AND status = $4
|
SELECT id, owner, repo, issue_number, title, body, labels, status, workflow, stage, branch, plan, notes, pr_number, tokens_used, iterations, attempt, park_reason, watch_comment_id, park_class, remediation_rounds, retry_count, source, identity, binding_id, binding_version, review_payload, workflow_definition_version, workflow_definition_digest, workflow_definition_yaml, created_at, updated_at, review_cursor, inputs FROM tasks WHERE owner = $1 AND repo = $2 AND pr_number = $3 AND status = $4
|
||||||
`
|
`
|
||||||
|
|
||||||
type TaskByPRParams struct {
|
type TaskByPRParams struct {
|
||||||
@@ -722,6 +733,7 @@ func (q *Queries) TaskByPR(ctx context.Context, arg TaskByPRParams) (Task, error
|
|||||||
&i.CreatedAt,
|
&i.CreatedAt,
|
||||||
&i.UpdatedAt,
|
&i.UpdatedAt,
|
||||||
&i.ReviewCursor,
|
&i.ReviewCursor,
|
||||||
|
&i.Inputs,
|
||||||
)
|
)
|
||||||
return i, err
|
return i, err
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -63,25 +63,25 @@ ON CONFLICT DO NOTHING;
|
|||||||
DELETE FROM mappings WHERE id = $1;
|
DELETE FROM mappings WHERE id = $1;
|
||||||
|
|
||||||
-- name: InsertBinding :exec
|
-- name: InsertBinding :exec
|
||||||
INSERT INTO bindings (id, name, source, mapping, filter, workflow, owner, repo, version, status)
|
INSERT INTO bindings (id, name, source, mapping, filter, workflow, owner, repo, version, status, inputs, repo_param)
|
||||||
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, 1, $9);
|
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, 1, $9, $10, $11);
|
||||||
|
|
||||||
-- name: GetBinding :one
|
-- name: GetBinding :one
|
||||||
SELECT id, name, source, mapping, workflow, owner, repo, version, status, created_at, updated_at, filter
|
SELECT id, name, source, mapping, workflow, owner, repo, version, status, created_at, updated_at, filter, inputs, repo_param
|
||||||
FROM bindings WHERE id = $1;
|
FROM bindings WHERE id = $1;
|
||||||
|
|
||||||
-- name: ListBindings :many
|
-- name: ListBindings :many
|
||||||
SELECT id, name, source, mapping, workflow, owner, repo, version, status, created_at, updated_at, filter
|
SELECT id, name, source, mapping, workflow, owner, repo, version, status, created_at, updated_at, filter, inputs, repo_param
|
||||||
FROM bindings ORDER BY created_at DESC;
|
FROM bindings ORDER BY created_at DESC;
|
||||||
|
|
||||||
-- name: ArmedBindingsForSource :many
|
-- name: ArmedBindingsForSource :many
|
||||||
SELECT id, name, source, mapping, workflow, owner, repo, version, status, created_at, updated_at, filter
|
SELECT id, name, source, mapping, workflow, owner, repo, version, status, created_at, updated_at, filter, inputs, repo_param
|
||||||
FROM bindings WHERE source = $1 AND status = 'armed' ORDER BY created_at DESC;
|
FROM bindings WHERE source = $1 AND status = 'armed' ORDER BY created_at DESC;
|
||||||
|
|
||||||
-- name: UpdateBinding :execrows
|
-- name: UpdateBinding :execrows
|
||||||
UPDATE bindings
|
UPDATE bindings
|
||||||
SET name = $2, source = $3, mapping = $4, filter = $5, workflow = $6, owner = $7, repo = $8,
|
SET name = $2, source = $3, mapping = $4, filter = $5, workflow = $6, owner = $7, repo = $8,
|
||||||
version = version + 1, status = $9, updated_at = now()
|
version = version + 1, status = $9, inputs = $10, repo_param = $11, updated_at = now()
|
||||||
WHERE id = $1;
|
WHERE id = $1;
|
||||||
|
|
||||||
-- name: SetBindingArmed :execrows
|
-- name: SetBindingArmed :execrows
|
||||||
|
|||||||
@@ -56,7 +56,7 @@ VALUES (
|
|||||||
RETURNING *;
|
RETURNING *;
|
||||||
|
|
||||||
-- name: StampTaskBinding :exec
|
-- name: StampTaskBinding :exec
|
||||||
UPDATE tasks SET binding_id = $2, binding_version = $3 WHERE id = $1;
|
UPDATE tasks SET binding_id = $2, binding_version = $3, inputs = $4 WHERE id = $1;
|
||||||
|
|
||||||
-- name: UpdateTask :exec
|
-- name: UpdateTask :exec
|
||||||
UPDATE tasks SET workflow = $2, stage = $3, branch = $4, plan = $5, notes = $6,
|
UPDATE tasks SET workflow = $2, stage = $3, branch = $4, plan = $5, notes = $6,
|
||||||
|
|||||||
@@ -11,6 +11,7 @@ import (
|
|||||||
|
|
||||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||||
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
"github.com/samcharles93/archie-core/internal/domain/workflow"
|
||||||
|
workflowtask "github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||||
"github.com/samcharles93/archie-core/internal/events"
|
"github.com/samcharles93/archie-core/internal/events"
|
||||||
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/postgresdb"
|
"github.com/samcharles93/archie-core/internal/infrastructure/postgres/postgresdb"
|
||||||
"github.com/samcharles93/archie-core/internal/taskstate"
|
"github.com/samcharles93/archie-core/internal/taskstate"
|
||||||
@@ -48,6 +49,9 @@ const syntheticIssueNumberBase = 1_000_000_000_000_000
|
|||||||
// taskFromRow maps the generated task row to the workflow.Task the daemon and
|
// taskFromRow maps the generated task row to the workflow.Task the daemon and
|
||||||
// webui consume.
|
// webui consume.
|
||||||
func taskFromRow(t postgresdb.Task) *workflow.Task {
|
func taskFromRow(t postgresdb.Task) *workflow.Task {
|
||||||
|
// The inputs column is only ever written by EncodeInputs, so a decode
|
||||||
|
// failure cannot arise from stored data this package produced.
|
||||||
|
inputs, _ := workflowtask.DecodeInputs(t.Inputs)
|
||||||
return &workflow.Task{
|
return &workflow.Task{
|
||||||
ID: t.ID,
|
ID: t.ID,
|
||||||
Owner: t.Owner,
|
Owner: t.Owner,
|
||||||
@@ -77,6 +81,7 @@ func taskFromRow(t postgresdb.Task) *workflow.Task {
|
|||||||
Identity: t.Identity,
|
Identity: t.Identity,
|
||||||
BindingID: t.BindingID,
|
BindingID: t.BindingID,
|
||||||
BindingVersion: int(t.BindingVersion),
|
BindingVersion: int(t.BindingVersion),
|
||||||
|
Inputs: inputs,
|
||||||
ReviewPayload: t.ReviewPayload,
|
ReviewPayload: t.ReviewPayload,
|
||||||
ParkClass: t.ParkClass,
|
ParkClass: t.ParkClass,
|
||||||
RemediationRounds: int(t.RemediationRounds),
|
RemediationRounds: int(t.RemediationRounds),
|
||||||
@@ -124,18 +129,23 @@ func (s *Store) EnqueueChatTask(ctx context.Context, owner, repo, title, body, w
|
|||||||
|
|
||||||
// EnqueueBindingTask enqueues a binding-triggered task and stamps its binding
|
// EnqueueBindingTask enqueues a binding-triggered task and stamps its binding
|
||||||
// provenance in a second statement (best-effort, as in the SQLite store).
|
// provenance in a second statement (best-effort, as in the SQLite store).
|
||||||
func (s *Store) EnqueueBindingTask(ctx context.Context, owner, repo, title, body, wf, identity, bindingID string, bindingVersion int) (*workflow.Task, error) {
|
func (s *Store) EnqueueBindingTask(ctx context.Context, owner, repo, title, body, wf, identity, bindingID string, bindingVersion int, inputs map[string]any) (*workflow.Task, error) {
|
||||||
|
encoded, err := workflowtask.EncodeInputs(inputs)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
t, err := s.EnqueueChatTask(ctx, owner, repo, title, body, wf, identity)
|
t, err := s.EnqueueChatTask(ctx, owner, repo, title, body, wf, identity)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if err := s.queries().StampTaskBinding(ctx, postgresdb.StampTaskBindingParams{
|
if err := s.queries().StampTaskBinding(ctx, postgresdb.StampTaskBindingParams{
|
||||||
ID: t.ID, BindingID: bindingID, BindingVersion: int64(bindingVersion),
|
ID: t.ID, BindingID: bindingID, BindingVersion: int64(bindingVersion), Inputs: encoded,
|
||||||
}); err != nil {
|
}); err != nil {
|
||||||
return nil, fmt.Errorf("store: stamp binding provenance: %w", err)
|
return nil, fmt.Errorf("store: stamp binding provenance: %w", err)
|
||||||
}
|
}
|
||||||
t.BindingID = bindingID
|
t.BindingID = bindingID
|
||||||
t.BindingVersion = bindingVersion
|
t.BindingVersion = bindingVersion
|
||||||
|
t.Inputs = inputs
|
||||||
return t, nil
|
return t, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -554,8 +554,12 @@ func (c *Client) ListUndispatchedCaptures(ctx context.Context, sources []string,
|
|||||||
|
|
||||||
// BindingTaskCreator
|
// BindingTaskCreator
|
||||||
|
|
||||||
func (c *Client) EnqueueBindingTask(ctx context.Context, owner, repo, title, body, wf, identity, bindingID string, bindingVersion int) (*task.Task, error) {
|
func (c *Client) EnqueueBindingTask(ctx context.Context, owner, repo, title, body, wf, identity, bindingID string, bindingVersion int, inputs map[string]any) (*task.Task, error) {
|
||||||
r, err := c.client.EnqueueBindingTask(ctx, &pb.EnqueueBindingTaskRequest{Owner: owner, Repo: repo, Title: title, Body: body, Workflow: wf, Identity: identity, BindingId: bindingID, BindingVersion: int64(bindingVersion)})
|
encoded, err := task.EncodeInputs(inputs)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
r, err := c.client.EnqueueBindingTask(ctx, &pb.EnqueueBindingTaskRequest{Owner: owner, Repo: repo, Title: title, Body: body, Workflow: wf, Identity: identity, BindingId: bindingID, BindingVersion: int64(bindingVersion), InputsJson: encoded})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, unmapError(err)
|
return nil, unmapError(err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -390,13 +390,18 @@ func TestStateStoreConformance(t *testing.T) {
|
|||||||
if err != nil || bindingID == "" {
|
if err != nil || bindingID == "" {
|
||||||
t.Fatalf("InsertBinding: %v %v", bindingID, err)
|
t.Fatalf("InsertBinding: %v %v", bindingID, err)
|
||||||
}
|
}
|
||||||
_, err = ec.InsertBinding(ctx, binding.Binding{
|
secondID, err := ec.InsertBinding(ctx, binding.Binding{
|
||||||
Name: "b2", Matcher: binding.Matcher{Source: "sentry"}, MappingID: mappingID,
|
Name: "b2", Matcher: binding.Matcher{Source: "sentry"}, MappingID: mappingID,
|
||||||
Workflow: "implement",
|
Workflow: "implement", RepoParam: "full",
|
||||||
|
Inputs: map[string]binding.InputSource{"a": {Param: "p"}, "b": {Value: "x"}},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("second InsertBinding on a source = %v, want accepted", err)
|
t.Fatalf("second InsertBinding on a source = %v, want accepted", err)
|
||||||
}
|
}
|
||||||
|
second, err := ec.GetBinding(ctx, secondID)
|
||||||
|
if err != nil || second.RepoParam != "full" || second.Inputs["a"].Param != "p" || second.Inputs["b"].Value != "x" {
|
||||||
|
t.Fatalf("binding inputs and repo_param over the wire = %+v, %v", second, err)
|
||||||
|
}
|
||||||
missingBinding, err := ec.GetBinding(ctx, "rabsent00000000")
|
missingBinding, err := ec.GetBinding(ctx, "rabsent00000000")
|
||||||
if err != nil || missingBinding != nil {
|
if err != nil || missingBinding != nil {
|
||||||
t.Fatalf("GetBinding missing: %+v %v", missingBinding, err)
|
t.Fatalf("GetBinding missing: %+v %v", missingBinding, err)
|
||||||
@@ -431,10 +436,13 @@ func TestStateStoreConformance(t *testing.T) {
|
|||||||
if _, err := ec.ArmedBindingsForSource(ctx, "sentry"); err != nil {
|
if _, err := ec.ArmedBindingsForSource(ctx, "sentry"); err != nil {
|
||||||
t.Fatalf("ArmedBindingsForSource: %v", err)
|
t.Fatalf("ArmedBindingsForSource: %v", err)
|
||||||
}
|
}
|
||||||
bindingTask, err := c.EnqueueBindingTask(ctx, "acme", "widget", "t", "b", "implement", "", bindingID, 1)
|
bindingTask, err := c.EnqueueBindingTask(ctx, "acme", "widget", "t", "b", "implement", "", bindingID, 1, map[string]any{"src_ip": "10.0.0.1"})
|
||||||
if err != nil || bindingTask == nil {
|
if err != nil || bindingTask == nil {
|
||||||
t.Fatalf("EnqueueBindingTask: %+v %v", bindingTask, err)
|
t.Fatalf("EnqueueBindingTask: %+v %v", bindingTask, err)
|
||||||
}
|
}
|
||||||
|
if stored, err := c.TaskByID(ctx, bindingTask.ID); err != nil || stored.Inputs["src_ip"] != "10.0.0.1" {
|
||||||
|
t.Fatalf("binding task inputs over the wire = %+v, %v; want src_ip", stored, err)
|
||||||
|
}
|
||||||
if err := ec.RecordDispatch(ctx, bindingID, 1, captureID, bindingTask.ID); err != nil {
|
if err := ec.RecordDispatch(ctx, bindingID, 1, captureID, bindingTask.ID); err != nil {
|
||||||
t.Fatalf("RecordDispatch: %v", err)
|
t.Fatalf("RecordDispatch: %v", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -22,6 +22,7 @@ import (
|
|||||||
"github.com/samcharles93/archie-core/internal/domain/mapping"
|
"github.com/samcharles93/archie-core/internal/domain/mapping"
|
||||||
"github.com/samcharles93/archie-core/internal/domain/source"
|
"github.com/samcharles93/archie-core/internal/domain/source"
|
||||||
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
"github.com/samcharles93/archie-core/internal/domain/storecontract"
|
||||||
|
"github.com/samcharles93/archie-core/internal/domain/workflow/task"
|
||||||
"github.com/samcharles93/archie-core/internal/logging"
|
"github.com/samcharles93/archie-core/internal/logging"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -812,7 +813,11 @@ func (s *server) EnqueueBindingTask(ctx context.Context, r *pb.EnqueueBindingTas
|
|||||||
if s.deps.BindingTaskCreator == nil {
|
if s.deps.BindingTaskCreator == nil {
|
||||||
return nil, errBindingTaskCreatorUnavailable
|
return nil, errBindingTaskCreatorUnavailable
|
||||||
}
|
}
|
||||||
t, err := s.deps.BindingTaskCreator.EnqueueBindingTask(ctx, r.Owner, r.Repo, r.Title, r.Body, r.Workflow, r.Identity, r.BindingId, int(r.BindingVersion))
|
inputs, err := task.DecodeInputs(r.InputsJson)
|
||||||
|
if err != nil {
|
||||||
|
return nil, status.Error(codes.InvalidArgument, err.Error())
|
||||||
|
}
|
||||||
|
t, err := s.deps.BindingTaskCreator.EnqueueBindingTask(ctx, r.Owner, r.Repo, r.Title, r.Body, r.Workflow, r.Identity, r.BindingId, int(r.BindingVersion), inputs)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, s.logErr("EnqueueBindingTask", err)
|
return nil, s.logErr("EnqueueBindingTask", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -68,7 +68,8 @@ func taskProto(t *task.Task) *pb.Task { //nolint:dupl // mirror-image field-by-f
|
|||||||
Attempt: int64(t.Attempt), ParkReason: t.ParkReason, RetryCount: int64(t.RetryCount),
|
Attempt: int64(t.Attempt), ParkReason: t.ParkReason, RetryCount: int64(t.RetryCount),
|
||||||
WatchCommentId: t.WatchCommentID, Source: t.Source, Identity: t.Identity,
|
WatchCommentId: t.WatchCommentID, Source: t.Source, Identity: t.Identity,
|
||||||
BindingId: t.BindingID, BindingVersion: int64(t.BindingVersion),
|
BindingId: t.BindingID, BindingVersion: int64(t.BindingVersion),
|
||||||
CreatedAt: timestamp(t.CreatedAt), UpdatedAt: timestamp(t.UpdatedAt),
|
InputsJson: inputsJSON(t.Inputs),
|
||||||
|
CreatedAt: timestamp(t.CreatedAt), UpdatedAt: timestamp(t.UpdatedAt),
|
||||||
ReviewPayload: t.ReviewPayload,
|
ReviewPayload: t.ReviewPayload,
|
||||||
ReviewCursor: t.ReviewCursor,
|
ReviewCursor: t.ReviewCursor,
|
||||||
ParkClass: t.ParkClass,
|
ParkClass: t.ParkClass,
|
||||||
@@ -91,6 +92,7 @@ func taskValue(t *pb.Task) *task.Task { //nolint:dupl // see taskProto above
|
|||||||
Attempt: int(t.Attempt), ParkReason: t.ParkReason, RetryCount: int(t.RetryCount),
|
Attempt: int(t.Attempt), ParkReason: t.ParkReason, RetryCount: int(t.RetryCount),
|
||||||
WatchCommentID: t.WatchCommentId, Source: t.Source, Identity: t.Identity,
|
WatchCommentID: t.WatchCommentId, Source: t.Source, Identity: t.Identity,
|
||||||
BindingID: t.BindingId, BindingVersion: int(t.BindingVersion),
|
BindingID: t.BindingId, BindingVersion: int(t.BindingVersion),
|
||||||
|
Inputs: inputsValue(t.InputsJson),
|
||||||
CreatedAt: timeValue(t.CreatedAt), UpdatedAt: timeValue(t.UpdatedAt),
|
CreatedAt: timeValue(t.CreatedAt), UpdatedAt: timeValue(t.UpdatedAt),
|
||||||
ReviewPayload: t.ReviewPayload,
|
ReviewPayload: t.ReviewPayload,
|
||||||
ReviewCursor: t.ReviewCursor,
|
ReviewCursor: t.ReviewCursor,
|
||||||
@@ -102,6 +104,19 @@ func taskValue(t *pb.Task) *task.Task { //nolint:dupl // see taskProto above
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// inputsJSON and inputsValue carry task inputs in task.EncodeInputs form.
|
||||||
|
// Both ends encode with EncodeInputs, so neither direction can fail on a
|
||||||
|
// value this contract produced.
|
||||||
|
func inputsJSON(inputs map[string]any) string {
|
||||||
|
s, _ := task.EncodeInputs(inputs)
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
|
||||||
|
func inputsValue(s string) map[string]any {
|
||||||
|
inputs, _ := task.DecodeInputs(s)
|
||||||
|
return inputs
|
||||||
|
}
|
||||||
|
|
||||||
// eventDataJSON and eventDataValue convert events.Event.Data (map[string]any)
|
// eventDataJSON and eventDataValue convert events.Event.Data (map[string]any)
|
||||||
// to/from a JSON object string, mirroring the store's own on-disk column.
|
// to/from a JSON object string, mirroring the store's own on-disk column.
|
||||||
func eventDataJSON(data map[string]any) string {
|
func eventDataJSON(data map[string]any) string {
|
||||||
@@ -221,6 +236,7 @@ func bindingProto(b binding.Binding) *pb.Binding {
|
|||||||
return &pb.Binding{
|
return &pb.Binding{
|
||||||
Id: b.ID, Name: b.Name, Matcher: &pb.BindingMatcher{Source: b.Matcher.Source},
|
Id: b.ID, Name: b.Name, Matcher: &pb.BindingMatcher{Source: b.Matcher.Source},
|
||||||
MappingId: b.MappingID, Filter: b.Filter, Workflow: b.Workflow, Owner: b.Owner, Repo: b.Repo,
|
MappingId: b.MappingID, Filter: b.Filter, Workflow: b.Workflow, Owner: b.Owner, Repo: b.Repo,
|
||||||
|
RepoParam: b.RepoParam, InputsJson: bindingInputsJSON(b.Inputs),
|
||||||
Version: int64(b.Version), Status: string(b.Status),
|
Version: int64(b.Version), Status: string(b.Status),
|
||||||
CreatedAt: timestamp(b.CreatedAt), UpdatedAt: timestamp(b.UpdatedAt),
|
CreatedAt: timestamp(b.CreatedAt), UpdatedAt: timestamp(b.UpdatedAt),
|
||||||
}
|
}
|
||||||
@@ -237,11 +253,32 @@ func bindingValue(b *pb.Binding) binding.Binding {
|
|||||||
return binding.Binding{
|
return binding.Binding{
|
||||||
ID: b.Id, Name: b.Name, Matcher: binding.Matcher{Source: source},
|
ID: b.Id, Name: b.Name, Matcher: binding.Matcher{Source: source},
|
||||||
MappingID: b.MappingId, Filter: b.Filter, Workflow: b.Workflow, Owner: b.Owner, Repo: b.Repo,
|
MappingID: b.MappingId, Filter: b.Filter, Workflow: b.Workflow, Owner: b.Owner, Repo: b.Repo,
|
||||||
|
RepoParam: b.RepoParam, Inputs: bindingInputsValue(b.InputsJson),
|
||||||
Version: int(b.Version), Status: binding.Status(b.Status),
|
Version: int(b.Version), Status: binding.Status(b.Status),
|
||||||
CreatedAt: timeValue(b.CreatedAt), UpdatedAt: timeValue(b.UpdatedAt),
|
CreatedAt: timeValue(b.CreatedAt), UpdatedAt: timeValue(b.UpdatedAt),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// bindingInputsJSON and bindingInputsValue carry a binding's input
|
||||||
|
// assignments as JSON. The domain type always marshals, and the client only
|
||||||
|
// decodes what the server marshalled.
|
||||||
|
func bindingInputsJSON(inputs map[string]binding.InputSource) string {
|
||||||
|
if len(inputs) == 0 {
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
data, _ := json.Marshal(inputs)
|
||||||
|
return string(data)
|
||||||
|
}
|
||||||
|
|
||||||
|
func bindingInputsValue(s string) map[string]binding.InputSource {
|
||||||
|
if s == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var inputs map[string]binding.InputSource
|
||||||
|
_ = json.Unmarshal([]byte(s), &inputs)
|
||||||
|
return inputs
|
||||||
|
}
|
||||||
|
|
||||||
func workflowStatProto(w storecontract.WorkflowStat) *pb.WorkflowStat {
|
func workflowStatProto(w storecontract.WorkflowStat) *pb.WorkflowStat {
|
||||||
return &pb.WorkflowStat{
|
return &pb.WorkflowStat{
|
||||||
Workflow: w.Workflow, Runs: int64(w.Runs), Merged: int64(w.Merged),
|
Workflow: w.Workflow, Runs: int64(w.Runs), Merged: int64(w.Merged),
|
||||||
|
|||||||
Loaded 100 of 118 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user