* feat(workflow): post line-anchored review comments with suggested fixes Bead archie-core-q9au (docs/prds/inline-review.md). A review's findings reached the operator only as a flat list in the PR body, so the author had to map every finding back to its line by hand and apply mechanical fixes by hand. Now the line-anchored findings are also posted as inline review comments, carrying a one-click suggestion when the reviewer could state the fix. Domain (internal/domain/workflow): - ReviewFinding gains Suggestion; Validate rejects one on anything but a confirmed finding, so the agent's record_finding call is refused with something it can act on. Validate splits into validateAnchor / validateClassification / validateDisposition, which also brings it back under the cyclop limit the gate enforces. - StagePostReviewComments runs after StageOpenPR in the implement workflow. It posts only line-anchored findings (Line > 0); a whole-file finding has no line to attach to and stays in the body list, which remains the complete record. It is best-effort: the PR is open and the body already lists every finding, so a failed comment is logged, never a park -- the same reasoning as OpenPR's existing best-effort LinkBranch call. - The suggestion fence is gated on Verdict == confirmed where the fence is written, because it is a one-click apply: a wrong suggestion breaks the code and the author has to notice. Forge (internal/forge): ReviewCommentWriter + GitHub/Gitea implementations, type-asserted like PullRequestReader so a forge that cannot is skipped rather than faked (NoopForge is pinned as not satisfying it). GitHub loops one CreateComment per comment and joins the failures, so one refused anchor cannot drop the rest; Gitea carries them on a single COMMENT-state review. Two deliberate departures from the PRD's sketch, both recorded in it: - The interface is batch (CreateReviewComments), not one call per comment: Gitea's only inline shape is one submitted review, and the set then crosses NATS once instead of once per finding. - The head SHA is resolved inside each forge implementation rather than by the caller. The stage runs in archie-agent, which holds no forge credentials and has no PullRequestReader; the daemon-side forgerpc handler is the only place that does. Transport (internal/forgerpc): workflow.Forger gains the method, so the agent reaches the daemon over the new create_review_comments subject; the server type-asserts the writer and answers an incapable forge with an error the stage logs. This is the one proxied method whose absence on an image skew does not park a task. The reviewer prompt and record_finding schema ask for a suggestion on a confirmed, mechanically fixable defect only. Regenerating the wfextract symbol table -- the documented way a new exported workflow symbol reaches interpreted stage plugins -- also picked up symbols that were missing from the committed table (the Status* and Source* constants, ReviewCheck, Store, Task, and the _Store wrapper). Nil-guards were re-applied to every wrapper afterwards, as that file's header requires. Evidence: task check exit 0 (incl. 229 UI tests). Three mutations of the new stage logic each fail a test that names the behaviour -- dropping the confirmed-only gate, dropping the Line > 0 filter, and letting a posting failure escape -- so the tests discriminate rather than passing vacuously. * fix(build): drop the Dockerfile pin assertion The guard fetched golangci-lint's go.mod at image build time to fail the build when GOLANGCI_LINT_VERSION and GOFUMPT_VERSION disagreed. It only ever tripped on self-inflicted pin drift, but it added a network fetch and a new failure mode to the release build, which is not where that belongs. The pins stay correct (golangci-lint 2.13.2, whose vendored gofumpt v0.11.0 equals GOFUMPT_VERSION) and the comment above them still states the invariant, so the reasoning survives without the build-time machinery. The pin change itself reaches the agent image through the normal release path: CI rebuilds and pushes archie-agent on an archie/vX.Y.Z release, and a gateway-only release deliberately does not. * feat(workflow): add the remediate workflow for PR review remediation Implements step 5 of docs/prds/pr-review-remediation.md: a remediate workflow that resumes the task's existing worktree onto its PR branch (worktree.Manager.Resume, not Prepare's refresh), runs a builder agent against the review's actionable comments, commits and pushes to the same branch, and replies via the forge's ReplyToReview. Task.RetryCount doubles as the remediation round counter, capped by the repo's max_retries; the round-cap stage parks the task and posts one comment when it's hit. Task gains a ReviewPayload field (JSON-encoded review unit) with full store/proto/staterpc plumbing. workflow.Forger gains Comment and ReplyToReview, proxied to archie-agent via a new forgerpc RPC. Resume and its Dir lookup are added as an unexported optional Trees capability rather than widening the exported interface, which is already at golangci-lint's interfacebloat cap and is projected into wfextract's Yaegi symbol table for repository-authored stage plugins. The daemon-side reaction consumer that dispatches into this workflow (subscribing ARCHIE_REACTIONS, grouping comments by review, enforcing per-task serialization) is tracked separately in archie-core-8li9.4. * feat(messaging): extract standalone messaging service binary and process composition * feat(messaging): adapt channel frontends to consume Gateway ChatContract * feat(messaging): remove in-process channels and Router from daemon bootstrap Telegram, email and webhook now run only in the Messaging Service, which reaches this process through the Gateway's ChatContract. The daemon builds no channel and no gateway.Router, so the wiring that existed only to serve them goes with it: setupGateways, setupEmailGateway, setupWebhookGateway, setupTelegramGateway, buildTelegramRouter, makeTelegramReload and the release-announcement path helper. The seams those paths owned move to where they can still be honoured: - Channel ValidateConfig runs in archiemessaging.compose, which now returns an error, so a configured-but-invalid front-end fails the service rather than being dropped silently. - The inbound rate limiter moves to the Gateway process, which owns the only Router and therefore the only place a per-sender budget applies to every channel's turns. - The daemon's gateway readiness probe becomes a ContractProbe against the Gateway. It cannot observe a channel lifecycle it no longer runs, and a channel-count probe would have reported every daemon degraded. - telegramSetup is now chatSetup (in chat_setup.go), trimmed to the fields a chat turn runner actually reads. Telegram operator features that were wired only in the daemon (/update, release announcements, version reporting, SIGHUP reload) are not yet wired in the Messaging Service: archie-core-8cda.6.6. Channel lifecycle reporting: archie-core-8cda.6.8. The Gateway process builds no worktree manager, so review_pr is unadvertised on every channel: archie-core-8cda.6.7. * feat(messaging): re-wire the Telegram operator surface in the Messaging Service setupTelegramGateway satisfied eight telegram.Gateway seams from the daemon's own process. Each is resolved by where the fact it needs is owned, not by where it used to be built; the decision table is in docs/architecture/migration-decisions.md. Wired here: /update (built from this service's own [chat.telegram] check and install commands), the update-report relay (same sha256(bot_user)[:8] path the daemon wrote, so a pending report is found rather than written afresh), show_tool_calls, and /restart's config reload, which re-resolves the token and allowlist from this service's own file and refuses a reload that resolves to an empty token. /version is sourced from the Gateway: ChatSnapshot gains a Version field carrying the Gateway's rendered build block, read per call and bounded by the dependency timeout. cmd/archie-messaging deliberately takes no version ldflags, so it cannot report a partial upgrade as a matched one. RunningVersions and ReleaseAnnouncements stay nil. Both need a component's real running version, and the only versions this process can obtain are a different binary's stamps -- reporting them would manufacture exactly the false success the update check exists to catch, and the announcer without parseable versions is a no-op that reads as configured. archie-core-8cda.6.9 carries the contract that unblocks them. Dangerous stays nil: no composition has ever supplied a command authority. Enrich is dropped from the update service: componentInstallTypeEnricher read [nats], which this service must not decode, so the check command's own reported install type stands. * fix(workflow): guard inline review comments against head drift Two findings from the review of q9au. A comment's line numbers describe exactly one revision of the branch, but the forge resolved the pull request's head at posting time -- after the review had already read the worktree. A push in that window (a collaborator, a retry) meant the findings were anchored to whatever now occupied those lines, mislabelling instead of failing. The revision the review measured now travels with the comment set: OpenPR records it from the same worktree read it already performs (TaskContext.ReviewedHeadSHA), StagePostReviewComments threads it through Forger to forgerpc, and the forge refuses the whole set once the head it reads has moved off it -- which the best-effort stage logs, the PR body remaining the record. An unmeasured revision posts unverified rather than dropping every finding, so a worktree that cannot report one degrades instead of regressing. Suggestion was documented single-line only, because GitHub applies a suggestion fence up to its first line and a multi-line replacement would silently corrupt the file. Only the verdict was validated, so a multi-line suggestion reached the renderer and posted as that truncated fence. Validate now rejects an embedded newline, which makes record_finding refuse it with feedback the model can act on instead of the reviewer discovering the corruption later. * fix(gateway): render chat reports as lines, not one run-on paragraph /status and /tasks are line-oriented reports, but their newlines were plain "\n" -- CommonMark's soft wrap. The Telegram rich-block renderer space-joins soft-wrapped lines, by design: a Telegram paragraph renders an embedded newline as nothing at all, so a space is the only separator that survives. The reports therefore arrived as one line each: "Runtime Provider: DeepSeek Model: deepseek/deepseek-flash", and every task's title, state and park reason jammed together. The renderer now implements CommonMark's hard line break (two trailing spaces, or a trailing backslash): the line closes its own block and the next one hugs it, so a stanza keeps its lines without the spacer paragraph that separates unrelated blocks. Ordinary prose is untouched -- a model's wrapped paragraph still joins with a space. The producers mark their report lines with it. Two report fixes alongside: - A park reason is capped to its first line at 160 runes. It carries whatever the failing stage reported, which for a lint gate is the whole tool log; unbounded, one parked task pushed every other task out of the reply. The full reason is still on the task. - /status loses its "Runtime" sub-header. A section header over two fields made a four-fact report read like a document. * fix(config): let diff_cap_lines = 0 actually switch the cap off Config.DiffCapLines was a plain int, so the defaults pass could not tell an absent key from an explicit 0 and rewrote both to 400. The dashboard schema said "0 means unlimited" and StageDiffCap treated <= 0 as unlimited, but 0 never reached the step: the documented way to switch the cap off was unreachable through config. Same root cause, worse instance: configForIdentity assigned root.DiffCapLines = identity.DiffCapLines unconditionally, so an identity that simply omitted the key overwrote the shared cap with the zero value and silently disabled the safety rail for that identity. DiffCapLines is now *int on both Config and IdentityConfig -- nil is absent, 0 is no cap -- read through Config.DiffCap(). Defaults fill only nil, and an identity overrides only when it set the key. TaskConfig keeps a resolved int, so the daemon/agent wire format is unchanged and a version-skewed agent cannot misread a null. The park reason changes too. It told the operator to "approve manually", but a parked task's actions are retry, abandon and reject (internal/taskstate): there is no approve path out of diff-cap, so it sent them looking for a button that does not exist. It now names the two things that work. * feat(config): make the daemon's readiness dependency timeout configurable The daemon's gateway probe bounded its call with a hardcoded 5s constant, while every other NewContractProbe call site (archie-ui, archie-messaging) passes the service's configured DependencyTimeout. It was the only magic number among them. [health].dependency_timeout carries it, defaulting to the same 5s the extracted services' -dependency-timeout flag defaults to, so every process bounds a probe into a dependency the same way out of the box. It is not added to the dashboard config schema or the reloadable set: [health] appears in neither today, health.listen included, and a probe assembled at boot would not observe a reload anyway.
672 lines
25 KiB
Go
672 lines
25 KiB
Go
package workflow
|
|
|
|
import (
|
|
"bytes"
|
|
"context"
|
|
"errors"
|
|
"log/slog"
|
|
"os"
|
|
"path/filepath"
|
|
"strconv"
|
|
"strings"
|
|
"testing"
|
|
|
|
"github.com/samcharles93/archie-core/internal/agentexec"
|
|
"github.com/samcharles93/archie-core/internal/config"
|
|
"github.com/samcharles93/archie-core/internal/events"
|
|
"github.com/samcharles93/archie-core/internal/forge"
|
|
"github.com/samcharles93/archie-core/internal/worktree"
|
|
)
|
|
|
|
func TestStageBaselineGateAccountsForRepairAgentUsage(t *testing.T) {
|
|
dir := t.TempDir()
|
|
runGit(t, dir, "init")
|
|
script := filepath.Join(dir, "gate.sh")
|
|
if err := os.WriteFile(script, []byte("#!/bin/sh\nexit 1\n"), 0o755); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
runner := agentRunnerFunc(func(context.Context, string, agentexec.Request, agentexec.ToolCallReporter) (agentexec.Result, error) {
|
|
return agentexec.Result{
|
|
Version: agentexec.ProtocolVersion, Status: agentexec.StatusPassed,
|
|
TokensUsed: 120, Iterations: 3,
|
|
Usage: agentexec.Usage{PromptTokens: 100, CompletionTokens: 20, TotalTokens: 120, CachedTokens: 80},
|
|
}, nil
|
|
})
|
|
task := &Task{ID: 1, Attempt: 1, Owner: "o", Repo: "r"}
|
|
tc := &TaskContext{
|
|
Task: task, Repo: config.Repo{Owner: "o", Name: "r", Gate: [][]string{{"sh", script}}},
|
|
Cfg: config.Config{Models: map[string]string{"builder": "provider/model"}},
|
|
Agent: runner, Trees: &worktree.Manager{BotUser: "archie", BotEmail: "archie@example.com"},
|
|
Dir: dir, Log: slog.New(slog.DiscardHandler),
|
|
}
|
|
if err := StageBaselineGate().Run(t.Context(), tc); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if task.TokensUsed != 120 || task.Iterations != 3 {
|
|
t.Fatalf("baseline usage = %d tokens/%d iterations, want 120/3", task.TokensUsed, task.Iterations)
|
|
}
|
|
if tc.RunUsage.PromptTokens != 100 || tc.RunUsage.CompletionTokens != 20 || tc.RunUsage.CachedTokens != 80 {
|
|
t.Fatalf("RunUsage = %+v, want prompt=100 completion=20 cached=80", tc.RunUsage)
|
|
}
|
|
}
|
|
|
|
// TestImplementPRBodyShowsFreshVsCachedTokens pins problem 1: a PR body
|
|
// built from a heavily-cached run must surface how much of the reported
|
|
// prompt-token sum was a cache hit, not read as the full-price total.
|
|
func TestImplementPRBodyShowsFreshVsCachedTokens(t *testing.T) {
|
|
tc := &TaskContext{
|
|
Task: &Task{Iterations: 14, TokensUsed: 405004},
|
|
BuildSummary: "did the thing",
|
|
RunUsage: agentexec.Usage{
|
|
PromptTokens: 400021, CompletionTokens: 4983, CachedTokens: 360018,
|
|
},
|
|
}
|
|
body := implementPRBody(tc)
|
|
if !strings.Contains(body, "405004 tokens") {
|
|
t.Fatalf("PR body missing the raw total: %s", body)
|
|
}
|
|
if !strings.Contains(body, "cached") {
|
|
t.Fatalf("PR body does not disclose cache hits: %s", body)
|
|
}
|
|
// fresh = prompt - cached + completion = 400021 - 360018 + 4983 = 44986
|
|
if !strings.Contains(body, "44986 fresh") {
|
|
t.Fatalf("PR body missing the fresh/net figure: %s", body)
|
|
}
|
|
}
|
|
|
|
// TestImplementPRBodyFallsBackWithoutUsageBreakdown guards the degrade
|
|
// path: a run with no populated Usage (e.g. no agent stage ran) must still
|
|
// produce a sane PR body rather than a "0 fresh + 0 cached" line implying
|
|
// billed usage was known and zero.
|
|
func TestImplementPRBodyFallsBackWithoutUsageBreakdown(t *testing.T) {
|
|
tc := &TaskContext{
|
|
Task: &Task{Iterations: 1, TokensUsed: 500},
|
|
BuildSummary: "did the thing",
|
|
}
|
|
body := implementPRBody(tc)
|
|
if !strings.Contains(body, "500 tokens") {
|
|
t.Fatalf("PR body missing the total: %s", body)
|
|
}
|
|
if strings.Contains(body, "fresh") || strings.Contains(body, "cached") {
|
|
t.Fatalf("PR body fabricated a breakdown with no usage data: %s", body)
|
|
}
|
|
}
|
|
|
|
// TestExtractFailingGateOutputDropsPassingPackages pins problem 2: the
|
|
// baseline-fix mission must not re-inject "ok" lines for packages that
|
|
// already pass -- those dominate a large repo's `go test ./...` output and
|
|
// squeeze the actual failure detail out of clip's size bound on later
|
|
// retries.
|
|
func TestExtractFailingGateOutputDropsPassingPackages(t *testing.T) {
|
|
out := "ok \tgithub.com/example/pkg/a\t0.004s\n" +
|
|
"ok \tgithub.com/example/pkg/b\t0.012s\n" +
|
|
"--- FAIL: TestSomething (0.00s)\n" +
|
|
" something_test.go:12: expected 1, got 2\n" +
|
|
"FAIL\tgithub.com/example/pkg/c\t0.010s\n" +
|
|
"ok \tgithub.com/example/pkg/d\t0.002s\n"
|
|
|
|
got := extractFailingGateOutput(out)
|
|
|
|
if strings.Contains(got, "pkg/a") || strings.Contains(got, "pkg/b") || strings.Contains(got, "pkg/d") {
|
|
t.Fatalf("passing-package lines survived extraction: %s", got)
|
|
}
|
|
if !strings.Contains(got, "--- FAIL: TestSomething") || !strings.Contains(got, "pkg/c") {
|
|
t.Fatalf("failure detail was dropped: %s", got)
|
|
}
|
|
if len(got) >= len(out) {
|
|
t.Fatalf("extraction did not reduce output size: got %d bytes, want < %d", len(got), len(out))
|
|
}
|
|
}
|
|
|
|
// TestExtractFailingGateOutputDegradesGracefullyForUnrecognizedFormat
|
|
// guards against silently dropping real failure information for a gate
|
|
// command whose output isn't shaped like `go test` output (a non-Go gate,
|
|
// or an unexpected format) -- the full output must be preserved.
|
|
func TestExtractFailingGateOutputDegradesGracefullyForUnrecognizedFormat(t *testing.T) {
|
|
out := "lint: 3 problems found\nfile.go:10: unused variable\nfile.go:22: missing return\n"
|
|
if got := extractFailingGateOutput(out); got != out {
|
|
t.Fatalf("extraction altered unrecognized output:\ngot: %q\nwant: %q", got, out)
|
|
}
|
|
}
|
|
|
|
// TestStageBaselineGateMissionOmitsPassingPackageOutput is the integration
|
|
// pin: a real StageBaselineGate run with go-test-shaped failure output must
|
|
// build a mission that omits the passing-package noise.
|
|
func TestStageBaselineGateMissionOmitsPassingPackageOutput(t *testing.T) {
|
|
dir := t.TempDir()
|
|
script := filepath.Join(dir, "gate.sh")
|
|
body := "#!/bin/sh\n" +
|
|
"echo 'ok \tgithub.com/example/pkg/quietpassingpkg\t0.004s'\n" +
|
|
"echo '--- FAIL: TestBroken (0.00s)'\n" +
|
|
"echo ' broken_test.go:5: boom'\n" +
|
|
"echo 'FAIL\tgithub.com/example/pkg/broken\t0.010s'\n" +
|
|
"exit 1\n"
|
|
if err := os.WriteFile(script, []byte(body), 0o755); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
|
|
var gotMission string
|
|
runner := agentRunnerFunc(func(_ context.Context, _ string, req agentexec.Request, _ agentexec.ToolCallReporter) (agentexec.Result, error) {
|
|
gotMission = req.Mission
|
|
return agentexec.Result{Version: agentexec.ProtocolVersion, Status: agentexec.StatusPassed}, nil
|
|
})
|
|
tc := &TaskContext{
|
|
Task: &Task{ID: 1, Owner: "o", Repo: "r"},
|
|
Repo: config.Repo{Owner: "o", Name: "r", Gate: [][]string{{"sh", script}}},
|
|
Cfg: config.Config{Models: map[string]string{"builder": "provider/model"}},
|
|
Agent: runner, Trees: &worktree.Manager{BotUser: "archie", BotEmail: "archie@example.com"},
|
|
Dir: dir, Log: slog.New(slog.DiscardHandler),
|
|
}
|
|
if err := StageBaselineGate().Run(t.Context(), tc); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if strings.Contains(gotMission, "quietpassingpkg") {
|
|
t.Fatalf("mission still contains a passing package's output: %s", gotMission)
|
|
}
|
|
if !strings.Contains(gotMission, "TestBroken") || !strings.Contains(gotMission, "pkg/broken") {
|
|
t.Fatalf("mission lost the actual failure detail: %s", gotMission)
|
|
}
|
|
}
|
|
|
|
// TestStageBaselineGateSetsBaselineFixedWhenItCommits is the companion
|
|
// regression case for archie-core-95dj: when the repair agent's changes
|
|
// are actually committed, tc.BaselineFixed must be set so a later
|
|
// no-op build stage cannot cause StageCommitPush to discard the fix.
|
|
func TestStageBaselineGateSetsBaselineFixedWhenItCommits(t *testing.T) {
|
|
dir := t.TempDir()
|
|
runGit(t, dir, "init")
|
|
// Real clones get commit.gpgsign=false via Manager.setIdentity; a bare
|
|
// `git init` here does not, so it inherits the host's global config --
|
|
// see worktree_test.go's disableSigning for the same fixture gap.
|
|
runGit(t, dir, "config", "commit.gpgsign", "false")
|
|
script := filepath.Join(dir, "gate.sh")
|
|
if err := os.WriteFile(script, []byte("#!/bin/sh\nexit 1\n"), 0o755); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
runner := agentRunnerFunc(func(_ context.Context, runDir string, _ agentexec.Request, _ agentexec.ToolCallReporter) (agentexec.Result, error) {
|
|
if err := os.WriteFile(filepath.Join(runDir, "fixed.txt"), []byte("fix\n"), 0o644); err != nil {
|
|
return agentexec.Result{}, err
|
|
}
|
|
return agentexec.Result{Version: agentexec.ProtocolVersion, Status: agentexec.StatusPassed}, nil
|
|
})
|
|
task := &Task{ID: 1, Attempt: 1, Owner: "o", Repo: "r"}
|
|
tc := &TaskContext{
|
|
Task: task, Repo: config.Repo{Owner: "o", Name: "r", Gate: [][]string{{"sh", script}}},
|
|
Cfg: config.Config{Models: map[string]string{"builder": "provider/model"}},
|
|
Agent: runner, Trees: &worktree.Manager{BotUser: "archie", BotEmail: "archie@example.com"},
|
|
Dir: dir, Log: slog.New(slog.DiscardHandler),
|
|
}
|
|
if err := StageBaselineGate().Run(t.Context(), tc); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if !tc.BaselineFixed {
|
|
t.Fatal("BaselineFixed = false, want true after a real committed fix")
|
|
}
|
|
}
|
|
|
|
func TestStageBaselineGateWarningIncludesFailureTail(t *testing.T) {
|
|
dir := t.TempDir()
|
|
script := filepath.Join(dir, "gate.sh")
|
|
// Pad past baselineWarnLogBytes (2048) so clipTail short-circuit does
|
|
// NOT save head-clipping from being detected. The marker near the END
|
|
// would otherwise be inside the window either way; a marker past the
|
|
// window asserts the direction.
|
|
body := "echo headmarker; head -c 5000 /dev/zero | tr '\\0' x; echo; echo tailmarker; exit 1\n"
|
|
if err := os.WriteFile(script, []byte("#!/bin/sh\n"+body), 0o755); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
var logs bytes.Buffer
|
|
tc := &TaskContext{
|
|
Task: &Task{ID: 1, Owner: "o", Repo: "r"},
|
|
Repo: config.Repo{Gate: [][]string{{"sh", script}}},
|
|
Agent: alwaysFailingRunner,
|
|
Dir: dir,
|
|
Log: slog.New(slog.NewJSONHandler(&logs, nil)),
|
|
}
|
|
|
|
_ = StageBaselineGate().Run(t.Context(), tc)
|
|
logsStr := logs.String()
|
|
if !strings.Contains(logsStr, "tailmarker") {
|
|
t.Fatalf("baseline warning omitted the useful failure tail: %s", logsStr)
|
|
}
|
|
if strings.Contains(logsStr, "headmarker") {
|
|
t.Fatalf("baseline warning included the head marker -- clip is retaining the head instead of the tail")
|
|
}
|
|
}
|
|
|
|
// TestStageBaselineGateAgentRunErrorSkipsAgentFinish pins the ordering
|
|
// inside StageBaselineGate: when the baseline-fix agent returns an error,
|
|
// the stage must surface that error WITHOUT first emitting an
|
|
// agent_finish event whose status/stop_reason/tokens are all zero. AgentStage
|
|
// (internal/domain/workflow/agent.go) bails on every error path before its
|
|
// own agent_finish emit, so the baseline-fix path must match.
|
|
func TestStageBaselineGateAgentRunErrorSkipsAgentFinish(t *testing.T) {
|
|
dir := t.TempDir()
|
|
script := filepath.Join(dir, "gate.sh")
|
|
if err := os.WriteFile(script, []byte("#!/bin/sh\nexit 1\n"), 0o755); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
bus := events.NewBus()
|
|
sub := bus.Subscribe(8)
|
|
t.Cleanup(sub.Close)
|
|
runner := agentRunnerFunc(func(context.Context, string, agentexec.Request, agentexec.ToolCallReporter) (agentexec.Result, error) {
|
|
return agentexec.Result{}, errors.New("simulated transport failure")
|
|
})
|
|
task := &Task{ID: 1, Owner: "o", Repo: "r"}
|
|
tc := &TaskContext{
|
|
Task: task,
|
|
Repo: config.Repo{Owner: "o", Name: "r", Gate: [][]string{{"sh", script}}},
|
|
Cfg: config.Config{Models: map[string]string{"builder": "provider/model"}},
|
|
Agent: runner,
|
|
Bus: bus,
|
|
Dir: dir,
|
|
Log: slog.New(slog.DiscardHandler),
|
|
}
|
|
if err := StageBaselineGate().Run(t.Context(), tc); err == nil {
|
|
t.Fatal("expected StageBaselineGate to surface the agent error")
|
|
}
|
|
for {
|
|
select {
|
|
case ev := <-sub.C:
|
|
if ev.Kind == "agent_finish" {
|
|
t.Fatalf("baseline-fix errored but agent_finish was still published: %+v", ev)
|
|
}
|
|
default:
|
|
return
|
|
}
|
|
}
|
|
}
|
|
|
|
// fakeForge implements forge.Forge for TestStageCommitPushCloses.
|
|
type fakeForge struct {
|
|
closed int
|
|
commented []string
|
|
calls []string
|
|
linkErr error
|
|
prNumber int
|
|
|
|
reviewCalls int
|
|
reviewComments []ReviewComment
|
|
reviewOwner string
|
|
reviewRepo string
|
|
reviewNumber int
|
|
reviewHeadSHA string
|
|
reviewErr error
|
|
|
|
replyCommentID int64
|
|
replyBody string
|
|
replyErr error
|
|
}
|
|
|
|
func (f *fakeForge) CloseIssue(ctx context.Context, owner, repo string, number int, comment string) error {
|
|
f.closed++
|
|
return nil
|
|
}
|
|
|
|
func (f *fakeForge) Comment(ctx context.Context, owner, repo string, number int, body string) (int64, error) {
|
|
f.commented = append(f.commented, body)
|
|
return 0, nil
|
|
}
|
|
|
|
func (f *fakeForge) ReplyToReview(ctx context.Context, owner, repo string, number int, commentID int64, body string) error {
|
|
f.replyCommentID, f.replyBody = commentID, body
|
|
return f.replyErr
|
|
}
|
|
|
|
// Remaining forge.Forge stubs -- only CloseIssue and Comment are used by StageCommitPush.
|
|
func (f *fakeForge) CreateIssue(ctx context.Context, owner, repo, title, body string, labels []string) (int, error) {
|
|
return 0, nil
|
|
}
|
|
func (f *fakeForge) Name() string { return "fake" }
|
|
func (f *fakeForge) AcceptInvitations(ctx context.Context) error { return nil }
|
|
func (f *fakeForge) AssignedIssues(ctx context.Context, owner, repo, user string) ([]forge.Issue, error) {
|
|
return nil, nil
|
|
}
|
|
|
|
func (f *fakeForge) IssuesWithLabel(ctx context.Context, owner, repo, label string) ([]forge.Issue, error) {
|
|
return nil, nil
|
|
}
|
|
|
|
func (f *fakeForge) CreatePR(ctx context.Context, owner, repo, title, head, base, body string) (int, error) {
|
|
f.calls = append(f.calls, "pr:"+head)
|
|
// A real forge never answers 0, and returning it let a test pass while
|
|
// the PR number was never recorded.
|
|
f.prNumber++
|
|
return f.prNumber, nil
|
|
}
|
|
|
|
func (f *fakeForge) PRState(ctx context.Context, owner, repo string, number int) (string, error) {
|
|
return "", nil
|
|
}
|
|
|
|
func (f *fakeForge) React(ctx context.Context, owner, repo string, number int, reaction string) error {
|
|
return nil
|
|
}
|
|
|
|
func (f *fakeForge) SetStateLabel(ctx context.Context, owner, repo string, number int, label string, known []string) {
|
|
}
|
|
|
|
func (f *fakeForge) LinkBranch(ctx context.Context, owner, repo string, number int, branch string) error {
|
|
f.calls = append(f.calls, "link:"+branch)
|
|
return f.linkErr
|
|
}
|
|
|
|
func (f *fakeForge) CreateReviewComments(ctx context.Context, owner, repo string, number int, reviewedHeadSHA string, comments []ReviewComment) error {
|
|
f.reviewCalls++
|
|
f.reviewOwner, f.reviewRepo, f.reviewNumber = owner, repo, number
|
|
f.reviewHeadSHA = reviewedHeadSHA
|
|
f.reviewComments = append(f.reviewComments, comments...)
|
|
return f.reviewErr
|
|
}
|
|
func (f *fakeForge) VerifyPush(ctx context.Context, owner, repo string) error { return nil }
|
|
|
|
// TestStageCommitPushClosesIssueWhenBuildNoChanges verifies that
|
|
// StageCommitPush closes the issue with a comment when BuildNoChanges
|
|
// is true, instead of trying to commit and failing.
|
|
func TestStageCommitPushClosesIssueWhenBuildNoChanges(t *testing.T) {
|
|
f := &fakeForge{}
|
|
|
|
stage := StageCommitPush(func(tc *TaskContext) string {
|
|
return "commit msg"
|
|
})
|
|
|
|
tc := &TaskContext{
|
|
Forge: f,
|
|
BuildSummary: "already fixed",
|
|
BuildNoChanges: true,
|
|
Dir: "/tmp/test",
|
|
Branch: "archie/issue-1",
|
|
Task: &Task{ID: 1, Owner: "o", Repo: "r", IssueNumber: 1},
|
|
}
|
|
|
|
err := stage.Run(context.Background(), tc)
|
|
if err != nil {
|
|
t.Errorf("unexpected error: %v", err)
|
|
}
|
|
|
|
if f.closed != 1 {
|
|
t.Errorf("expected CloseIssue to be called once, got %d", f.closed)
|
|
} else {
|
|
t.Log("CloseIssue called (no changes needed)")
|
|
}
|
|
if tc.Outcome.Status != StatusMerged {
|
|
t.Errorf("expected Outcome=Merged, got %s", tc.Outcome.Status)
|
|
} else {
|
|
t.Log("Outcome set to Merged (workflow stops)")
|
|
}
|
|
|
|
if len(f.commented) != 0 {
|
|
t.Errorf("expected no forge comments, got %d", len(f.commented))
|
|
}
|
|
}
|
|
|
|
func TestOpenPRLinksSourceBranchBeforeCreatingPR(t *testing.T) {
|
|
f := &fakeForge{}
|
|
tc := &TaskContext{
|
|
Forge: f,
|
|
Task: &Task{ID: 1, Owner: "acme", Repo: "widget", IssueNumber: 42, Title: "Fix bug"},
|
|
Repo: config.Repo{Owner: "acme", Name: "widget", Base: "main"},
|
|
Branch: "fix/42-bug",
|
|
Log: slog.New(slog.DiscardHandler),
|
|
}
|
|
if err := OpenPR(context.Background(), tc, "summary"); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
if len(f.calls) != 2 || f.calls[0] != "link:fix/42-bug" || f.calls[1] != "pr:fix/42-bug" {
|
|
t.Fatalf("forge calls = %v, want [link:fix/42-bug pr:fix/42-bug]", f.calls)
|
|
}
|
|
}
|
|
|
|
func TestStageCommitPushDoesNotUseSyntheticIssueForChatNoOp(t *testing.T) {
|
|
f := &fakeForge{}
|
|
stage := StageCommitPush(func(*TaskContext) string { return "commit msg" })
|
|
tc := &TaskContext{
|
|
Forge: f,
|
|
BuildSummary: "already fixed",
|
|
BuildNoChanges: true,
|
|
Task: &Task{
|
|
ID: 1,
|
|
Owner: "o",
|
|
Repo: "r",
|
|
IssueNumber: 999_001,
|
|
Source: SourceChat,
|
|
},
|
|
}
|
|
|
|
if err := stage.Run(context.Background(), tc); err != nil {
|
|
t.Fatalf("StageCommitPush.Run(): %v", err)
|
|
}
|
|
if f.closed != 0 || len(f.commented) != 0 {
|
|
t.Fatalf("chat no-op used synthetic issue: close=%d comments=%d", f.closed, len(f.commented))
|
|
}
|
|
if tc.Outcome.Status != StatusMerged {
|
|
t.Errorf("Outcome.Status = %q, want %q", tc.Outcome.Status, StatusMerged)
|
|
}
|
|
}
|
|
|
|
// fakeTrees implements Trees, tracking calls without touching a real
|
|
// worktree. commitAllChanged controls what CommitAll reports.
|
|
//
|
|
// It deliberately does NOT implement the unexported changeStatsReader
|
|
// capability: every stage test that drives a fake tree therefore also covers
|
|
// the path where a capture cannot be taken at all.
|
|
type fakeTrees struct {
|
|
commitAllChanged bool
|
|
pushed bool
|
|
pushBranch string
|
|
pushErr error
|
|
|
|
dir string
|
|
// changedLines is what ChangedLines reports, for StageDiffCap.
|
|
changedLines int
|
|
|
|
resumed bool
|
|
resumeDir string
|
|
resumeBranch string
|
|
resumeErr error
|
|
}
|
|
|
|
func (f *fakeTrees) Prepare(context.Context, string, string, string, int, string, string, string) (string, string, error) {
|
|
return "", "", nil
|
|
}
|
|
|
|
func (f *fakeTrees) CommitAll(context.Context, string, string) (bool, error) {
|
|
return f.commitAllChanged, nil
|
|
}
|
|
|
|
func (f *fakeTrees) Push(_ context.Context, _, branch string) error {
|
|
if f.pushErr != nil {
|
|
return f.pushErr
|
|
}
|
|
f.pushed = true
|
|
f.pushBranch = branch
|
|
return nil
|
|
}
|
|
|
|
func (f *fakeTrees) Diff(context.Context, string, string) (string, error) { return "", nil }
|
|
|
|
func (f *fakeTrees) ChangedFiles(context.Context, string, string) ([]string, error) { return nil, nil }
|
|
|
|
func (f *fakeTrees) ChangedLines(context.Context, string, string) (int, error) {
|
|
return f.changedLines, nil
|
|
}
|
|
|
|
func (f *fakeTrees) Snapshot(context.Context, string, string) error { return nil }
|
|
|
|
func (f *fakeTrees) Resume(_ context.Context, dir, branch string) error {
|
|
f.resumed, f.resumeDir, f.resumeBranch = true, dir, branch
|
|
return f.resumeErr
|
|
}
|
|
|
|
func (f *fakeTrees) Dir(string, string, int) string { return f.dir }
|
|
|
|
// TestStageCommitPushPushesBaselineFixEvenWithNothingNewToCommit is the
|
|
// regression case for archie-core-95dj: StageBaselineGate already
|
|
// committed a real, gate-verified fix for a pre-existing failure, and the
|
|
// build stage found nothing further to do. The worktree has no *new*
|
|
// uncommitted changes (CommitAll correctly reports changed=false), but
|
|
// the baseline-fix commit already sitting on the branch must still reach
|
|
// a PR -- not be silently discarded as "no changes required".
|
|
func TestStageCommitPushPushesBaselineFixEvenWithNothingNewToCommit(t *testing.T) {
|
|
trees := &fakeTrees{commitAllChanged: false}
|
|
stage := StageCommitPush(func(*TaskContext) string { return "commit msg" })
|
|
tc := &TaskContext{
|
|
Trees: trees,
|
|
BaselineFixed: true,
|
|
Dir: "/tmp/test",
|
|
Branch: "archie/issue-1",
|
|
Task: &Task{ID: 1, Owner: "o", Repo: "r", IssueNumber: 1},
|
|
Log: slog.New(slog.DiscardHandler),
|
|
}
|
|
|
|
if err := stage.Run(context.Background(), tc); err != nil {
|
|
t.Fatalf("StageCommitPush.Run() = %v, want nil", err)
|
|
}
|
|
if !trees.pushed {
|
|
t.Fatal("Push was not called; the baseline-fix commit would be discarded")
|
|
}
|
|
if trees.pushBranch != "archie/issue-1" {
|
|
t.Errorf("pushed branch = %q, want %q", trees.pushBranch, "archie/issue-1")
|
|
}
|
|
}
|
|
|
|
// TestStageCommitPushStillErrorsOnEmptyTreeWithoutBaselineFix confirms the
|
|
// existing "nothing to commit" guard survives: only a real baseline-fix
|
|
// commit excuses an empty tree, not a plain no-op build.
|
|
func TestStageCommitPushStillErrorsOnEmptyTreeWithoutBaselineFix(t *testing.T) {
|
|
trees := &fakeTrees{commitAllChanged: false}
|
|
stage := StageCommitPush(func(*TaskContext) string { return "commit msg" })
|
|
tc := &TaskContext{
|
|
Trees: trees,
|
|
Dir: "/tmp/test",
|
|
Branch: "archie/issue-1",
|
|
Task: &Task{ID: 1, Owner: "o", Repo: "r", IssueNumber: 1},
|
|
Log: slog.New(slog.DiscardHandler),
|
|
}
|
|
|
|
if err := stage.Run(context.Background(), tc); err == nil {
|
|
t.Fatal("StageCommitPush.Run() = nil, want an error for an empty tree with no baseline fix")
|
|
}
|
|
if trees.pushed {
|
|
t.Fatal("Push was called despite nothing to commit and no baseline fix")
|
|
}
|
|
}
|
|
|
|
// alwaysFailingRunner reports the gate as unfixable, so StageBaselineGate
|
|
// falls through to its terminal error without needing a real agent loop.
|
|
var alwaysFailingRunner = agentRunnerFunc(func(context.Context, string, agentexec.Request, agentexec.ToolCallReporter) (agentexec.Result, error) {
|
|
return agentexec.Result{Status: "failed"}, nil
|
|
})
|
|
|
|
// TestStageBaselineGateParkErrorCarriesGateOutput guards against the gate's
|
|
// real failure output being thrown away. StageBaselineGate used to build the
|
|
// park error from res.Status alone -- the actual compiler/test error was
|
|
// never in it at all, so a park reason like "go build ./... fails ...
|
|
// (status: failed)" gave no way to see why. It also guards the direction and
|
|
// size of the bound: baselineParkOutputBytes keeps the *tail* of the output
|
|
// (where a test runner's failure detail lives, after pages of "ok" lines),
|
|
// not the head, and must actually truncate rather than always including
|
|
// everything regardless of the constant's value.
|
|
func TestStageBaselineGateParkErrorCarriesGateOutput(t *testing.T) {
|
|
tests := []struct {
|
|
name string
|
|
script string // shell body; markers must appear only in output, never in argv
|
|
wantContains []string
|
|
wantNotContain []string
|
|
}{
|
|
{
|
|
name: "short output is included in full",
|
|
script: "echo shortmarker",
|
|
wantContains: []string{"shortmarker"},
|
|
},
|
|
{
|
|
name: "long output is bounded to the tail",
|
|
// filler comfortably exceeds baselineParkOutputBytes so the
|
|
// head marker falls outside the kept window and the tail
|
|
// marker falls inside it.
|
|
script: "echo headmarker; " +
|
|
"head -c " + strconv.Itoa(baselineParkOutputBytes*2) + " /dev/zero | tr '\\0' 'y'; " +
|
|
"echo; echo tailmarker",
|
|
wantContains: []string{"tailmarker"},
|
|
wantNotContain: []string{"headmarker"},
|
|
},
|
|
}
|
|
|
|
for _, tt := range tests {
|
|
t.Run(tt.name, func(t *testing.T) {
|
|
dir := t.TempDir()
|
|
script := filepath.Join(dir, "gate.sh")
|
|
if err := os.WriteFile(script, []byte("#!/bin/sh\n"+tt.script+"\nexit 1\n"), 0o755); err != nil {
|
|
t.Fatal(err)
|
|
}
|
|
|
|
tc := &TaskContext{
|
|
Task: &Task{ID: 1, Owner: "o", Repo: "r"},
|
|
Repo: config.Repo{Gate: [][]string{{"sh", script}}},
|
|
Cfg: config.Config{},
|
|
Agent: alwaysFailingRunner,
|
|
Dir: dir,
|
|
Log: slog.New(slog.DiscardHandler),
|
|
}
|
|
|
|
err := StageBaselineGate().Run(context.Background(), tc)
|
|
if err == nil {
|
|
t.Fatal("expected an error when the baseline gate is unfixable")
|
|
}
|
|
for _, want := range tt.wantContains {
|
|
if !strings.Contains(err.Error(), want) {
|
|
t.Errorf("park error missing %q: %v", want, err)
|
|
}
|
|
}
|
|
for _, notWant := range tt.wantNotContain {
|
|
if strings.Contains(err.Error(), notWant) {
|
|
t.Errorf("park error contains %q, want truncated away: %v", notWant, err)
|
|
}
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// LinkBranch is cosmetic: it puts the branch in the issue's sidebar on Gitea
|
|
// and does nothing at all on GitHub. Failing the stage on it meant the pull
|
|
// request -- the entire point of the run -- was never opened, and the task
|
|
// parked. Worse, a retry re-links the same branch, which Gitea answers with
|
|
// a conflict, so every retry parked again.
|
|
func TestOpenPRSurvivesALinkBranchFailure(t *testing.T) {
|
|
tests := []struct {
|
|
name string
|
|
linkErr error
|
|
}{
|
|
{name: "already linked", linkErr: errors.New("409 Conflict: branch already linked")},
|
|
{name: "endpoint unavailable", linkErr: errors.New("404 Not Found")},
|
|
}
|
|
|
|
for _, tc := range tests {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
f := &fakeForge{linkErr: tc.linkErr}
|
|
taskCtx := &TaskContext{
|
|
Forge: f,
|
|
Task: &Task{ID: 1, Owner: "acme", Repo: "widget", IssueNumber: 42, Title: "Fix bug"},
|
|
Repo: config.Repo{Owner: "acme", Name: "widget", Base: "main"},
|
|
Branch: "fix/42-bug",
|
|
Log: slog.New(slog.DiscardHandler),
|
|
}
|
|
|
|
if err := OpenPR(context.Background(), taskCtx, "summary"); err != nil {
|
|
t.Fatalf("OpenPR: %v\nthe PR is the load-bearing step; cosmetic "+
|
|
"linkage must not stop it", err)
|
|
}
|
|
if taskCtx.Outcome.Status != StatusPROpen {
|
|
t.Errorf("Outcome.Status = %q, want %q", taskCtx.Outcome.Status, StatusPROpen)
|
|
}
|
|
if taskCtx.Task.PRNumber == 0 {
|
|
t.Error("PRNumber is 0: the pull request was never opened")
|
|
}
|
|
// The link is still attempted -- it is useful when it works.
|
|
if len(f.calls) != 2 || f.calls[0] != "link:fix/42-bug" {
|
|
t.Errorf("forge calls = %v, want the link attempted before the PR", f.calls)
|
|
}
|
|
})
|
|
}
|
|
}
|