canary: tie the migration comparison test to the real git adapter and the rollback hold #1793

Open
opened 2026-11-07 16:11:19 +00:00 by sam · 0 comments
Owner

Follow-up to the review of a6a0c7c1 ("fix(archied): say why the canary can't
compare migrations"), which split the three-way exit classification out of
migrated() and put git's stderr in the hold reason. Two P2 findings from that
review are recorded here rather than fixed in the change.

  1. The fake does not mirror the adapter's error contract.
    internal/app/archied/release_canary_test.go stubs runGitDiff with
    return tt.code, tt.stderr, nil for every class, but the production adapter
    returns a non-nil *exec.ExitError for every non-zero exit
    (release_canary.go:547-548). A change that treated a non-nil run error as
    "could not tell" before checking exit 1 would pass the table while reporting a
    real schema change as undetermined.

  2. Nothing ties the classification to the safety guard.
    Only migrated() is called; rollBackCanary's
    if err != nil { return holdCanary(...) } branch (release_canary.go:511-513)
    is never executed, so the property that matters — an undetermined comparison
    holds the canary instead of pinning it back to stable — is asserted by
    inspection only.

Done when a test drives the real adapter with a fake git executable on PATH
(no repository needed) for exit 0, exit 1 and a failing git, and asserts the
third case pins the failed release and never installs stable.

Follow-up to the review of a6a0c7c1 ("fix(archied): say why the canary can't compare migrations"), which split the three-way exit classification out of `migrated()` and put git's stderr in the hold reason. Two P2 findings from that review are recorded here rather than fixed in the change. 1. **The fake does not mirror the adapter's error contract.** `internal/app/archied/release_canary_test.go` stubs `runGitDiff` with `return tt.code, tt.stderr, nil` for every class, but the production adapter returns a non-nil `*exec.ExitError` for every non-zero exit (`release_canary.go:547-548`). A change that treated a non-nil run error as "could not tell" before checking exit 1 would pass the table while reporting a real schema change as undetermined. 2. **Nothing ties the classification to the safety guard.** Only `migrated()` is called; `rollBackCanary`'s `if err != nil { return holdCanary(...) }` branch (`release_canary.go:511-513`) is never executed, so the property that matters — an undetermined comparison holds the canary instead of pinning it back to stable — is asserted by inspection only. Done when a test drives the real adapter with a fake `git` executable on PATH (no repository needed) for exit 0, exit 1 and a failing git, and asserts the third case pins the failed release and never installs stable.
sam added the mediumtask labels 2026-11-14 14:45:42 +00:00
sam added the deferred label 2026-11-15 17:26:12 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sam/archie#1793