A daemon restart sends running tasks back to their first step #1841

Closed
opened 2026-11-09 03:27:19 +00:00 by sam · 1 comment
Owner

Each daemon restart re-queues running tasks from scratch (one task was re-queued 33 times), paying for the same model work again. Done when a re-queued task resumes from its last completed step, and re-queues after restarts are capped before the task parks with a clear reason.

Each daemon restart re-queues running tasks from scratch (one task was re-queued 33 times), paying for the same model work again. Done when a re-queued task resumes from its last completed step, and re-queues after restarts are capped before the task parks with a clear reason.
Author
Owner

Fixed in e75d5c8ad. Crash recovery now re-queues a running task with resume_from set to the interrupted root step, the results of the last step it finished, and keep_worktree. A crash inside the first resumed step keeps its resume point, because the run clears it only after that step finishes. A task re-queued by 3 restarts in a row parks as needs_decision with a reason; an operator retry or requeue resets the count (new column restart_requeues, migration 0071). The decision is a pure function with one table-driven test. task check was clean. If a crash lands between two steps with none running, the task still starts from the first step.

Fixed in e75d5c8ad. Crash recovery now re-queues a running task with resume_from set to the interrupted root step, the results of the last step it finished, and keep_worktree. A crash inside the first resumed step keeps its resume point, because the run clears it only after that step finishes. A task re-queued by 3 restarts in a row parks as needs_decision with a reason; an operator retry or requeue resets the count (new column restart_requeues, migration 0071). The decision is a pure function with one table-driven test. task check was clean. If a crash lands between two steps with none running, the task still starts from the first step.
sam closed this issue 2026-11-09 03:36:19 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sam/archie#1841