[bug] workflow.call deadlocks the dispatcher: intake stalls while a caller waits #1223

Closed
opened 2026-10-27 13:39:31 +00:00 by sam · 0 comments
Owner

drainNATS submits a whole pass and then blocks in dispatcher.Wait() until every submitted task finishes (internal/daemon/daemon.go). A wait:true workflow.call is one of those tasks: it polls its callee for the callees whole life, so the pass cannot return, the next cycle cannot claim the queued callee, and the callee (same owner/repo) also queues behind the callers repo baton. With containers.max_concurrency saturated, or any repo without allow_concurrent, the wait never completes.

Design:
(a) A drain pass must not wait for its tasks. The wait belongs at shutdown: drainNATS returns once the stream and the stores queued fallback are empty; the daemon waits for in-flight work once, on ctx cancellation.
(b) A caller waiting on its callee holds no execution capacity. taskDispatcher gains Suspend(id)/Resume(id), refcounted per task: Suspend releases the tasks global slot and its repo baton, Resume re-acquires both asynchronously (a resumed task never blocks its own run waiting for capacity).
(c) The daemon watches its own events.Bus (already fed by the agent-event bridge) for workflow_call_started with data wait==true and workflow_call_finished, for the callers task id, and drives (b). No store change, no new wire contract.

Acceptance: a dispatcher test proves Suspend frees the slot and the repo baton (a second task in the same repo runs) and Resume restores them; drainNATS no longer blocks the cycle on a running task.

Notes

Fixed in 629404a5; verified in HEAD 5124bfd4. Closing as already shipped.

Close reason

Closed


Imported from beads archie-core-apdo · GitHub #1522 · type bug · priority P1 · created 2026-10-08 · closed 2026-10-10

drainNATS submits a whole pass and then blocks in dispatcher.Wait() until every submitted task finishes (internal/daemon/daemon.go). A wait:true workflow.call is one of those tasks: it polls its callee for the callees whole life, so the pass cannot return, the next cycle cannot claim the queued callee, and the callee (same owner/repo) also queues behind the callers repo baton. With containers.max_concurrency saturated, or any repo without allow_concurrent, the wait never completes. Design: (a) A drain pass must not wait for its tasks. The wait belongs at shutdown: drainNATS returns once the stream and the stores queued fallback are empty; the daemon waits for in-flight work once, on ctx cancellation. (b) A caller waiting on its callee holds no execution capacity. taskDispatcher gains Suspend(id)/Resume(id), refcounted per task: Suspend releases the tasks global slot and its repo baton, Resume re-acquires both asynchronously (a resumed task never blocks its own run waiting for capacity). (c) The daemon watches its own events.Bus (already fed by the agent-event bridge) for workflow_call_started with data wait==true and workflow_call_finished, for the callers task id, and drives (b). No store change, no new wire contract. Acceptance: a dispatcher test proves Suspend frees the slot and the repo baton (a second task in the same repo runs) and Resume restores them; drainNATS no longer blocks the cycle on a running task. ### Notes Fixed in 629404a5; verified in HEAD 5124bfd4. Closing as already shipped. ### Close reason Closed --- _Imported from beads `archie-core-apdo` · GitHub #1522 · type bug · priority P1 · created 2026-10-08 · closed 2026-10-10_
sam added the highbug labels 2026-10-27 13:39:31 +00:00
sam self-assigned this 2026-10-27 13:39:31 +00:00
sam closed this issue 2026-10-27 13:39:31 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sam/archie#1223