Run credential lookup failures hide their cause #1808

Closed
opened 2026-11-08 06:15:13 +00:00 by sam · 1 comment
Owner

What is wrong

TaskGrants.taskFor returns any run-credential lookup failure (database timeout, cancelled context) to the caller as Unavailable: run credentials unavailable, and no log line records the underlying error. The State Store journal shows nothing for these calls, so a worker's "run credentials unavailable" cannot be traced to its cause (in the snapshot it coincided with put presence: context deadline exceeded on the same database). The same message is returned from the register, revoke and resolve paths.

Done when

  • The underlying error is logged once at the State Store, with the task id and RPC name, whenever a lookup fails for a reason other than "unknown token".
  • Verified by a test with a failing lookup store that asserts the log carries the cause and the caller still sees only run credentials unavailable.
## What is wrong `TaskGrants.taskFor` returns any run-credential lookup failure (database timeout, cancelled context) to the caller as `Unavailable: run credentials unavailable`, and no log line records the underlying error. The State Store journal shows nothing for these calls, so a worker's "run credentials unavailable" cannot be traced to its cause (in the snapshot it coincided with `put presence: context deadline exceeded` on the same database). The same message is returned from the register, revoke and resolve paths. ## Done when - The underlying error is logged once at the State Store, with the task id and RPC name, whenever a lookup fails for a reason other than "unknown token". - Verified by a test with a failing lookup store that asserts the log carries the cause and the caller still sees only `run credentials unavailable`.
Author
Owner

Fixed in 6ae7f319: the real error is logged server-side with the RPC name (and task id on register); not-found stays Unauthenticated/NotFound, store failures stay Unavailable. Logging and error mapping only, so no test.

Fixed in 6ae7f319: the real error is logged server-side with the RPC name (and task id on register); not-found stays Unauthenticated/NotFound, store failures stay Unavailable. Logging and error mapping only, so no test.
sam closed this issue 2026-11-08 06:28:42 +00:00
sam added the bug label 2026-11-12 15:52:54 +00:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sam/archie#1808