Verify the chat refusal, approval binding, run cancel and connector checks on a running deployment #2096

Open
opened 2026-11-21 15:01:47 +00:00 by sam · 2 comments
Owner

Why: #2071, #2072, #2088, #2091 and #2092 were closed once their commits landed on main, without the manual checks their issues describe. This issue holds those checks.

Checks, on a deployment built from main at or after 592d36e0d:

  • #2071 (bc72655c8): signed in as the owner, ask chat to run an admin action and see it refused; open that action's class in the dashboard and see the refusal recorded; ask chat to export a connector secret and see it refused with no approve path.
  • #2072 (b9a60e03a): trigger a held action in chat, approve it on the card and see it run once in the turn's tool calls; replay the answer through the API and get a conflict; let the deadline pass and see the action expire unrun. The People page from 4b494bf75 is removed by #2095, so check approval from the dashboard after that lands.
  • #2088 (d318f912a): cancel a parent run waiting on a workflow.call child and see both cancelled; cancelling an ended run is refused with a clear error.
  • #2091 (592d36e0d): change a connector's declarations so a saved workflow would become invalid and see the connector refuse to start with a clear reason, keeping the previous declarations.
  • #2092 (592d36e0d): load a connector whose message or chat_message schema rejects a payload core sends and see it fail at load with a clear reason.

Done when: every check above passes, or a bug is filed for each one that doesn't.

**Why**: #2071, #2072, #2088, #2091 and #2092 were closed once their commits landed on main, without the manual checks their issues describe. This issue holds those checks. **Checks**, on a deployment built from main at or after 592d36e0d: - **#2071** (bc72655c8): signed in as the owner, ask chat to run an admin action and see it refused; open that action's class in the dashboard and see the refusal recorded; ask chat to export a connector secret and see it refused with no approve path. - **#2072** (b9a60e03a): trigger a held action in chat, approve it on the card and see it run once in the turn's tool calls; replay the answer through the API and get a conflict; let the deadline pass and see the action expire unrun. The People page from 4b494bf75 is removed by #2095, so check approval from the dashboard after that lands. - **#2088** (d318f912a): cancel a parent run waiting on a workflow.call child and see both cancelled; cancelling an ended run is refused with a clear error. - **#2091** (592d36e0d): change a connector's declarations so a saved workflow would become invalid and see the connector refuse to start with a clear reason, keeping the previous declarations. - **#2092** (592d36e0d): load a connector whose message or chat_message schema rejects a payload core sends and see it fail at load with a clear reason. **Done when**: every check above passes, or a bug is filed for each one that doesn't.
sam added the taskmedium labels 2026-11-21 15:01:47 +00:00
Author
Owner

Checked on staging at 48d021278.

  • #2071 pass. Privileged and secret-access actions are not offered to chat; an action that resolves as privileged is refused by the chat-prohibited-effect hard rule, recorded at level never in its class history, with no approval request. Checked with the bootstrap admin token, not an owner session.
  • #2072 fail. Approving an unchanged held action was refused with "the held action differs from the one approved": the binding read back from jsonb carries reformatted input and was compared byte for byte without canonicalising. Fixed in 9c0ea938b with a regression case. Still to check after the fix is deployed: approve once and see it run once, replay the answer from an owner session for a conflict, let a hold expire unrun, and the passkey confirmation for an irreversible action (after #2095).
  • #2088 pass. Cancelling a parent waiting on workflow.call cancelled both; an ended run gives 409, an unknown one 404. No trail entry is written: #2100.
  • #2091 pass. Re-accepting a connector that drops an action a saved workflow uses is refused with 422 and keeps the previous declarations. The refused settings are saved anyway: #2099.
  • #2092 pass. message and chat_message schemas that reject a core payload fail the connector at load with a clear reason.

Throwaway workflows and a trigger could not be removed without database access: #2098.

Checked on staging at 48d021278. - #2071 pass. Privileged and secret-access actions are not offered to chat; an action that resolves as privileged is refused by the chat-prohibited-effect hard rule, recorded at level never in its class history, with no approval request. Checked with the bootstrap admin token, not an owner session. - #2072 fail. Approving an unchanged held action was refused with "the held action differs from the one approved": the binding read back from jsonb carries reformatted input and was compared byte for byte without canonicalising. Fixed in 9c0ea938b with a regression case. Still to check after the fix is deployed: approve once and see it run once, replay the answer from an owner session for a conflict, let a hold expire unrun, and the passkey confirmation for an irreversible action (after #2095). - #2088 pass. Cancelling a parent waiting on workflow.call cancelled both; an ended run gives 409, an unknown one 404. No trail entry is written: #2100. - #2091 pass. Re-accepting a connector that drops an action a saved workflow uses is refused with 422 and keeps the previous declarations. The refused settings are saved anyway: #2099. - #2092 pass. message and chat_message schemas that reject a core payload fail the connector at load with a clear reason. Throwaway workflows and a trigger could not be removed without database access: #2098.
Author
Owner

Added after e3130d53e, once it is deployed with the rebuilt Telegram connector (archipelago 890a1b9):

  • #2095: on an empty database, finish setup, link Telegram from the account page, trigger a held action, approve once by tap and once from the dashboard; a member without approve is refused on both paths; staging's existing Telegram identity is rebound to the owner account by migration 00049.
  • #2078: fire a trigger and see the run's trail carry the workload's authorization; call a run or source endpoint with a low-privilege token and get 403; revoke a role while a run is suspended and see its next step refused.
  • #2090: delete an alias from /settings/models.
Added after e3130d53e, once it is deployed with the rebuilt Telegram connector (archipelago 890a1b9): - #2095: on an empty database, finish setup, link Telegram from the account page, trigger a held action, approve once by tap and once from the dashboard; a member without approve is refused on both paths; staging's existing Telegram identity is rebound to the owner account by migration 00049. - #2078: fire a trigger and see the run's trail carry the workload's authorization; call a run or source endpoint with a low-privilege token and get 403; revoke a role while a run is suspended and see its next step refused. - #2090: delete an alias from /settings/models.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sam/archie#2096