Policy and Confidence guards, advisory until measured #2006
Notifications
Due Date
No due date set.
Blocks
Depends on
Reference: sam/archie#2006
Reference in new issue
Block a user
Model-based guards: Policy reads plain-language policy; Confidence checks the agent's evidence for its diagnosis. Both answer with levels and reasons, and stay advisory (recorded, no ceiling) until their judge passes a measured evaluation against a recorded threshold.
Done when: both guards' verdicts appear in the decision trail as advisory.
Spec: section 5 rule 5, R37.
Built: archipelago guards/policy and guards/confidence (bd087e3) with shared sdk/judgegate and sdk/systemone; core
88afd1ebcgives guards the agent's diagnosis and the evidence it read (guard.v1 JudgeRequest diagnosis and evidence, from the run journal),b6f705921adds a multi-line text setting so the policy text is edited on the guard's settings page. Both are advisory until their recorded eval passes. clef-flash, threshold recall >= 0.8, precision >= 0.8, false flag <= 0.05: policy 35 cases recall 0.92, precision 0.85, false flag 0.09 (fails on false flags); confidence 29 cases recall 0.73, precision 0.80, false flag 0.11 (fails). Not yet installed on staging, so the done-when (verdicts in the trail as advisory) isn't shown: the guards need a judge endpoint the staging host can reach.