SAFETY SENTRY: Context-Aware Human Intervention via EXECUTE-ASK-REFUSE Routing
Tianyu Chen, Chujia Hu, Wenjie Wang
Why It Matters
What makes this one worth your time
This approach could enhance the reliability and user experience of AI systems by reducing false alarms and improving safety in real-world applications.
Safety Sentry refines AI safety by making context-aware decisions on action execution.
Summary
The paper introduces Safety Sentry, a guard model for LLM agents that makes per-instance routing decisions among EXECUTE, ASK, and REFUSE, aiming to improve safety and reduce unnecessary interruptions in human-AI interactions.
Key contributions
- Proposes a three-way routing decision model for AI safety.
- Introduces Safety Sentry, a lightweight guard model with a single decoding call.
- Demonstrates superior performance over existing baselines in accuracy and safety recall.
Notable insights
- Reframes safety as a three-way decision problem rather than binary classification.
- Uses a single decoding-time threshold to adjust risk tolerance without retraining.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.13594v1 Announce Type: new Abstract: LLM agents act on real-world environments through tool calls, and a single misjudged action can cause irreversible harm. The standard safeguard is a guard model that labels each proposed action as safe or unsafe, but this binary view conflates two distinct decisions: whether the action is harmful in itself, and whether it is appropriate given the user's context. It also operates at the granularity of action categories rather than individual instances, producing routine interruptions that erode autonomy and train users to wave through the most consequential alerts. We reframe the problem as a per-instance three-way routing decision over {EXECUTE, ASK, REFUSE} and instantiate it with Safety Sentry, a lightweight guard model whose inference reduces to a single decoding call. A single decoding-time threshold lets one fixed checkpoint be re-positioned across deployments of differing risk tolerance without retraining. Safety Sentry outperforms a broad set of open-weight and frontier closed-source baselines on overall accuracy and safety-related recall, while controlling both directional error rates simultaneously.