Silent Failures in Physical AI: A Literature Review of Runtime Action Authorization for Autonomous Systems
Barak Or
Why It Matters
What makes this one worth your time
Understanding and mitigating silent failures in autonomous systems is crucial for ensuring the safety and reliability of AI-driven physical actions in real-world applications.
The paper identifies and addresses gaps in safety mechanisms for runtime action authorization in Physical AI systems.
Summary
The paper reviews existing literature on runtime action authorization for autonomous systems, identifying a gap in current safety mechanisms for Physical AI systems that can lead to silent failures. It proposes a bounded problem formulation, defines silent physical-action failure, and develops a taxonomy of runtime guardrail functions to evaluate Physical AI assurance mechanisms.
Key contributions
- Identification of a gap in runtime authorization for Physical AI systems.
- Development of a bounded problem formulation and definition of silent physical-action failure.
- Creation of a taxonomy of runtime guardrail functions for evaluating Physical AI assurance mechanisms.
Notable insights
- Silent failures can occur despite models appearing confident and semantically aligned.
- Current safety mechanisms for Physical AI systems are advancing along separate technical tracks, leading to gaps in runtime authorization.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2606.00090v1 Announce Type: cross Abstract: Physical AI systems increasingly map multimodal observations, language instructions, and learned world representations into physically consequential actions. Robotics foundation models, vision-language-action models, and world-model-based autonomous systems can condition decisions that move vehicles, robots, drones, and industrial machines. This transition exposes a safety problem that is not fully captured by conventional AI content moderation or by classical robot safety alone: a black-box model may issue a physically consequential action while appearing confident, plausible, and semantically aligned. The resulting failure can be silent, arising from sensor drift, occlusion, state-estimation error, distribution shift, hallucinated affordances, or invalid physical assumptions before downstream hardware controllers detect a violation. Across embodied foundation models, world models, robotics simulation, embodied safety benchmarks, safe control, runtime assurance, uncertainty estimation, verification, and guardrail evaluation, model capability and safety mechanisms have advanced along largely separate technical tracks. A recurring gap synthesized here is that no single stream surveyed in this review supplies a complete runtime authorization boundary between black-box Physical AI models and physical execution. The resulting analysis develops a bounded problem formulation, a definition of silent physical-action failure, a taxonomy of runtime guardrail functions, and evaluation requirements for comparing guardrails as Physical AI assurance mechanisms.