When and How Severely: Scenario-Specific Safety Envelopes for Driving VLAs
Abhinaw Priyadershi, Jelena Frtunikj
Why It Matters
What makes this one worth your time
Understanding and improving the safety of autonomous driving systems is crucial for their deployment and acceptance in real-world scenarios.
The paper proposes a two-dimensional safety envelope for VLA driving planners to improve safety certification.
Summary
The paper evaluates the safety certification of Vision-Language-Action (VLA) driving planners under ISO 21448 by analyzing when and how severely these planners fail. It uses a Gaussian Mixture Model to identify severity bands and suggests that a two-dimensional safety envelope is necessary for a deployable SOTIF ODD specification.
Key contributions
- Evaluation of Alpamayo R1 VLA on 15,968 (clip, attack) pairs.
- Identification of six discrete severity bands using a Gaussian Mixture Model.
- Proposal of a two-dimensional safety envelope for SOTIF ODD specification.
Notable insights
- A Gaussian Mixture Model is used to identify discrete severity bands in failure scenarios.
- The study finds that scenarios with the loosest noise thresholds do not necessarily have the lowest high-severity failure rates.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2606.14238v1 Announce Type: cross Abstract: Safety certification of Vision-Language-Action (VLA) driving planners under ISO 21448 (SOTIF) rests on an Operational Design Domain (ODD) specification that answers two complementary questions: when does the planner start to fail, and how severely does it fail once it does? We evaluate Alpamayo R1, a 10B-parameter open-weight driving VLA, on 15,968 (clip, attack) pairs. We find a conservative-aggregate gap: an aggregate safe threshold of $\sigma \leq 50$ under a 15% average displacement error (ADE) budget masks well-sampled scenarios that tolerate the top of the tested grid ($\sigma = 70$). A Gaussian Mixture Model (GMM) on the changed-explanation subset identifies six discrete severity bands (BIC-optimal $k{=}6$), so two perturbation conditions with the same mean error can differ materially in their share of high-severity (C4/C5) failures. Joining the two analyses on the same corpus surfaces a finding neither yields in isolation: the scenarios with the loosest noise thresholds are not those with the lowest high-severity rate: STOP_SIGNAL concentrates roughly $4\times$ the C4/C5 share of LANE_KEEPING despite tolerating a larger $\sigma$. A deployable SOTIF ODD specification for driving VLAs therefore requires a two-dimensional safety envelope, not a single aggregate value per hazard.