The Inattentional Gap: Task-Conditioned Language and Vision Models Omit the Safety-Critical Signals They Can Otherwise Report
Kwan Soo Shin, In Seok Kang, Yunkyung Min, Judy Yang, Munho Lee
Why It Matters
What makes this one worth your time
Understanding and mitigating the Inattentional Gap is crucial for deploying AI systems in safety-critical applications, ensuring they do not overlook hazards not explicitly specified in their tasks.
Task-focused AI models can miss safety-critical signals, but this can be mitigated with external oversight.
Summary
The paper investigates how task-conditioned language and vision models can fail to report safety-critical signals due to a phenomenon termed the 'Inattentional Gap'. This gap arises when models are narrowly focused on specific tasks, leading to a suppression of reporting on other important signals. The study demonstrates this effect across various scenarios and proposes a method to mitigate it by using an independent open-ended critic to restore omitted findings.
Key contributions
- Identification and characterization of the Inattentional Gap in task-conditioned AI models.
- Demonstration of the gap's persistence across different model families and tasks.
- Proposal of a mitigation strategy using an independent open-ended critic.
Notable insights
- The Inattentional Gap is a machine analogue of human inattentional blindness, affecting AI models conditioned on narrow tasks.
- An independent open-ended critic can restore omitted findings, suggesting a practical mitigation strategy.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2606.26529v2 Announce Type: replace-cross Abstract: AI safety is evaluated by how reliably a model detects the hazards it is told to find, yet accidents often arise from the hazard no one specified. We show that conditioning a language or vision model on a narrow task suppresses its reporting of co-present, safety-critical signals it can otherwise report, a machine analogue of human inattentional blindness, produced by a different mechanism. Across radiology and driving text scenarios and chest-radiograph vision tasks, the ordinary focused instructions under which such systems are deployed suppressed reporting by up to 0.92 in report rate relative to the same models when unconstrained, and an explicit exclusive instruction abolished reporting entirely in radiology. Suppression appeared in every model tested, did not diminish with scale, persisted in a reasoning model, and varied more by model family than by size. We name this dissociation the Inattentional Gap and argue that it decouples measured benchmark safety from real-world safety: a system can score near-perfectly on the hazards an evaluation specifies while remaining blind to those that cause harm. Probing the mechanism, we localize the proximal trigger to output scope and find System-1-style task capture without reliable intrinsic oversight in the sampled systems. Oversight could, however, be supplied externally: routing each narrow report to an independent open-ended critic restored every omitted finding, demonstrating that the gap is both measurable and mitigable. We propose reporting-complete evaluation, scoring what a system fails to report alongside what it is asked to find, as a requirement for safety-critical deployment.