The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Models
Adeel Yousaf, Soumik Ghosh, James Beetham, Amrit Singh Bedi, Mubarak Shah
Why It Matters
What makes this one worth your time
Understanding and improving the safety alignment of T2I models is crucial for developing reliable AI systems that can generate accurate and safe content.
The paper exposes the limitations of current utility metrics in T2I models and proposes SAGE to enhance semantic fidelity.
Summary
The paper critiques the use of coarse global utility metrics in evaluating the safety alignment of text-to-image diffusion models, revealing a drop in semantic fidelity when using structured evaluation. It introduces a new method, StructureAware Geometric Regularization (SAGE), to maintain semantic fidelity while ensuring safety.
Key contributions
- Identification of semantic fidelity issues in safety-aligned T2I models using structured evaluation.
- Proposal of StructureAware Geometric Regularization (SAGE) to preserve semantic fidelity.
- Demonstration of improved structured utility with SAGE while maintaining safety and competitive coarse-grained utility scores.
Notable insights
- Coarse global utility metrics can create an illusion of high utility by overlooking fine-grained semantic correctness.
- Semantic collapse in text-encoder prompt embedding space correlates with structured utility loss.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.00402v1 Announce Type: cross Abstract: Safety alignment of text-to-image (T2I) diffusion models aims to suppress harmful generations while preserving utility on benign prompts. Recent methods often appear to deliver high safety with high utility, but this conclusion rests largely on coarse global utility metrics (e.g., FID, CLIPScore) that are insensitive to fine-grained semantic correctness, creating an illusion of high utility. We show that when utility is measured with structured evaluation, this illusion breaks: on TIFA (Text-to-Image Faithfulness evaluation with Question Answering), safety-aligned models suffer substantial drops in semantic fidelity, including failures in object counts, attributes, and relationships. To diagnose the source of this gap, we analyze the text-encoder prompt embedding space and uncover semantic collapse, a contraction of embedding spread coupled with distortion of inter-prompt similarity structure, which strongly correlates with structured utility loss. Guided by this insight, we propose StructureAware Geometric Regularization (SAGE), a safety alignment objective that explicitly preserves embedding spread and inter-prompt relational structure during adaptation. Our method restores structured utility (TIFA +5.0% over prior state-of-the-art) while maintaining strong safety performance and competitive coarse-grained utility scores. Our source code and trained models are available at https://adeelyousaf.github.io/SAGE_ECCV26_Project_Page/.