The Geometry of Refusal: Linear Instability in Safety-Aligned LLMs
Shivam Ratnakar, Kartikeya Vats
Why It Matters
What makes this one worth your time
Understanding and improving the safety alignment of LLMs is crucial for deploying these models in real-world applications where safety and compliance are paramount.
The paper introduces a novel method to probe and manipulate the safety alignment of LLMs through output distribution analysis.
Summary
The paper investigates the mechanistic basis of refusal in safety-aligned large language models (LLMs) by introducing Contrastive Logit Steering (CLS), a framework that isolates the 'refusal direction' in output distributions. This method reveals alignment vulnerabilities and enables both attacks and defenses without retraining.
Key contributions
- Introduction of Contrastive Logit Steering (CLS) to analyze and manipulate safety alignment in LLMs.
- Demonstration of architectural determinism in safety implementation across different model families.
- Identification of a 'safety axis' that can be used for both attacking and hardening models.
Notable insights
- Safety compliance in LLMs may be a manipulable linear feature rather than a deep semantic decision.
- The linearity of the 'safety axis' allows for both exploiting vulnerabilities and reinforcing defenses without retraining.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2606.22686v2 Announce Type: replace-cross Abstract: Modern Large Language Models (LLMs) rely on extensive safety alignment, yet the mechanistic basis of refusal remains opaque. In this work, we investigate whether safety compliance is a deep semantic decision or a manipulable linear feature. We introduce Contrastive Logit Steering (CLS), a zero-optimization framework that isolates the "refusal direction" by contrasting hidden states derived from safe and unrestricted system prompts. Unlike representation engineering methods that intervene on internal activations, CLS operates directly on the output distribution, serving as a diagnostic probe for alignment fragility. When coupled with prefix injection to bypass initial refusal reflexes, this method induces a phase transition where guardrails collapse. Our experiments on 7 model families reveal that safety implementation is architecturally deterministic. While models like Llama-3.1 exhibit a "Late Decision" topology that is easily bypassed by CLS (reaching 95% ASR in approximately one second), others like Qwen-2.5 demonstrate "Early Divergence" by integrating safety mid-computation. Direct comparison with established activation-level steering methods shows that CLS achieves substantially higher attack success rates on Llama 2 (73% vs. 22.6%) and Qwen 7B (91% vs. 79.2%), demonstrating that logit-level intervention exposes alignment vulnerabilities that hidden-state methods underestimate. Beyond attacks, we show that this linearity enables bidirectional control: inverting the steering vector "hardens" models against jailbreaks without retraining. Our findings suggest that current alignment techniques create a steerable "safety axis" that serves as both a critical vulnerability and a precise primitive for defense.