HARC: Coupling Harmfulness and Refusal Directions for Robust Safety Alignment
Shei Pern Chua, Hao Wu, Qianli Ma, Fangzhao Wu
Why It Matters
What makes this one worth your time
Improving the safety alignment of language models is crucial for preventing harmful outputs and ensuring reliable AI systems, making this research relevant for AI safety and deployment.
HARC enhances safety alignment by coupling harmfulness and refusal directions in language models.
Summary
The paper introduces HARC, a fine-tuning method that couples harmfulness and refusal directions in aligned language models to enhance safety alignment. It demonstrates that jailbreaks exploit separable directions in the residual stream and proposes a method to couple these directions without degrading model capabilities. The approach is tested across multiple model families and scales, showing improved robustness-capability-usability trade-offs compared to existing methods.
Key contributions
- Introduction of HARC, a fine-tuning method for coupling harmfulness and refusal directions.
- Analysis of jailbreak mechanisms through separable directions in the residual stream.
- Demonstration of HARC's effectiveness across multiple model families and scales.
Notable insights
- Jailbreaks succeed by suppressing either harmfulness or refusal directions in the residual stream.
- HARC confines intervention to the harmfulness-refusal subspace, preserving general model capabilities.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.00572v3 Announce Type: replace Abstract: Understanding how aligned LLMs internally represent safety is critical for diagnosing alignment vulnerabilities, as it explains why jailbreaks succeed and informs the design of robust alignment strategies. Prior work shows that aligned LLMs encode harmfulness and refusal as separable directions in the residual stream at prompt-side token positions. We show that jailbreaks succeed at prompt encoding by suppressing either the refusal or harmfulness direction before any token is generated, with distinct attack classes occupying separable regions of the harmfulness-refusal plane. Extending the analysis to response-token positions, we find that the model recognizes harmful content while it is generating that content, even when it failed to recognize the input as harmful at the prompt side. Motivated by our findings, we introduce HARC (Harmfulness-And-Refusal Coupling), a fine-tuning method that pairs the two directions across both prompt and response positions. Since the intervention is confined to the harmfulness-refusal subspace, it leaves the rest of the residual stream intact and does not degrade general capability or inflate over-refusal. Across extensive experiments, HARC achieves the strongest robustness-capability-usability trade-off among six baselines spanning the major training-time and inference-time safety methods. The harmfulness and refusal directions at prompt and response positions transfer across the five model families and two scales we tested without architecture-specific tuning.