LASA: Language-Agnostic Semantic Alignment at the Semantic Bottleneck for LLM Safety
Junxiao Yang, Haoran Liu, Jinzhe Tu, Jiale Cheng, Zhexin Zhang, Shiyao Cui, Jiaqi Weng, Jialing Tao, Hui Xue, Hongning Wang, Han Qiu, Minlie Huang
Why It Matters
What makes this one worth your time
Improving LLM safety across languages is crucial for deploying these models globally without compromising security in low-resource languages.
LASA enhances LLM safety by aligning safety measures with language-agnostic semantic understanding.
Summary
The paper identifies a semantic bottleneck in large language models (LLMs) where semantic understanding is language-agnostic, and proposes a method called Language-Agnostic Semantic Alignment (LASA) to improve safety alignment across languages by anchoring safety in this semantic space. Experiments demonstrate a significant reduction in attack success rates across multiple models.
Key contributions
- Identification of a semantic bottleneck in LLMs for language-agnostic understanding.
- Proposal of the LASA method to enhance safety alignment across languages.
- Empirical demonstration of reduced attack success rates using LASA.
Notable insights
- Semantic bottlenecks in LLMs can be leveraged for language-agnostic safety alignment.
- Anchoring safety alignment in semantic space rather than surface text can improve model robustness.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2604.12710v2 Announce Type: replace-cross Abstract: Large language models (LLMs) often demonstrate strong safety performance in high-resource languages, yet exhibit severe vulnerabilities when queried in low-resource languages. We attribute this gap to a mismatch between language-agnostic semantic understanding ability and language-dominant safety alignment biased toward high-resource languages. Consistent with this hypothesis, we empirically identify the semantic bottleneck in LLMs, an intermediate layer in which the geometry of model representations is governed primarily by shared semantic content rather than language identity. Building on this observation, we propose Language-Agnostic Semantic Alignment (LASA), which anchors safety alignment directly in semantic bottlenecks. Experiments show that LASA substantially improves safety across all languages: average attack success rate (ASR) drops from 24.7% to 2.8% on LLaMA-3.1-8B-Instruct and remains around 3-4% across Qwen2.5 and Qwen3 Instruct models (7B-32B). Together, our analysis and method offer a representation-level perspective on LLM safety, suggesting that safety alignment requires anchoring safety understanding not in surface text, but in the model's language-agnostic semantic space.