Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale
Vadym Hadetskyi, Dario Pasquini, Artem Sorokin
Why It Matters
What makes this one worth your time
Understanding how to adjust safety mechanisms in LLMs for specific domains like cybersecurity can enhance their utility and effectiveness in sensitive applications.
The paper explores how safety alignment in LLMs can be tailored for cybersecurity by identifying and manipulating refusal subspaces.
Summary
The paper investigates the limitations of safety alignment in large language models (LLMs) within the cybersecurity domain, conducting a large-scale experiment on 24 open-source LLMs to demonstrate that domain-specific 'abliteration' can be achieved. It focuses on the distribution of refusal subspaces across layers in LLMs, particularly in trillion-parameter models, and classifies models based on their susceptibility to domain-specific interventions.
Key contributions
- Conducted a large-scale experiment on 24 open-source LLMs to study domain-specific safety alignment.
- Identified the distribution of refusal subspaces across layers in trillion-parameter models.
- Classified models into tiers based on their susceptibility to domain-specific safety interventions.
Notable insights
- Refusal in LLMs occupies a multi-dimensional subspace that is distributed across layers, especially in large models.
- The type of safety training and model architecture are key predictors of a model's susceptibility to domain-specific interventions.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.02714v2 Announce Type: replace-cross Abstract: There is no doubt that safety alignment is an essential step in LLM training. However, conceptually it does not distinguish between various domains and the level of potential harm of a query, which creates significant complications in the fields like cyber security, where a model should not be constrained by its safety circuits to accomplish the goals of legitimate, authorized operations. In this work, we share our findings from a large scale abliteration experiment on 24 open-source LLMs and show that domain-specific abliteration is achievable with standard methodology on the example of a 1T-parameter Kimi K2. Building on recent work showing that refusal in LLMs occupies a multi-dimensional subspace within layers, we find that it is also distributed widely across layers, especially in trillion-parameter MoE architectures, and so we aim to capture the part of it that represents harmful concepts in the cybersecurity domain exclusively. We also investigate the correlation between models' features and the effect of domain-specific abliteration, identifying that the type of safety training and architecture are the most reliable predictors. Finally, we classify the models into 3 abliteration susceptibility tiers and put forward a set of conjectures as to why a particular effect from this intervention might be observed in a given model.