ALIGNBEAM : Inference-Time Alignment Transfer via Cross-Vocabulary Logit Mixing
Chirag Chawla, Pratinav Seth, Vinay Kumar Sankarapu
Why It Matters
What makes this one worth your time
This approach addresses a critical safety issue in large language models, allowing for improved robustness against harmful prompts while maintaining task performance.
ALIGNBEAM enables safe inference across different language model families without retraining.
Summary
The paper introduces ALIGNBEAM, a method for transferring safety alignment between large language models at inference time without modifying model weights, enabling the use of logits from a safe anchor model even when vocabularies differ.
Key contributions
- Presents a training-free method for cross-vocabulary logit mixing to enhance safety in language models.
- Demonstrates substantial improvements in refusal rates on adversarial benchmarks while maintaining task accuracy.
- Enables inference-time safety alignment transfer without modifying model weights.
Notable insights
- The method's token-by-token translation of logits allows for flexibility in handling cross-vocabulary scenarios, which is a significant advancement over existing techniques.
- The ability to tune the safety-utility trade-off at deployment without retraining provides practical advantages for real-world applications.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2606.12342v1 Announce Type: cross Abstract: Domain fine-tuning degrades the safety of large language models: fine-tuned specialists readily comply with harmful prompts framed in domain language. Existing inference-time defenses that mix logits from a safe anchor model require both models to share a vocabulary, which rules them out for the cross-family specialists where safety is most degraded. We present ALIGNBEAM, a training-free method that lifts this restriction by translating anchor logits into the target model's vocabulary token-by-token at each decoding step; a small LLM judge then selects the safest among K candidate continuations. No weights are changed, and the safety-utility trade-off can be tuned at deployment without retraining. Across both cross-vocabulary and same-vocabulary evaluation pairs, ALIGNBEAM substantially raises refusal on adversarial benchmarks while keeping task accuracy and inference overhead within practical bounds. The results show that safety alignment can be transferred between model families at inference time, without touching either model's weights.