Controllable LLM Reasoning via Sparse Autoencoder-Based Steering
Yi Fang, Wenjie Wang, Mingfeng Xue, Boyi Deng, Fengli Xu, Dayiheng Liu, Fuli Feng
Why It Matters
What makes this one worth your time
This work is relevant for AI researchers and engineers interested in improving the reliability and accuracy of reasoning models by providing a method to control and steer reasoning strategies effectively.
SAE-Steering leverages Sparse Autoencoders to control reasoning strategies in LRMs, enhancing accuracy and flexibility.
Summary
The paper introduces a method called SAE-Steering, which uses Sparse Autoencoders to disentangle strategy-specific features in Large Reasoning Models (LRMs) to improve control over reasoning strategies. This approach aims to enhance the reliability and flexibility of reasoning by redirecting models from erroneous paths to correct ones, resulting in improved accuracy.
Key contributions
- Development of SAE-Steering, a method for controlling reasoning strategies in LRMs.
- Introduction of a two-stage feature identification pipeline for strategy-specific feature selection.
- Demonstration of improved control effectiveness and accuracy in reasoning tasks.
Notable insights
- Using Sparse Autoencoders to disentangle hidden states into a feature space for better control over reasoning strategies.
- A two-stage feature identification pipeline that filters and ranks strategy-specific features for effective control.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2601.03595v2 Announce Type: replace Abstract: Large Reasoning Models (LRMs) exhibit human-like cognitive reasoning strategies (\eg backtracking, cross-verification) during the reasoning process, which improves their performance on complex tasks. Currently, reasoning strategies are autonomously selected by LRMs themselves. However, such autonomous selection often produces inefficient or even erroneous reasoning paths. To make reasoning more reliable and flexible, it is important to develop methods for controlling reasoning strategies. Existing methods struggle to control fine-grained reasoning strategies due to conceptual entanglement in LRMs' hidden states. To address this, we leverage Sparse Autoencoders (SAEs) to decompose strategy-entangled hidden states into a disentangled feature space. To identify the few strategy-specific features from the vast pool of SAE features, we propose SAE-Steering, an efficient two-stage feature identification pipeline. SAE-Steering first recalls features that amplify the logits of strategy-specific keywords, filtering out over 99\% of features, and then ranks the remaining features by their control effectiveness. Using the identified strategy-specific features as control vectors, SAE-Steering outperforms existing methods by over 15\% in control effectiveness. Furthermore, controlling reasoning strategies can redirect LRMs from erroneous paths to correct ones, achieving a 7\% absolute accuracy improvement. Our code and data are available at https://github.com/Peter-Fy/SAE-Steering.