LeAct: Learning to Reason from Expert Actions
Ziran Yang, Chengshuai Shi, Raj Ghugare, Benjamin Eysenbach, Karthik Narasimhan, Chi Jin
Why It Matters
What makes this one worth your time
This approach provides a novel way to extract reasoning from expert systems, potentially enhancing the capabilities of AI models in complex domains.
LeAct leverages expert actions to learn reasoning processes, outperforming traditional methods in games and robotics.
Summary
The paper introduces LeAct, a method for learning reasoning from expert actions by treating the reasoning process as a latent variable. The approach involves generating candidate chains of thought (CoTs) for each expert action and retaining those that improve the probability of recovering the action. LeAct is evaluated on imperfect-information games and a simulated robotics benchmark, showing improved performance over existing baselines.
Key contributions
- Introduction of LeAct, a method to learn reasoning from expert actions.
- Demonstration of LeAct's effectiveness in games and robotics benchmarks.
- Proposal of a framework that uses expert systems as reasoning teachers for AI models.
Notable insights
- Treating the reasoning process as a latent variable allows for the extraction of implicit knowledge from expert actions.
- The method effectively uses candidate CoTs to improve action recovery, demonstrating a new way to learn reasoning.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.21856v1 Announce Type: cross Abstract: Modern reasoning models depend on reasoning data, today sourced from human annotations or distilled from stronger LLMs. However, a rich and largely untapped source of supervision lies in expert systems (e.g., game engines, classical planners, theorem provers), which routinely produce near-optimal actions across diverse domains. But these experts are silent: they commit to an action without writing down the chain of thought (CoT) behind it. Recovering that CoT as natural-language reasoning would distill expert knowledge into a student that generalizes beyond the demonstrated actions. We treat it as a latent variable and study how to recover it from the action alone. Our approach, LeAct (Learning to reason from Actions), optimizes this latent variable: the student samples candidate CoTs for each expert action, and we retain those that measurably improve its own probability of recovering the action. Across imperfect-information games at multiple scales and a simulated robotics benchmark, LeAct reaches the solver's numerical floor on small enumerable games. At larger scale, it is $5\times$ closer to the solver than the strongest expert-iteration baseline. At Flop Hold'em ($\sim 10^9$ infosets), LeAct wins head-to-head by $+60$ mbb/g, and on the robotics probe it is the only training recipe that improves on direct imitation. We present a principled framework and the result: expert systems become a categorically new source of reasoning teachers for foundation models.