Doing What They Say, Not What They Reason: Locating the Faithfulness Gap in LLM Agents
Yufeng Wang
Why It Matters
What makes this one worth your time
Understanding the faithfulness gap in LLM agents is crucial for improving their reliability in applications requiring consistent reasoning and action alignment.
The paper explores the discrepancy between LLM agents' stated reasoning and their actions in a poker simulation.
Summary
The paper investigates the faithfulness of large language model (LLM) agents in executing actions based on their stated reasoning within a controlled Texas Poker simulation environment. It decomposes the faithfulness gap into reasoning-conclusion and conclusion-action steps, observing opposite behaviors in these steps.
Key contributions
- Decomposition of the faithfulness gap into two distinct steps.
- Application of a controlled Texas Poker simulation to study LLM agent behavior.
Notable insights
- The faithfulness gap is decomposed into reasoning-conclusion and conclusion-action steps, which behave oppositely.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2606.00476v1 Announce Type: new Abstract: Do LLM agents act on the reasoning they state? This question of process fidelity is central to using LLMs in social simulation, yet it is hard to measure where no reference for correct behavior exists. We study it in acontrolled setting, a Texas Poker simulator with a verifiable reference action for every decision by decomposing the faithfulness gap into two steps: reasoning-conclusion and conclusion-action. The two steps behave oppositely.