How Long Reasoning Chains Influence LLMs' Judgment of Answer Factuality
Minzhu Tu, Shiyu Ni, Keping Bi
Why It Matters
What makes this one worth your time
Understanding how reasoning chains influence LLM judgments can help improve the robustness and accuracy of AI systems in evaluating factuality, which is crucial for reliable AI applications.
The study explores how reasoning chains affect LLMs' ability to judge answer factuality.
Summary
The paper investigates the influence of reasoning chains on the judgment of answer factuality by large language models (LLMs), finding that weak judges are easily misled by fluent reasoning, while strong judges can partially use reasoning as evidence but are still susceptible to high-quality but incorrect reasoning chains.
Key contributions
- Systematic investigation of reasoning chains' impact on LLM judgment.
- Controlled experiments revealing the role of fluency and factuality in reasoning chains.
Notable insights
- Weak LLM judges are prone to accepting incorrect answers if accompanied by fluent reasoning.
- Even strong LLM judges can be misled by reasoning chains that appear high-quality but are factually incorrect.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2604.06756v2 Announce Type: replace Abstract: Large language models (LLMs) has been widely adopted as a scalable surrogate for human evaluation, yet such judges remain imperfect and susceptible to surface-level biases. One possible reason is that these judges lack sufficient information in assessing answer correctness. With the rise of reasoning-capable models, exposing a generator's reasoning content to the judge provides richer information and is a natural candidate for improving judgment accuracy. However, its actual impact on judge behavior remains understudied. In this paper, we systematically investigate how access to reasoning chains affects LLM-based judgment across factual question answering (QA) and mathematical reasoning benchmarks. We find that weak judges are easily swayed by reasoning presence, frequently accepting incorrect answers accompanied by fluent reasoning, while strong judges can partially leverage reasoning as informative evidence. Nevertheless, even strong judges are misled by seemingly high-quality reasoning chains. Controlled experiments further reveal that both fluency and factuality of reasoning chains are critical signals driving judge decisions. These findings highlight the need for more robust LLM judges that can distinguish genuine reasoning quality from superficial fluency when evaluating modern reasoning models.