LLMs as a Jury: Cross-Model Consensus Can Outperform Process Reward Models for LLM Reasoning
Ning Liu
Why It Matters
What makes this one worth your time
This approach offers a potentially more robust and scalable method for improving reasoning accuracy in LLMs without the need for additional training data or complex reward models, which could be valuable for AI engineers and researchers working on enhancing model performance.
Cross-model consensus in LLMs can outperform traditional methods for reasoning tasks by leveraging error decorrelation.
Summary
The paper investigates the use of cross-model consensus as a method for selecting correct answers from candidate reasoning chains in large language models (LLMs). It proposes treating multiple independently trained models as a jury, where the agreement among models serves as a verification signal. The study demonstrates that this approach can outperform traditional methods like self-consistency and trained reward models, particularly in mathematical reasoning tasks.
Key contributions
- Introduction of cross-model consensus as a verification signal for LLM reasoning.
- Demonstration of superior performance of cross-model consensus over self-consistency and reward models in mathematical reasoning tasks.
- Development of a parameter-free law to predict consensus accuracy.
Notable insights
- Cross-model consensus leverages error decorrelation among independently trained models to improve answer selection.
- The method provides a parameter-free law to predict consensus accuracy, offering a theoretical framework for its application.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.10139v2 Announce Type: replace-cross Abstract: Selecting the correct answer from a pool of candidate reasoning chains is the engine of test-time scaling, yet the standard selectors each carry a cost: self-consistency inherits the errors of the single model it resamples, and trained reward models need labeled data and transfer poorly off-distribution. We study a third signal, free at inference time: cross-model consensus, the degree to which independently trained models, each solving the problem once, agree on a final answer. We treat the panel as an LLM-jury, in which the structure of agreement, not any model's score of another, is the verification signal. Across seven benchmarks it selects correct answers better than self-consistency and far better than a model scoring its own candidates: on competition math it closes the entire gap to an oracle selector, while self-scoring closes almost none. The mechanism is error decorrelation: independently trained models err differently, so their wrong answers scatter while the correct one accumulates agreement. We make this precise with a parameter-free law, derived in closed form, that predicts consensus accuracy from three measured panel statistics to a mean absolute error of $0.03$ and exposes the method's ceiling: a shared-error floor where models share a misconception, near zero on math but non-trivial on science. Against four trained verifiers spanning discriminative, outcome, and generative reward models, the free LLM-jury matches the strongest inside their math training domain and is the top selector outside it. Cross-model consensus is thus a verifier we can characterize in advance: a law that says when to trust it, and a floor that marks where it cannot.