Diversity Without Fidelity: A Solver-Sampler Mismatch in Multi-Agent LLM Negotiation Simulation
Sandro Andric
Why It Matters
What makes this one worth your time
Understanding how language models simulate human negotiation can improve their application in policy exercises and stakeholder simulations, ensuring more realistic and human-like interactions.
The paper explores the gap between solving and simulating human-like negotiation in multi-agent language models.
Summary
The paper investigates the mismatch between solver and sampler roles in multi-agent negotiation simulations using language models, highlighting that reasoning modes affect negotiation diversity but not agreement outcomes. It emphasizes the need for models to be evaluated on their ability to simulate realistic human-like negotiation behaviors.
Key contributions
- Identification of the solver-sampler mismatch in multi-agent negotiation simulations.
- Demonstration that reasoning modes affect negotiation diversity but not agreement outcomes.
- Highlighting the importance of structured memory in achieving negotiation agreements.
Notable insights
- Enabling reasoning in language models increases negotiation diversity but does not improve agreement rates.
- Structured memory of negotiations significantly enhances agreement outcomes, suggesting its importance in simulation fidelity.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2604.11840v3 Announce Type: replace-cross Abstract: Language models are increasingly used to simulate people: survey respondents, negotiators, stakeholders in policy exercises. In that role a model should reproduce how people plausibly behave, hesitating, conceding late, and settling for imperfect deals, rather than playing the best move. We call this the sampler role, in contrast to the solver role of finding the best move, and we test how the reasoning modes providers ship to strengthen models as solvers affect it. Our testbed is multi-party negotiation: five agents bargain over a regulation for fifteen turns, and unresolved issues are decided by an authority. Agents without a structured memory of the negotiation almost never reach agreement, whether reasoning is on or off: 314 of 315 such runs end with the authority deciding. What reasoning changes is how the failure looks. With reasoning enabled, one model family negotiates visibly, with varied moves, concessions in most runs, and a different path every time, yet still ends without agreement in fifteen runs of fifteen. Diversity checks would pass a model whose endings never change. Two further results show the task permits agreement: when agents write their own short running notes on the negotiation, agreement becomes the norm, while the same notes supplied ready-made change nothing; and hand-coded agents following textbook concession strategies agree in most runs under identical rules. Simulation pipelines should therefore vet models as samplers, on the distributions of outcomes they produce. Fidelity as a sampler must be tested on its own: solver strength is no guide to it, and switching on reasoning leaves it where it was.