L-MAD: A Systematic Evaluation of Multi-Agent Debate Structures in Legal Reasoning
Tan-Minh Nguyen, Hoang-Trung Nguyen, Huu-Dong Nguyen, Dinh-Truong Do, Thi-Hai-Yen Vuong, Le-Minh Nguyen
Why It Matters
What makes this one worth your time
Understanding how multi-agent systems can improve legal reasoning is crucial for developing AI that can assist in high-stakes legal environments.
L-MAD framework enhances legal reasoning by systematically evaluating multi-agent debate structures.
Summary
The paper introduces the Legal Multi-Agent Debate (L-MAD) framework to evaluate different debate structures and aggregation methods in legal reasoning, showing improvements over single-agent baselines and identifying trade-offs in agent population and discussion rounds.
Key contributions
- Introduction of the L-MAD framework for legal reasoning.
- Systematic evaluation of debate structures and aggregation methods.
- Identification of trade-offs in agent population and discussion rounds.
Notable insights
- Increasing agent population reduces inconsistency and improves accuracy.
- Extending discussion rounds can lead to over-deliberation drift, where agents reinforce each other's mistakes.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.09099v1 Announce Type: new Abstract: While multi-agent debate (MAD) frameworks have shown significant potential in general reasoning, their effectiveness in highly structured, knowledge-heavy legal domains remains under-explored. In this work, we introduce the Legal Multi-Agent Debate (L-MAD) framework to systematically evaluate different debate structures and aggregation methods within Legal Textual Entailment. By assigning distinct expert personas to multiple agents, L-MAD improves upon strong single-agent baselines by up to 8\%. Furthermore, analyzing how debate scales reveals a clear trade-off: increasing the agent population reduces inconsistency and improves accuracy, whereas extending discussion rounds induces a detrimental \textit{over-deliberation drift} where agents reinforce each other's mistakes. Ultimately, our findings outline the practical boundaries and safety margins of deploying collaborative multi-agent systems in high-stakes legal reasoning environments.