OMAC: A Holistic Optimization Framework for LLM-Based Multi-Agent Collaboration
Shijun Li, Hilaf Hasson, Joydeep Ghosh
Why It Matters
What makes this one worth your time
This work addresses a gap in the systematic design and optimization of LLM-based multi-agent systems, which is crucial for enhancing their performance in complex tasks.
OMAC offers a structured approach to optimize collaboration in LLM-based multi-agent systems.
Summary
The paper introduces OMAC, a framework for optimizing multi-agent systems powered by large language models, identifying five key optimization dimensions and proposing algorithms for both single and joint optimization across these dimensions.
Key contributions
- Introduction of the OMAC framework for holistic optimization of LLM-based multi-agent systems.
- Development of algorithms for optimizing single and multiple dimensions of agent functionality and collaboration.
- Empirical validation demonstrating superior performance in specific tasks compared to state-of-the-art methods.
Notable insights
- The identification of five key optimization dimensions provides a comprehensive approach to improving agent collaboration.
- The use of a Semantic Initializer and Contrastive Comparator as part of the optimization algorithm is a novel methodological contribution.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2505.11765v4 Announce Type: replace-cross Abstract: Agents powered by advanced large language models (LLMs) have demonstrated impressive capabilities across diverse complex applications. Recently, Multi-Agent Systems (MAS), wherein multiple agents collaborate and communicate with each other, have exhibited enhanced capabilities in complex tasks, such as high-quality code generation and arithmetic reasoning. However, the development of such systems often relies on handcrafted methods, and the literature on systematic design and optimization of LLM-based MAS remains limited. In this work, we introduce \textbf{OMAC}, a general framework designed for holistic optimization of LLM-based MAS. Specifically, we identify five key optimization dimensions for MAS, encompassing both agent functionality and collaboration structure. Building upon these dimensions, we first propose a general algorithm, utilizing two actors termed the Semantic Initializer and the Contrastive Comparator, to optimize any single dimension. Then, we present an algorithm for joint optimization across multiple dimensions. Extensive experiments demonstrate the superior performance of OMAC on diverse tasks against recent approaches.