Beyond Generalist LLMs: Specialist Agentic Systems for Structured Code Workflow Execution
Harris Borman, Herman Wandabwa, Fusun Yu, Sandeepa Kannangara, Justin Liu, Anna Leontjeva, Ritchie Ng
Why It Matters
What makes this one worth your time
Understanding the benefits of specialist agents over generalist ones can lead to more efficient and reliable software development tools, particularly in business process automation.
Specialist agents outperform generalist ones in executing structured code workflows from BPMN diagrams.
Summary
The paper investigates the effectiveness of specialist agents in transforming BPMN diagrams into executable workflows, comparing their performance against generalist agents in terms of tool-use exactness, latency, tool-call errors, and token cost.
Key contributions
- Introduction of a specialist workflow for transforming BPMN diagrams into executable workflows.
- Empirical comparison showing specialist agents' superior performance over generalist agents in several metrics.
Notable insights
- Specialist agents can significantly reduce token costs and eliminate repair iterations compared to generalist agents.
- Generalist agents show inconsistent code generation, affecting their reliability in industrial applications.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.14456v1 Announce Type: cross Abstract: Large Language Models (LLMs) have accelerated the adoption of software development agents, now widely available as Integrated Development Environment (IDE) extensions and standalone applications. While these agents are typically general-purpose, it remains unclear whether specialist agents justify their additional development effort. We investigate this question in the context of business process automation, focusing on the transformation of Business Process Model and Notation (BPMN) diagrams into executable agentic workflows. Since BPMN specifies explicit control-flow semantics, we focus on deterministic workflows in which a fixed process model and inputs uniquely determine the executed path. We introduce a specialist workflow for this task and compare it against generalist agents such as Roo and Cline. Our results show that the specialist solution produces agents that outperform generalist baselines by approximately 9-20 percentage points in tool-use exactness, 2-4x in penalty-adjusted latency, and 3x fewer tool-call errors, while reducing generation token cost by over 95% and eliminating repair iterations. We also find that generalist agents generate code inconsistently in both functionality and quality, limiting their suitability for industrial settings where reliability and maintainability are essential.