Indexing the Unreadable: LLM-Native Recursive Construction and Search of Service Taxonomies
Wei Zheng, Yang Yan, Yiyang Shao, Jinyang Li, Zeze Chang, Yukuang Jia, Qiming Mao, Chihyung Wang, Jingbin Zhou
Why It Matters
What makes this one worth your time
This work is relevant for AI engineers and researchers focusing on improving LLM efficiency and retrieval accuracy in complex service environments, which is crucial for the evolving Internet of Agents.
A2X enhances LLM service discovery by optimizing context management through hierarchical taxonomies.
Summary
The paper proposes a novel LLM-native progressive-disclosure scheme called A2X for organizing and retrieving service taxonomies, addressing the limitations of context management in LLMs when dealing with large service registries.
Key contributions
- Introduction of a progressive-disclosure scheme for service discovery in LLMs.
- Development of the A2X pipeline that organizes services into a hierarchical taxonomy.
- Demonstration of significant improvements in Hit Rate and token efficiency over existing methods.
Notable insights
- The proposed hierarchical taxonomy allows for more efficient context management, mitigating the Lost-in-the-Middle phenomenon.
- The A2X pipeline significantly reduces token consumption while improving retrieval accuracy compared to existing methods.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2605.29270v1 Announce Type: new Abstract: The era of the Internet of Agents (IoA) is taking shape: LLM agents are expected to fulfill user goals by orchestrating fast-growing populations of Model Context Protocol (MCP) servers, Agent-to-Agent (A2A) endpoints, reusable skills, and other LLM-callable services. Yet LLMs face a structural mismatch with this regime: effective context is a scarce resource that does not scale with the number of services. Concatenating thousands of service descriptions into a prompt overflows the context window, and even when the window is large enough, models systematically under-attend to information in the middle of long inputs, the well-documented Lost-in-the-Middle phenomenon. This is fundamentally a question of context management for service discovery. To address this, we propose an LLM-native progressive-disclosure scheme and its concrete instantiation, A2X (Agent-to-Anything service discovery): an LLM-driven pipeline that automatically organizes the registered services into a hierarchical taxonomy and walks it layer by layer at query time, so that every LLM call sees only a small candidate set highly relevant to the user query. This decouples effective-context scarcity from registry size and significantly reduces token consumption while improving retrieval accuracy. Compared to full-context dumping, A2X achieves a 6.2-point Hit Rate gain at one-ninth the prompt-token cost; compared to the state-of-the-art open-source embedding-based baseline, A2X improves Hit Rate by more than 20 points.