Exploration Structure in LLM Agents for Multi-File Change Localization
Akeela Darryl Fattha, Kia Ying Chua, Lingxiao Jiang, Laura Wynter
Why It Matters
What makes this one worth your time
Understanding and improving how AI agents localize software changes can enhance the efficiency and accuracy of software maintenance and development processes.
The paper explores non-linear exploration strategies for LLM agents in software change localization.
Summary
The paper investigates different exploration strategies for LLM-based agents in localizing multi-file changes in software repositories, comparing linear and non-linear exploration methods. It introduces a domain-scoped parallel exploration approach and evaluates it against several baselines using benchmarks, finding that domain-scoped agents perform well but face challenges like documentation evolution and test-file over prediction.
Key contributions
- Introduction of a domain-scoped parallel exploration strategy for LLM agents.
- Evaluation of exploration strategies using a persistent-session approach on GitHub issues.
- Identification of challenges such as documentation evolution and test-file over prediction in change localization.
Notable insights
- Domain-scoped parallel agent spawning can outperform linear exploration in multi-file change localization.
- Documentation evolution remains a challenge for accurate change localization across different approaches.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2606.11976v1 Announce Type: cross Abstract: Software engineering tools increasingly rely on LLM based agents to localize files to change to resolve a software issue. Most AI agents explore repositories linearly, that is, visiting one directory or file per step. We postulate that this is a structural mismatch for changes that span several subsystems. We compare linear sequential exploration against non-linear, domain-scoped parallel agentic exploration. Using SWE Bench Pro as initial benchmark, we focus on ansible as an exemplar. We construct an approach for persistent-session evaluation of GitHub issues anchored at a single base commit. We compare our non-linear domain-agent file traversal system against a base LLM without direct repository access, a single agent Recursive Language Model (RLM) baseline with a persistent Python REPL and an external CLI baseline using Codex 5.5 High. Domain scoped parallel agent spawning with a small Haiku-class model achieves the highest micro F1 among Haiku class models by a large margin. Domain-agents is the second highest behind only the much larger Codex 5.5 High on our own expanded benchmark including over more recent PRs from 2025 and 2026. On the original, curated, 2020 SWE-bench Pro benchmark, a larger Sonnet plain LLM baseline attains higher micro F1 by predicting few files, leading to higher precision, but at significantly lower all gold recall. We also present three additional findings. First, documentation evolution is a latent dependency unresolved by any approach. Second, naive file system access can degrade localization driven by test-file over prediction. Lastly, forced multi-agent consultation does not measurably help and raises token cost substantially.