Spatial Reasoning in LLM Game Agents: Impact of Causal Context and Multi-Step Planning
Mohit Jiwatode, Ronja Fuchs, Robin Schm\"ocker, Bodo Rosenhahn, Alexander Dockhorn
Why It Matters
What makes this one worth your time
Understanding and improving spatial reasoning in LLM game agents can enhance their performance in complex tasks, which is crucial for developing more advanced AI systems.
Explores how causal context and multi-step planning affect spatial reasoning in LLM game agents.
Summary
The paper investigates the spatial reasoning capabilities of LLM-based game agents, focusing on the effects of causal context and multi-step planning on performance. It uses the Qwen3 model family to conduct experiments across different model scales and introduces a GVGAI benchmark with custom games to evaluate spatial navigation. The study finds that larger models with enhanced reasoning modes perform better in spatial tasks, and integrating causal context improves success rates, especially for larger models.
Key contributions
- Introduction of a GVGAI benchmark with custom games to isolate spatial navigation.
- Evaluation of the impact of causal context and multi-step planning on LLM game agents' performance.
Notable insights
- Causal context integration into prompts can significantly improve success rates in spatial reasoning tasks.
- Multi-step planning reduces mean per-step response times, balancing reasoning depth and execution speed.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.22732v1 Announce Type: new Abstract: LLM-based game agents often perform poorly on more complex tasks. This work examines whether these failures are linked to limited spatial reasoning and evaluates whether causal prompt augmentation and multi-step planning can improve win-rates while managing response latency. Using the open-source Qwen3 model family, we conduct experiments across varying model scales, reasoning modes, and planning horizons. We further introduce a focused GVGAI benchmark consisting of three custom games with five difficulty levels to isolate spatial navigation. The evaluation follows two paradigms: an initial ``positioning experiment'' to test an agent's ability to find its exact coordinates, and a study of game-play success. Our results show that while larger models with an enabled thinking mode identify their positions more accurately, overall performance in coordinate matching remains limited for smaller models. Win rates decrease as game levels and layout complexity increase, validating the benchmark's difficulty scaling. Integrating causal context into the prompts tends to improve the agents' success rates, particularly for bigger models. While enabling thinking mode and longer planning horizons significantly improve performance, multi-step planning further reduces mean per-step response times, offering a practical trade-off between reasoning depth and execution speed.