Think Short, Defer Smart, Act, and Repeat: Calibrated Reasoning and Uncertainty-Aware Deferral for Edge LLM Agents
Amirmohammad Farzaneh, Osvaldo Simeone
Why It Matters
What makes this one worth your time
This work addresses the critical challenge of deploying LLM agents in resource-constrained environments, making it relevant for practical applications in robotics and real-time decision-making systems.
TSDS optimizes edge LLM agent reasoning by balancing local computation and cloud deferral.
Summary
The paper presents the Think Short, Defer Smart (TSDS) framework, which integrates a convergence probe and a perplexity-based deferral rule to optimize reasoning budgets for edge LLM agents, enhancing their performance on multi-step tasks while managing uncertainty.
Key contributions
- Introduction of the TSDS framework for edge LLM agents.
- Development of a lightweight convergence probe for on-device reasoning.
- Establishment of a perplexity-based deferral rule for cloud-side model escalation.
Notable insights
- The integration of a convergence probe with a perplexity-based deferral rule is a novel approach to managing uncertainty in LLM agents.
- The multi-objective Learn-Then-Test procedure provides a robust calibration mechanism for balancing performance metrics.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2607.26865v1 Announce Type: cross Abstract: LLM agents following the ReAct paradigm are promising enablers of complex multi-step tasks, including multi-hop question answering, code generation, and control of physical AI systems. Yet, when deployed at the edge, they must tightly manage their reasoning budget while remaining reliable and deferring to a cloud-side model only when local uncertainty is too high to act safely. We propose Think Short, Defer Smart (TSDS), a framework that synergistically integrates a lightweight convergence probe, which halts on-device reasoning once the intended action has stabilized, with a perplexity-based deferral rule that escalates uncertain actions to a cloud-side model. Both mechanisms are jointly calibrated on end-to-end episode trajectories via a multi-objective Learn-Then-Test (LTT) procedure, providing simultaneous finite-sample guarantees on expected episode reward and cloud-call rate. We evaluate TSDS on four ReAct benchmarks spanning arithmetic reasoning (GSM8K), multi-hop question answering (HotpotQA), code generation (MBPP), and multi-step embodied planning (household robot), and compare against thought-calibration-only and calibrated-deferral-only standalone baselines. TSDS reduces per-episode thinking compute by 43%-73% over deferral-only baselines across HotpotQA, MBPP, and the household robot task, while maintaining certified reward and cloud-call rate guarantees.