Perceive Before Reasoning: A Pre-Reasoning Perception Framework for Efficient and Reliable Proactive Mobile Agents
Zhijie Ding (HyperAI Team, Xiaomi Corporation, Zhongnan University of Economics and Law), Weinan Hong (HyperAI Team, Xiaomi Corporation, Jilin University), Zicheng Zhu (HyperAI Team, Xiaomi Corporation, The Chinese University of Hong Kong, Shenzhen), Lei Li (HyperAI Team, Xiaomi Corporation), Dezhi Kong (HyperAI Team, Xiaomi Corporation), Hao Wang (HyperAI Team, Xiaomi Corporation), Peng Zhou (HyperAI Team, Xiaomi Corporation), Xuchu Jiang (HyperAI Team, Xiaomi Corporation), Jiaming Xu (HyperAI Team, Xiaomi Corporation)
Why It Matters
What makes this one worth your time
This framework could improve the efficiency and reliability of mobile agents by reducing unnecessary interventions and optimizing resource use, which is crucial for real-time applications.
PRPF enhances proactive mobile agents by separating intervention timing from assistance generation.
Summary
The paper proposes a two-stage framework called the Pre-Reasoning Perception Framework (PRPF) for proactive mobile agents, which separates the decision of when to intervene from how to assist. It introduces a Multimodal Proactive Perceptor (MPP) for intervention gating and context compression, activating the Proactive Agent Reasoner (PAR) only when necessary, thereby reducing false trigger rates and improving success rates and inference efficiency.
Key contributions
- Introduction of the Pre-Reasoning Perception Framework (PRPF) for mobile agents.
- Development of a Multimodal Proactive Perceptor (MPP) for efficient intervention gating.
- Demonstration of improved performance on the ProactiveMobile benchmark.
Notable insights
- Separating the decision-making process into perception and reasoning stages can reduce computational redundancy.
- Using a lightweight perceptor for initial intervention gating can improve system efficiency.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2606.03236v1 Announce Type: new Abstract: Multimodal large language models (MLLMs) have substantially advanced mobile agents, yet proactive mobile assistance remains challenging because agents must decide \emph{when} to intervene before determining \emph{how} to assist. Existing systems often implement these two decisions within a unified MLLM-based pipeline, leading to goal misalignment between conservative intervention filtering and comprehensive assistance generation, as well as redundant inference when the agent should remain silent. To address these limitations, we propose the \textbf{Pre-Reasoning Perception Framework (PRPF)}, a two-stage framework built on perceiving before reasoning. PRPF introduces a lightweight Multimodal Proactive Perceptor (MPP) for intervention gating and context compression, and activates the Proactive Agent Reasoner (PAR) only when intervention is warranted. Experiments on the ProactiveMobile benchmark show that PRPF substantially reduces false trigger rates (FTR) while improving success rates (SR) and inference efficiency over the ProactiveMobile baseline.