MILD: Mediator Agent System with Bidirectional Perception and Multi-Layered Alignment for Human-Vehicle Collaboration
Jiyao Wang, Yunbiao Wang, Yubo Jiao, Xiao Yang, Dengbo He, Sasan Jafarnejad, Luis Miranda-Moreno, Raphael Frank, Jiangbo Yu
Why It Matters
What makes this one worth your time
This research addresses critical challenges in human-vehicle collaboration, potentially improving safety and user experience in partially automated driving systems.
MILD transforms human-vehicle interaction by making drivers active managers through advanced perception and strategy alignment.
Summary
The paper introduces the Mediator-in-the-Loop-Driving (MILD) system, which enhances human-vehicle collaboration by integrating a perception agent and a strategy agent that generates explainable action suggestions, while ensuring alignment with safety regulations and human values through Evidence- and Constraint-weighted Policy Optimization (ECPO).
Key contributions
- Development of the MILD system architecture for enhanced human-vehicle collaboration.
- Introduction of ECPO for aligning agent strategies with safety and human values.
- Demonstration of improved performance in perception accuracy and strategy quality through field experiments.
Notable insights
- The integration of a perception agent for both in-cabin and out-of-cabin understanding is a novel approach to enhance situational awareness.
- The use of Evidence- and Constraint-weighted Policy Optimization (ECPO) to align agent behavior with human values and safety regulations is a significant methodological advancement.
Possible limitations
- Potential challenges in real-time implementation and scalability of the proposed system in diverse driving environments.
- Not stated in the abstract.
Abstract
arXiv:2605.01507v1 Announce Type: new Abstract: Prior studies report that partial driving automation can increase the cognitive demands on human drivers. This effect largely arises from human drivers' lack of transparent insight into the vehicle's intentions and decision logic, as well as from automated systems' limited awareness of the driver's dynamic state and preferences. This bidirectional misalignment undermines shared situational awareness and exacerbates coordination failures in human-vehicle interaction. To address these limitations, we argue for a paradigm shift that elevates the human role from passive supervisor to active manager. We introduce the Mediator-in-the-Loop-Driving (MILD) system, based on an agentic system architecture to facilitate synergistic human-vehicle collaboration. MILD integrates a perception agent for joint in-cabin and out-of-cabin understanding with a lightweight strategy agent that generates compliant and explainable action suggestions. To ensure these strategies are strictly aligned with safety regulations and human values, we develop Evidence- and Constraint-weighted Policy Optimization (ECPO). ECPO leverages automatic validators to steer the agent toward behaviors that are not only accurate but also structurally complete, substantiated by evidence, and free from constraint violations. Furthermore, a retrieval-augmented generation module dynamically incorporates constraints from traffic regulations, speed recommendations, and driver preferences into the decision loop. Field experiments across three open datasets demonstrate that MILD consistently outperforms baselines in both perception accuracy and strategy quality under auditable offline metrics, and yields higher human-rated policy adequacy, comfort, and explanation than baselines. This work offers a practical pathway for building auditable and aligned agents for human-vehicle collaborative driving.