JANUS: Foreseeing Latent Risk for Long-Horizon Agent Safety
Yuan Xiong, Linji Hao, Shizhu He, Yequan Wang, Lijun Li
Why It Matters
What makes this one worth your time
Improving agent safety by predicting and preventing risks can significantly enhance the reliability and trustworthiness of AI systems in real-world applications.
Janus framework anticipates and mitigates long-term risks in agent trajectories to enhance safety.
Summary
The paper introduces Janus, a framework designed to enhance long-horizon agent safety by predicting and preventing potential risks in agent trajectories. It employs a dual-task approach with anticipation and adjudication tasks, optimized using CoAA-RL, to improve safety judgments. The resulting model, Vanguard, demonstrates improved safety and task completion across benchmarks.
Key contributions
- Introduction of Janus, a foresight-oriented framework for agent safety.
- Development of Vanguard, a guard model that blocks unsafe actions based on predictions.
- Demonstrated improvement in safety and task completion across benchmarks.
Notable insights
- The use of a dual-task approach combining anticipation and adjudication for safety forecasting.
- Optimization with CoAA-RL to align forecasts with downstream safety judgments.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.19913v1 Announce Type: new Abstract: Agent safety is moving from content moderation toward preventing operational failures before tool-using agents act. We propose Janus, a foresight-oriented framework for long-horizon agent safety that trains guards to anticipate delayed risks from partial trajectories. Janus synthesizes diverse agent trajectories via multi-agent simulation and learns a shared policy with two coupled tasks: an anticipation task that forecasts safety-relevant futures and an adjudication task that decides safety from both the observed prefix and anticipated future. The two tasks are jointly optimized with CoAA-RL, which rewards forecasts by their utility for downstream safety judgment. The resulting guard model, Vanguard, blocks unsafe actions before execution. Across four agent-safety benchmarks, Vanguard improves average protection by 15.9 percentage points over baseline guards while increasing benign task completion by 5.1 percentage points.