Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents
Xu Li, Simon Yu, Minzhou Pan, Yiyou Sun, Bo Li, Dawn Song, Xue Lin, Weiyan Shi
Why It Matters
What makes this one worth your time
As LLM-based agents become more capable, understanding and mitigating their safety risks in complex interactions is crucial for responsible deployment in real-world applications.
This research benchmarks and mitigates safety risks in multi-turn interactions of tool-using agents.
Summary
The paper introduces a new taxonomy for evaluating safety risks in multi-turn interactions of tool-using agents and presents the MT-AgentRisk benchmark, revealing significant safety degradation in these settings. It also proposes ToolShield, a defense mechanism that autonomously generates test cases to improve safety outcomes.
Key contributions
- Development of the MT-AgentRisk benchmark for evaluating multi-turn tool-using agent safety.
- Introduction of a taxonomy for transforming single-turn tasks into multi-turn attack sequences.
- Proposal of ToolShield, a training-free, tool-agnostic defense mechanism for enhancing agent safety.
Notable insights
- The introduction of a taxonomy that transforms single-turn harmful tasks into multi-turn attack sequences is a novel approach to systematically assess safety.
- ToolShield's self-exploration mechanism for safety testing represents an innovative method for agents to autonomously improve their safety protocols.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2602.13379v2 Announce Type: replace-cross Abstract: LLM-based agents are becoming increasingly capable, yet their safety lags behind. This creates a gap between what agents can do and should do. This gap widens as agents engage in multi-turn interactions and employ diverse tools, introducing new risks overlooked by existing benchmarks. To systematically scale safety testing into multi-turn, tool-realistic settings, we propose a principled taxonomy that transforms single-turn harmful tasks into multi-turn attack sequences. Using this taxonomy, we construct MT-AgentRisk (Multi-Turn Agent Risk Benchmark), the first benchmark to evaluate multi-turn tool-using agent safety. Our experiments reveal substantial safety degradation: the Attack Success Rate (ASR) increases by 16% on average across open and closed models in multi-turn settings. To close this gap, we propose ToolShield, a training-free, tool-agnostic, self-exploration defense: when encountering a new tool, the agent autonomously generates test cases, executes them to observe downstream effects, and distills safety experiences for deployment. Experiments show that ToolShield effectively reduces ASR by 30% on average in multi-turn interactions. Our code is available at https://github.com/CHATS-lab/ToolShield.