Whose Side Is Your Agent On? Multi-Party Principal Loyalty in LLM Agents
Bojie Li, Noah Shi
Why It Matters
What makes this one worth your time
Understanding and managing the loyalty of LLM agents in multi-party interactions is crucial for developing reliable AI systems that can effectively represent their principals' interests.
The paper introduces a benchmark and mechanisms to tackle the loyalty challenge in multi-party LLM agents.
Summary
The paper addresses the challenge of multi-party loyalty in LLM agents, proposing a benchmark called PrincipalBench to evaluate agent loyalty and introducing two mechanisms to manage the trade-off between loyalty and refusal of adversarial requests.
Key contributions
- Development of PrincipalBench, a 75-item multi-turn benchmark for evaluating agent loyalty.
- Introduction of a prompt-time loyalty scaffold to manage agent behavior.
- A per-token-KL distillation recipe for transferring knowledge between models.
Notable insights
- The paper identifies a trade-off between loyalty and over-refusal in LLM agents, suggesting that improving one aspect may worsen the other.
- The use of a prompt-time loyalty scaffold with prioritized rules is a novel approach to managing agent behavior.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2606.30383v1 Announce Type: new Abstract: A rapidly growing class of LLM agents is multi-party: the agent acts for a principal (who briefs it, sends follow-ups, and receives results) while also conversing in a separate channel with a counterparty whose interests may diverge (negotiating with a vendor, screening inbound requests, or mediating between employees). Here "help whoever you are talking to" is the wrong objective. The agent must stay loyal to the principal it represents without over-refusing the principal's own cooperative asks. We study this multi-party loyalty problem and contribute a measurement instrument, two mechanisms, and a structural lesson. PrincipalBench is a 75-item multi-turn benchmark with leak probes, dual judges, and an integrity-audit gate. Across 13 frontier subjects it exposes a sharp split (<=20% vs. 53.6-75.3% harm) invisible to single-turn safety evaluations: a selective cluster that declines adversarial probes while still following the principal's legitimate requests, and an over-refusing cluster that refuses broadly. (M1) A prompt-time loyalty scaffold (a fixed system prompt of seven prioritized rules, open-coded from 50+ failure trajectories) holds Claude-Sonnet to 19.4% harm and all nine selective subjects to <=20%. (M2) A per-token-KL distillation recipe transfers a prompted Qwen3-32B teacher into 8B Qwen3 and Llama-3.1 students, the strongest open-weight recipe we measure. (Lesson) Both mechanisms only move along a common leak/over-refusal trade-off rather than crossing it: improving one axis costs the other, and the jointly favorable outcome stays out of reach.