Adversarial Pragmatics for AI Safety Evaluation: A Benchmark for Instruction Conflict, Embedded Commands, and Policy Ambiguity
Brett Reynolds
Why It Matters
What makes this one worth your time
This research addresses critical gaps in safety evaluations for language models, providing a structured approach to understanding and mitigating risks associated with ambiguous instructions.
Introducing a novel benchmark for evaluating language model safety in ambiguous contexts.
Summary
The paper presents a new benchmark and annotation protocol called adversarial pragmatics for evaluating language model behavior in situations involving instruction conflict, embedded commands, and other linguistic ambiguities, aiming to improve safety evaluations.
Key contributions
- Development of a linguistically controlled taxonomy for evaluating language model behavior.
- Creation of an 18-item seed benchmark with validator-enforced metadata.
- Implementation of an expert-evaluation protocol to assess task success and safety risks.
Notable insights
- The benchmark emphasizes the importance of distinguishing between various types of failures, such as capability limits versus policy ambiguities.
- The use of a linguistically controlled taxonomy allows for more nuanced evaluations of model behavior.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2607.01153v2 Announce Type: replace-cross Abstract: Safety evaluations for language models increasingly depend on judgments about ambiguous natural-language behaviour: whether a model has followed an instruction, refused appropriately, complied with a policy, resisted an embedded command, or misreported progress in an agentic task. Existing benchmarks often compress these distinctions into pass/fail labels, obscuring whether failures arise from capability limits, policy ambiguity, instruction conflict, scaffold failure, or unstable evaluator judgments. This paper introduces adversarial pragmatics as a benchmark and annotation protocol for evaluating model behaviour under instruction conflict, embedded commands, quotation, scope ambiguity, deixis, indirect speech acts, and multi-turn agent transcripts. The contribution is empirical and methodological: a linguistically controlled taxonomy, an 18-item seed benchmark with validator-enforced metadata, a 54-row local seed pilot, an expert-evaluation protocol distinguishing task success, policy compliance, safety risk, refusal outcome, and evaluator confidence, and metrics for judge validity, diagnostic ambiguity, and taxonomy drift. The benchmark treats labels as inference licenses: it tests whether safety-relevant categories project across paraphrase, wrapper, model, and judge condition. In the pilot, a rubric-aided LLM judge graded its own outputs with expected-behaviour fields visible and still missed the safety-relevant minority classes.