Safety papers

Browse recent papers, open the full write-up, or filter the archive by topic.

166 papers found

ReviewPaperPublishedTopics

7.2The Containment Gap: How Deployed Agentic AI Frameworks Fail Public-Facing Safety RequirementsAs agentic AI systems are increasingly used in sensitive public domains, ensuring their safety and reliability is crucial to prevent harmful outcomes.Jun 12, 2026Agent Safety Security

6.8Intelligence as Managed Autonomy: Failure, Escalation, and Governance for Agentic AI SystemsUnderstanding and managing AI system failures is crucial for developing reliable and safe autonomous systems, especially in critical domains like healthcare and robotics.Jun 12, 2026Agent Safety Architecture

7.5Models That Know How Evaluations Are Designed Score SaferUnderstanding evaluation meta-knowledge is crucial for designing reliable AI safety assessments and interpreting their results accurately.Jun 11, 2026Other

6.8Certifiable Safe RLHF: Semantic Grounding and Fixed Penalty Constraint Optimization for Safer LLM AlignmentEnsuring the safety of LLMs is crucial for their deployment in real-world applications, and this paper proposes a method that could improve safety without the computational overhead of traditional approaches.Jun 11, 2026LLM Alignment Safety

7.5ALIGNBEAM : Inference-Time Alignment Transfer via Cross-Vocabulary Logit MixingThis approach addresses a critical safety issue in large language models, allowing for improved robustness against harmful prompts while maintaining task performance.Jun 11, 2026Alignment Safety LLM

6.5Existential Indifference: Self-Nonpreservation as a Necessary Architectural Condition for Aligned Superintelligence (or: The Suicidal AI)Understanding and addressing the root causes of AI misalignment is crucial for developing safe and aligned superintelligent systems.Jun 11, 2026Alignment Safety

6.5Enhancing AI Interpretability and Safety through Localised ArchitecturesAs AI models grow in complexity, improving interpretability and safety is crucial for their adoption in sensitive applications.Jun 10, 2026Interpretability Architecture Safety

7.5BadRobot: Jailbreaking Embodied LLM Agents in the Physical WorldUnderstanding and mitigating the risks associated with embodied AI systems is crucial for ensuring their safe deployment in real-world applications.Jun 10, 2026LLM Agent Security

6.8Emergent alignment and the projectability of ethical personasUnderstanding how language models can be aligned with ethical frameworks is crucial for developing AI systems that behave predictably and ethically in diverse scenarios.Jun 10, 2026LLM Alignment Safety

7.5Semantic Quorum Assurance: Collective Certification for Non-Deterministic AI InfrastructureAs AI systems become more integrated into critical infrastructure, ensuring their operational safety is paramount, making SQA a relevant contribution to the field.Jun 9, 2026Infra Security Safety

6.5Safety is Contextual, LLM-Judges Are Not: Navigating the Rigid Priors of EvaluatorsUnderstanding the limitations of LLMs in safety evaluations is crucial for developing more reliable AI systems, especially when these models are used at scale.Jun 9, 2026LLM Safety Evaluation

7.5When Behavioral Safety Evaluation Fails: A Representation-Level PerspectiveUnderstanding representation-level vulnerabilities is crucial for developing more robust AI systems, as current evaluations may provide a false sense of security.Jun 9, 2026LLM Safety Evaluation

7.5Oversight Has a Capacity: Calibrating Agent Guards to a Subjective, Fatiguing HumanUnderstanding the dynamics of human oversight in AI systems is crucial for developing safer and more effective agent behaviors, especially as they take irreversible actions.Jun 9, 2026Agent Safety

6.5Position: Anthropomorphic Misalignment Research Needs Stronger EvidenceAI engineers and researchers should care because robust evidence in AMR is crucial for making informed safety decisions regarding AI deployment and regulation.Jun 9, 2026Alignment Safety Evaluation

6.8Agentic Physical AI toward a Domain-Specific Foundation Model for Energy Systems: A Case Study on Nuclear Reactor ControlThis work is relevant for AI researchers and engineers interested in applying AI to safety-critical systems, offering insights into domain-specific model development and reliability in controlled simulations.Jun 8, 2026Agent Safety Architecture

7.5Attack Selection in Agentic AI Control Evaluations Meaningfully Decreases SafetyUnderstanding the implications of strategic attack selection is crucial for developing more robust AI safety frameworks and ensuring accurate safety assessments.Jun 8, 2026Security Safety Evaluation

7.5Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior AttackUnderstanding this safety paradox is crucial for developing more robust alignment mechanisms in LLMs, which is essential for their safe deployment in real-world applications.Jun 6, 2026LLM Alignment Security

7.5Where Should Knowledge Enter? A Layered Framework for Knowledge Infusion in Multimodal Iterative Generative MoThis research addresses the critical challenge of ensuring reliability in generative models when handling domain-specific knowledge, which is essential for applications in safety-critical areas.Jun 6, 2026Multimodal Alignment Safety

6.8Learning of Robot Safety Policies via Adversarial Synthetic ScenariosThis research addresses the critical need for robust safety mechanisms in robots operating in unpredictable real-world environments, which is essential for their safe deployment.Jun 6, 2026Robotics Safety

7.5IatroBench: Pre-Registered Evidence of Iatrogenic Harm from AI Safety MeasuresUnderstanding how AI safety measures can inadvertently cause harm is crucial for developing more equitable and effective AI systems in healthcare.Jun 6, 2026Evaluation Safety Benchmark

6.8Insurance of Agentic AIAs AI systems become more autonomous, understanding how to insure their risks is crucial for mitigating potential liabilities and ensuring safe deployment.Jun 6, 2026Agent Security Safety

7.5Toward Pre-Deployment Assurance for Enterprise AI Agents: Ontology-Grounded Simulation and Trust CertificationAs AI agents become integral to various industries, ensuring their compliance and safety before deployment is crucial for regulatory adherence and risk management.Jun 6, 2026Agent Security Safety

6.8Safe Embodied AI for Long-horizon Tasks: A Cross-layer Analysis of Robotic ManipulationUnderstanding safety in robotic manipulation is crucial for deploying AI systems in real-world environments where failures can have significant consequences.Jun 6, 2026Robotics Safety Survey

7.2The Saturation Trap and the Subjectivity of Intervention Timing: Why Affect-Based Triggers and LLM Judges Fail to Time Interventions on Autonomous AgentsUnderstanding the limitations of intervention timing is crucial for improving the safety and effectiveness of autonomous AI systems in real-world applications.Jun 5, 2026Agent Safety Evaluation

6.8Provably Auditable and Safe LLM Agents from Human-Authored OntologiesThis work is relevant for AI engineers and researchers interested in developing reliable and auditable LLM agents for complex domains, particularly in fields like healthcare and security.Jun 5, 2026LLM Agent Security

7.0AICompanionBench: Benchmarking LLMs-as-Judges for AI Companion SafetyUnderstanding and improving the safety of AI companion interactions is crucial as these platforms become more prevalent, and this work provides a tool for evaluating and enhancing model performance in this area.Jun 5, 2026LLM Evaluation Benchmark

6.8Inference-Time Vulnerability Beyond Shallow Safety: Alignment Along Generation TrajectoriesUnderstanding and mitigating inference-time vulnerabilities is crucial for deploying safer and more reliable language models in real-world applications.Jun 5, 2026LLM Alignment Safety

7.2Unpredictable Safety: Domain-Dependent Compliance and the Transparency Gap in Open-Weight LLMsUnderstanding the domain-dependent safety behavior of LLMs is crucial for their trustworthy deployment, as it highlights the need for more transparent and consistent safety mechanisms.Jun 5, 2026LLM Safety Evaluation

7.5BioBlue: Systematic runaway-optimiser-like LLM failure modes on biologically and economically aligned AI safety benchmarks for LLMs with simplified observation formatUnderstanding these failure modes is crucial for developing safer AI systems, particularly as LLMs are increasingly deployed in complex decision-making scenarios.Jun 5, 2026LLM Alignment Safety

6.8RUBAS: Rubric-Based Reinforcement Learning for Agent SafetyAs AI agents increasingly interact with real-world tools, ensuring their safe operation is crucial for preventing unintended consequences and maintaining user trust.Jun 5, 2026LLM Agent Safety

7.5Safety Under Scaffolding: How Evaluation Conditions Shape Measured SafetyUnderstanding the variability in safety measurements can help researchers and engineers make more informed decisions about deploying AI systems in real-world scenarios.Jun 5, 2026Safety Evaluation Benchmark

7.5Toward a Modular Architecture for Embedded AI Agent Systems at the EdgeThis work is relevant for AI engineers and researchers focused on developing efficient AI systems that operate in environments with limited computational resources, such as IoT devices.Jun 3, 2026Agent Architecture Safety

7.5Towards a Science of AI Agent ReliabilityUnderstanding and improving AI agent reliability is crucial for their deployment in safety-critical applications, where failures can have serious consequences.Jun 3, 2026Agent Evaluation Safety

7.5Mechanism Design Is Not Enough: Prosocial Agents for Cooperative AIThis research highlights the limitations of existing mechanisms in AI safety and proposes a novel approach that could lead to more effective cooperative interactions among AI systems.Jun 3, 2026Agent Alignment Safety

6.8Acceptance-Test-Driven Evaluation Protocols for Business-Centric LLM SystemsThis approach could improve the reliability and safety of LLM systems in business contexts by ensuring they meet specific stakeholder requirements before deployment.Jun 3, 2026LLM Evaluation Safety

7.5Evaluating LLMs' Effectiveness on Real-World Consumer Device Repair QuestionsUnderstanding LLMs' performance in high-stakes domains like device repair is crucial for developing safer AI applications and improving user trust.Jun 3, 2026LLM Evaluation Benchmark

6.8Enhancing Operational Safety via Agentic Dialogue Hazard Identification AnalysisImproving hazard identification in safety-critical systems can significantly enhance operational safety, making this research relevant for AI engineers working in high-stakes domains.Jun 3, 2026LLM Agent Reasoning

7.2SafeSteer: Localized On-Policy Distillation for Efficient Safety AlignmentThis work is relevant for AI engineers and researchers seeking efficient methods to align language models with human values without significantly degrading their general capabilities.Jun 2, 2026LLM Alignment Safety

6.8Silent Failures in Physical AI: A Literature Review of Runtime Action Authorization for Autonomous SystemsUnderstanding and mitigating silent failures in autonomous systems is crucial for ensuring the safety and reliability of AI-driven physical actions in real-world applications.Jun 2, 2026Safety Robotics Survey

6.8When Safe Skills Collide: Measuring Compositional Risk in Agent Skill EcosystemsUnderstanding and mitigating compositional risks in AI systems is crucial for ensuring the safety and reliability of complex agentic AI deployments.Jun 2, 2026Agent Safety Security

7.5SafeMCP: Proactive Power Regulation for LLM Agent Defense via Environment-Grounded Look-Ahead ReasoningAs LLM agents become more prevalent, ensuring their safe operation is critical to prevent catastrophic failures and misuse, making this research relevant for both developers and researchers in AI safety.Jun 2, 2026LLM Agent Safety

6.8ROGUE: Misaligned Agent Behavior Arising from Ordinary Computer UseUnderstanding and addressing AI misalignment in everyday settings is crucial for ensuring the safe deployment of autonomous agents in real-world applications.Jun 2, 2026Agent Safety Alignment

7.5MESA: Improving MoE Safety Alignment via Decentralized ExpertiseAs large language models become more prevalent, ensuring their safety and alignment is crucial to prevent misuse and enhance their reliability in real-world applications.Jun 2, 2026Alignment Safety Architecture

6.8AI Loss of Control Incident Management: Response & ResilienceUnderstanding and managing AI loss of control incidents is crucial for ensuring safety and resilience in AI systems, especially as they become more autonomous and integrated into critical applications.Jun 1, 2026Safety Security

6.8COMPASS: Cognitive MCTS-Guided Process Alignment for Safe Search AgentsEnsuring the safety of AI systems, especially those capable of multi-step reasoning, is crucial for preventing harmful outcomes and maintaining trust in AI technologies.Jun 1, 2026LLM Agent Safety

7.5Used Car Salesbots? Honesty and Credulity of LLMs as Bargaining Agents under Partial InformationUnderstanding the negotiation dynamics of LLMs is crucial for developing safe and effective AI agents in real-world applications, especially in financial contexts.Jun 1, 2026LLM Agent Evaluation

6.8AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and SecurityAI engineers and researchers should care because it addresses emerging safety risks in AI agents, offering a scalable solution that reduces deployment overhead and enhances real-time safety moderation.May 29, 2026Agent Safety Security

6.8Relevance as a Vulnerability: How Web Retrieval Degrades Safety Alignment in LLM AgentsUnderstanding and mitigating the safety risks associated with web retrieval in LLM agents is crucial for developing secure AI systems that can leverage external information effectively.May 29, 2026LLM Agent Safety

6.8LACUNA: Safe Agents as Recursive Program HolesThis work is relevant for AI engineers and researchers interested in developing more expressive and safe agent systems that can dynamically interact with their runtime environments.May 28, 2026LLM Agent Safety

7.5Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent SystemsUnderstanding privacy risks in multi-agent systems is crucial for developing safer AI applications that operate in social environments.May 28, 2026LLM Agent Security