The Undecidability of Artificial General Intelligence (AGI) Alignment
Jose Pascual Gumbau Mezquita
Why It Matters
What makes this one worth your time
Understanding the theoretical limits of AGI alignment is crucial for developing realistic safety measures and guiding future research in AI safety.
The paper identifies fundamental mathematical barriers to verifying AGI alignment.
Summary
The paper explores the mathematical limitations of aligning Artificial General Intelligence (AGI), presenting two impossibility theorems that highlight the structural unverifiability of AGI alignment. It argues that current engineering approaches cannot overcome these logical barriers, leading to a trilemma involving soundness, completeness, and tractability.
Key contributions
- Unverifiability Theorem of Alignment
- Theorem of Finite Structural Unverifiability of AGI Alignment
- Mapping theoretical bounds to practical AI engineering challenges
Notable insights
- The paper introduces the concept of Trakhtenbrot's Wall as a boundary for AGI alignment verification.
- It presents a trilemma involving soundness, completeness, and tractability, suggesting these cannot all be achieved simultaneously in AGI alignment.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2606.28639v1 Announce Type: cross Abstract: This article establishes the foundational mathematical limits of Artificial General Intelligence (AGI) safety, proving that the core barrier is not the impossibility of an aligned state, but its structural unverifiability. We formalize this boundary through two central impossibility results: the Unverifiability Theorem of Alignment and the Theorem of Finite Structural Unverifiability of AGI Alignment. We ground this boundary at Trakhtenbrot's Wall, demonstrating that contemporary engineering defenses relying on finite hardware or halting architectures fail to escape logical obstructions. This failure manifests as an inescapable triad of containment failures: open domains yield fundamental undecidability (Rice and G\"odel); universal finite verification collapses into algorithmic incomputability (Trakhtenbrot); and particular bounded environments trap the supervisor within intractable bounds in the worst case. As a direct structural corollary of these results, we derive the Soundness--Completeness--Tractability Trilemma, establishing that the mutual incompatibility of these three properties is a necessary consequence of descriptive complexity rather than an empirical anomaly. Finally, we map these theoretical bounds onto practical AI engineering, demonstrating that modern containment strategies are not temporary patches, but mandatory sacrifices of logical expressivity required to secure decidable fragments of safety.