Provably Auditable and Safe LLM Agents from Human-Authored Ontologies
Aaron Sterling
Why It Matters
What makes this one worth your time
This work is relevant for AI engineers and researchers interested in developing reliable and auditable LLM agents for complex domains, particularly in fields like healthcare and security.
Agentic Redux ensures auditable and semantically correct LLM agent operations using human-authored ontologies.
Summary
The paper introduces the Agentic Redux architecture for LLM agents, which ensures linear auditability and correctness in problem domains by using typed lambda calculus and an append-only ledger. It also presents Ontology-First Agent Design, a methodology where human experts create ontologies for problem domains, allowing LLMs to define roles for agents and humans. The approach is demonstrated in healthcare billing compliance and security vulnerability disclosure.
Key contributions
- Introduction of the Agentic Redux architecture for auditable LLM agents.
- Demonstration of the architecture in healthcare billing compliance and security vulnerability disclosure.
- Development of Ontology-First Agent Design methodology.
Notable insights
- The use of typed lambda calculus to guarantee semantic correctness in LLM agent operations.
- Ontology-First Agent Design leverages human expertise to structure problem domains for LLMs.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2606.04903v1 Announce Type: cross Abstract: We introduce the LLM agent architecture Agentic Redux, intended for use with nontrivial problem domains that require linear auditability. Using the typed lambda calculus, we prove that, run on appropriate domains, Agentic Redux executions are semantically guaranteed to be correct, with all decisions recorded in an append-only ledger. We present two production-grade appropriate domains, in healthcare billing compliance, and security vulnerability disclosure. Working code for Agentic Redux run on both domains is available in a supporting code repository. We also introduce Ontology-First Agent Design, a methodology for creation of agent frameworks on a problem domain, in which a human expert ontologizes the problem domain with Basic Formal Ontology, and then assigns an LLM to derive roles that agents and humans-in-the-loop can fill, in order to work the problems in the domain.