When Does Belief-Based Agent Memory Help? Reliability-Conditional Updating and Provenance-Capped Poisoning Defense
Pranav Singh
Why It Matters
What makes this one worth your time
Understanding how to improve memory in LLMs is crucial for developing more reliable and trustworthy AI systems, especially in applications requiring nuanced reasoning over conflicting information.
This research enhances LLM memory by addressing reliability and provenance in belief updating.
Summary
The paper investigates the effectiveness of belief-based memory in large language model agents, introducing reliability-conditioned updating and provenance-capped belief updating to enhance memory performance in the presence of conflicting evidence and to defend against memory poisoning attacks.
Key contributions
- Development of a long-term memory architecture called Nous that utilizes Bayesian inference for belief updating.
- Introduction of reliability-conditioned updating to improve memory performance in the presence of unreliable evidence.
- Proposal of provenance-capped belief updating to mitigate memory poisoning attacks.
Notable insights
- The introduction of reliability-conditioned updating based on epistemic language is a novel approach to enhance memory accuracy.
- Provenance-capped belief updating offers a unique defense mechanism against memory poisoning, which is a growing concern in AI security.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2606.22030v2 Announce Type: replace Abstract: We investigate when belief-based memory actually improves large language model (LLM) agents. Our vehicle is Nous, a long-term memory architecture that represents each entity-attribute pair as a categorical probability distribution updated through closed-form Bayesian inference, with information-theoretic surprise driving belief revision and entropy-based forgetting. A controlled ablation on the LoCoMo benchmark shows that Bayesian belief updating alone provides little benefit over naive last-write-wins because existing conversational memory benchmarks rarely contain contradictory or differently reliable evidence. We then introduce reliability-conditioned updating, estimating per-observation reliability from epistemic language, and show on a controlled contradiction benchmark that belief updating substantially outperforms last-write-wins and raw-memory retrieval when observations differ in trustworthiness. Because content-derived reliability is itself vulnerable to manipulation, we further propose provenance-capped belief updating, where trust is bounded by source provenance rather than textual confidence. Under controlled memory-poisoning experiments, this approach resists volumetric poisoning attacks while revealing the utility costs and implementation requirements of provenance-aware memory. Finally, we quantify a 27.5-point discrepancy between strict token-F1 and LLM-as-judge evaluation on identical outputs, highlighting important reproducibility concerns for long-term memory benchmarks. Our results suggest that probabilistic belief-based memory is most beneficial in environments requiring reasoning over conflicting and differently trustworthy evidence, rather than conventional conversational recall alone.