LLMs as ASP Programmers: Self-Correction Enables Task-Agnostic Nonmonotonic Reasoning
Adam Ishay, Joohyung Lee
Why It Matters
What makes this one worth your time
This approach could simplify the deployment of LLMs in complex reasoning tasks by reducing the need for manual task-specific engineering and improving logical consistency.
LLM+ASP leverages nonmonotonic reasoning and self-correction to enhance LLM performance without task-specific tuning.
Summary
The paper introduces 'LLM+ASP,' a framework that translates natural language into Answer Set Programming (ASP) to enable nonmonotonic reasoning without task-specific engineering. It employs an automated self-correction loop using feedback from an ASP solver to iteratively refine outputs, demonstrating improved performance over SMT-based methods across diverse benchmarks.
Key contributions
- Development of a task-agnostic framework for translating natural language into ASP.
- Implementation of an automated self-correction loop for iterative refinement.
- Demonstration of improved performance on nonmonotonic reasoning tasks over SMT-based methods.
Notable insights
- Iterative self-correction can replace handcrafted domain knowledge in reasoning tasks.
- Excessive context can hinder constraint adherence, a phenomenon termed 'context rot.'
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2604.27960v2 Announce Type: replace Abstract: Recent large language models (LLMs) have achieved impressive reasoning milestones but continue to struggle with high computational costs, logical inconsistencies, and sharp performance degradation on high-complexity problems. While neuro-symbolic methods attempt to mitigate these issues by coupling LLMs with symbolic reasoners, existing approaches typically rely on monotonic logics (e.g., SMT) that cannot represent defeasible reasoning -- essential components of human cognition. We present "LLM+ASP," a framework that translates natural language into Answer Set Programming (ASP), a nonmonotonic formalism based on stable model semantics. Unlike prior "LLM+ASP" approaches that require manually authored knowledge modules, domain-specific prompts, or evaluation restricted to single problem classes, our framework operates without any per-task engineering and applies uniformly across diverse reasoning tasks. Our system utilizes an automated self-correction loop where structured feedback from the ASP solver enables iterative refinement. Evaluating across six diverse benchmarks, we demonstrate that: (1) stable model semantics allow LLMs to naturally express default rules and exceptions, outperforming SMT-based alternatives by significant margins on nonmonotonic tasks; (2) iterative self-correction is the primary driver of performance, effectively replacing the need for handcrafted domain knowledge; (3) compact in-context reference guides substantially outperform verbose documentation, revealing a "context rot" phenomenon where excessive context hinders constraint adherence.