ProofCouncil: An LLM Agent for Solving Open Mathematical Problems
Johannes Schmitt, Tim Gehrunger, Jasper Dekoninck, Gergely B\'erczi, Uri Kreitner, Liam Price, David Holmes
Why It Matters
What makes this one worth your time
This work advances the application of AI in solving complex mathematical problems, potentially aiding mathematicians and researchers in exploring unsolved questions.
ProofCouncil is an LLM agent that autonomously tackles open mathematical problems with notable success.
Summary
The paper introduces ProofCouncil, a large language model-based agent designed to solve open mathematical problems using an author-critic architecture. It demonstrates competitive performance in a mathematical challenge and provides partial solutions to additional open problems. The development and the agent-building library are released as open source.
Key contributions
- Development of ProofCouncil, an LLM-based mathematical problem-solving agent.
- Introduction of an author-critic architecture for improved problem-solving.
- Release of an open-source agent-building library.
Notable insights
- The use of an author-critic architecture to enhance LLM performance in mathematical problem-solving.
- Evaluation of the agent on real-world mathematical challenges and open problems.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.09474v1 Announce Type: new Abstract: Large language models (LLMs) have shown increasing promise in solving open problems in mathematics. However, their performance can be further improved through agentic workflows tailored to real-world mathematical practice. To this end, we introduce ProofCouncil, a mathematical agent that is designed to tackle open problems using an author-critic architecture. ProofCouncil served as a submission to the second batch of FirstProof, a challenge consisting of 10 real-world mathematical problems that agents must solve autonomously. Its submissions for 6 of the 10 problems were judged by the referees to be correct up to at most minor revisions, showing the best performance among participating teams. We also evaluate ProofCouncil on 30 open problems collected from mathematical researchers. Among the 21 solutions that received human feedback, 5 were judged completely correct, 2 more were judged promising pending final verification, and a further 8 contained useful partial progress. In this short paper, we describe the development of ProofCouncil and the agent-building library used to create it, which we release as open source to the community.