Analytica: Soft Propositional Reasoning for Robust and Scalable LLM-Driven Analysis
Junyan Cheng, Kyle Richardson, Peter Chin
Why It Matters
What makes this one worth your time
Improving the reasoning capabilities of LLMs in complex analysis tasks can significantly enhance their applicability and reliability in real-world scenarios, such as financial forecasting and scientific discovery.
Analytica enhances LLM reasoning for complex analyses by reducing bias and variance through structured problem decomposition and robust modeling.
Summary
The paper introduces Analytica, an agent architecture based on Soft Propositional Reasoning (SPR) to improve the reasoning capabilities of large language models (LLMs) in complex real-world analysis tasks. It aims to reduce bias and variance in LLM-driven analysis through a structured decomposition of problems and the use of robust linear models. The architecture is evaluated on economic, financial, and political forecasting tasks, showing improved accuracy and cost-effectiveness.
Key contributions
- Introduction of Soft Propositional Reasoning (SPR) for structured analysis.
- Development of a parallel, divide-and-conquer framework to reduce bias and variance.
- Empirical validation showing improved accuracy and cost-effectiveness in forecasting tasks.
Notable insights
- The use of Soft Propositional Reasoning (SPR) to model and minimize estimation error in LLM-driven analysis.
- The integration of a Jupyter Notebook agent for cost-effective data-driven analysis.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2604.23072v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly tasked with complex real-world analysis (e.g., in financial forecasting, scientific discovery), yet their reasoning suffers from stochastic instability and lacks a verifiable, compositional structure. To address this, we introduce Analytica, a novel agent architecture built on the principle of Soft Propositional Reasoning (SPR). SPR reframes complex analysis as a structured process of estimating the soft truth values of different outcome propositions, allowing us to formally model and minimize the estimation error in terms of its bias and variance. Analytica operationalizes this through a parallel, divide-and-conquer framework that systematically reduces both sources of error. To reduce bias, problems are first decomposed into a tree of subpropositions, and tool-equipped LLM grounder agents are employed, including a novel Jupyter Notebook agent for data-driven analysis, that help to validate and score facts. To reduce variance, Analytica recursively synthesizes these grounded leaves using robust linear models that average out stochastic noise with superior efficiency, scalability, and enable interactive "what-if" scenario analysis. Our theoretical and empirical results on economic, financial, and political forecasting tasks show that Analytica improves 15.84% accuracy on average over diverse base models, achieving 71.06% accuracy with the lowest variance of 6.02% when working with a Deep Research grounder. Our Jupyter Notebook grounder shows strong cost-effectiveness that achieves a close 70.11% accuracy with 90.35% less cost and 52.85% less time. Analytica also exhibits highly noise-resilient and stable performance growth as the analysis depth increases, with a near-linear time complexity, as well as good adaptivity to open-weight LLMs and scientific domains.