Enhancing Small Language Models Reasoning through Knowledge Graph Grounding
Dimitrios Kelesis, Konstantinos Bougiatiotis, Georgios Paliouras
Why It Matters
What makes this one worth your time
This research addresses the limitations of small language models in reasoning tasks, offering a more sustainable alternative to large models while highlighting the challenges of symbolic grounding.
A novel approach to boost small language models' reasoning using knowledge graph grounding.
Summary
The paper investigates a neuro-symbolic framework to enhance the reasoning capabilities of small language models by integrating knowledge graph grounding, specifically using two tool calls for fact extraction and expert reasoning, and evaluates the approach on the CLUTRR kinship benchmark.
Key contributions
- Development of a neuro-symbolic agentic framework for small language models.
- Evaluation of the framework on the CLUTRR kinship benchmark, demonstrating performance gains.
- Identification of extraction bottlenecks and the distraction effect in reasoning tasks.
Notable insights
- The use of a Relational Graph Convolutional Network (RGCN) for expert reasoning is a clever integration that could enhance performance in low-resource settings.
- The identification of a 'distraction effect' due to noisy self-generated facts provides a critical insight into the limitations of current approaches.
Possible limitations
- The abstract does not address potential scalability issues of the proposed framework or the computational resources required for the RGCN.
- Not stated in the abstract.
Abstract
arXiv:2607.14149v1 Announce Type: new Abstract: Although large language models (LLMs) have set benchmarks for zero-shot reasoning, their deployment remains cost-prohibitive and environmentally taxing. Small Language Models (SLMs) offer a sustainable alternative, but prone to errors, on tasks requiring complex, multi-hop logical grounding. We investigate a neuro-symbolic agentic framework to enhance the reasoning capabilities of SLMs, specifically Gemma 3 (1B, 4B) and Llama 3.2 (3B), using the CLUTRR kinship benchmark. Our approach transforms the SLM into a minimalist agent utilizing two specialized tool calls: extract_facts for symbolic triplet extraction and get_hint for expert reasoning via a Relational Graph Convolutional Network (RGCN). We evaluate these models across two configurations, both in an Oracle scenario with ground-truth triplets and a Realistic scenario relying on self-extracted knowledge. Our results reveal that while RGCN-derived hints provide a 1.5 - 2x performance gain over story-only baselines, the system is constrained by the extraction bottleneck and sequential deductive fragility, where early extraction errors compound over multi-hop chains. Furthermore, we identify a "distraction effect" in specific architectures where noisy, self-generated facts degrade performance despite the presence of expert hints. This work characterizes the challenges of symbolic grounding in low-resource agentic systems and provides a roadmap for iterative verification in neuro-symbolic agentic pipelines.