Behavioral Controllability of Agentic Models for Information Extraction: From Fixed Workflows to Reflective Agents
Lujia Zhang, Xingzhou Chen, Hongwei Feng
Why It Matters
What makes this one worth your time
Understanding the impact of agentic components on information extraction can lead to more efficient and adaptable AI systems, which is crucial for complex data processing tasks.
The paper explores how agentic components like reflection and memory affect information extraction tasks.
Summary
The paper investigates the behavioral controllability of agentic models in information extraction tasks, comparing fixed workflows with reflective agents using a conference-paper dataset extraction task. It evaluates process-level behaviors and characterizes when agentic mechanisms alter system behavior and task completion.
Key contributions
- Comparison of fixed workflows with reflective agent variants in information extraction.
- Characterization of when agentic mechanisms alter system behavior and task completion.
Notable insights
- Agentic mechanisms such as reflection and memory can change system behavior in information extraction tasks.
- The study emphasizes process-level behaviors over traditional outcome measures, providing a different perspective on evaluating AI systems.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.15715v1 Announce Type: new Abstract: Large language model (LLM) agents are increasingly used for complex information-extraction tasks, yet it remains unclear whether agentic components such as reflection and memory lead to observable and controllable improvements over fixed LLM workflows. We study this question through conference-paper dataset extraction, where a system must identify datasets mentioned in scholarly PDFs and produce structured records. We compare a fixed workflow baseline with reflective agent variants and specify an optimized agent condition (S2) that extends the same task with richer PDF tools and dynamic tool selection. Our evaluation emphasizes process-level behavior--including tool execution, retries, reflection, memory use, runtime, and failure recovery--while treating extraction coverage and field completeness as secondary outcome measures. The paper characterizes when agentic mechanisms change system behavior, whether these changes improve task completion, and how the observed failure modes motivate an optimized agent design under the same evaluation harness.