Governance by Design: Architecting Agentic AI for Organizational Learning and Scalable Autonomy
Nelly Dux, Cristina Alaimo, Philippe Roussiere, Abhishek Kumar Mishra
Why It Matters
What makes this one worth your time
As organizations increasingly deploy agentic AI systems, understanding governance mechanisms is crucial for ensuring accountability and safety while achieving scalable autonomy.
The paper provides actionable insights for governing agentic AI in enterprise environments.
Summary
The paper explores the governance of agentic AI systems in organizational settings, detailing a case study of a large IT services company's development and rollout of such a system, and distilling seven lessons for effective governance during operationalization and scaling.
Key contributions
- A qualitative case study on the implementation of agentic AI in a large organization.
- Identification of seven lessons for effective governance in the context of agentic AI.
- Framework for balancing autonomy with accountability in AI deployments.
Notable insights
- Governance is shaped by architectural and operational arrangements that dictate system capabilities and data access.
- The integration of agentic AI with enterprise tools requires careful consideration of memory handling and performance improvement processes.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2605.20210v1 Announce Type: cross Abstract: Agentic AI systems - systems that can pursue goals through multi-step planning and tool-mediated action with limited direct supervision - are moving from experimental prototypes to enterprise deployments. This transition introduces tensions in implementation, scaling, and governance: organizations seek scalable autonomy for knowledge and coordination work, yet must preserve accountability, safety, cost control, and responsibility as systems initiate actions, access enterprise data, and evolve through iterative updates. Building on an in-depth qualitative case of a large IT services company's 2025 development and staged rollout of an agentic system integrated with enterprise tools; we show that governance is implemented through concrete architectural and working arrangements that determine what the system is allowed to do, which tools and data it can use, how memory is handled, and how performance improvements are introduced over time. We then distill seven lessons that explain how to build effective governance into agentic AI during operationalization and scaling.