Risk Architecture for AI-Native Engineering Teams: An Organizational Framework for Agentic System Governance
Laxmipriya Ganesh Iyer
Why It Matters
What makes this one worth your time
AI engineers and researchers need effective risk management frameworks to handle the unique challenges of agentic AI systems, which differ significantly from traditional software systems.
A framework for managing risks in AI-native engineering teams.
Summary
The paper proposes a risk architecture framework for AI-native engineering teams, addressing the unique challenges posed by agentic AI systems. It introduces a seven-dimension profile to distinguish between different team types, a six-cluster failure-mode taxonomy, and a methodology for evaluating framework adequacy in detecting and managing risks.
Key contributions
- A seven-dimension profile distinguishing team types.
- A six-cluster failure-mode taxonomy including a new cluster.
- A methodology for scoring framework adequacy in risk management.
Notable insights
- The degradation of risk coverage is more pronounced at the organizational boundary where AI outputs interact with deterministic systems.
- The paper identifies a new failure-mode cluster: dependency-boundary determinism mismatch.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.01421v1 Announce Type: cross Abstract: Engineering management research has produced mature frameworks for software risk: ownership by feature, escalation by severity, and assurance by test coverage. These frameworks implicitly assume deterministic behavior, discrete and auditable change events, and clear component-to-owner mappings. Teams that build and operate agentic AI systems violate all three assumptions at once: outputs are probabilistic, systems take autonomous multi-step actions, and the risk surface mutates silently between deployments. Existing AI risk literature addresses this from above (policy frameworks such as the NIST AI RMF and ISO/IEC 42001) or below (threat taxonomies such as OWASP's agentic AI guidance), but not at the layer where an engineering manager (EM) operates: roles, decision rights, and escalation structures. This paper contributes (i) a seven-dimension profile distinguishing pure software-engineering, hybrid, and AI-native teams; (ii) a six-cluster failure-mode taxonomy including a previously unarticulated cluster, dependency-boundary determinism mismatch; and (iii) a synthetic framework-adequacy methodology scoring how well each profile's risk architecture detects, contains, and escalates a defined scenario set. Because the object of study is framework adequacy rather than human behavior, the evaluation yields derived rather than observed coverage claims. Coverage degrades as teams move from pure software engineering to AI-native operation, monotonically in the median and abruptly in the count of uncovered, high-consequence failures appearing only at the AI-native step. The degradation concentrates in specific failure-mode categories, and the most severe, least-covered failures arise not inside AI-native teams but at the organizational boundary where their probabilistic outputs are consumed by determinism-assuming dependencies.