How Do You Choose Your AI Component? An Interview Study of Secure AI Integration in Practice
Mahzabin Tamanna, Elizabeth Lin, Sparsha Gowda, Laurie Williams, Dominik Wermke
Why It Matters
What makes this one worth your time
AI engineers and researchers should care because overlooking security in AI component integration can lead to vulnerabilities, repeating past mistakes in software dependency management.
The study reveals a lack of security considerations in AI component integration and offers recommendations for improvement.
Summary
The paper conducts an exploratory semi-structured interview study with 22 software developers, architects, and AI practitioners to understand their decision-making process and security considerations when selecting and integrating AI components into software systems. The study finds that security is often overlooked in favor of functional criteria like performance and cost, and it provides recommendations for a security-by-design approach.
Key contributions
- Conducted interviews with 22 practitioners to assess AI component integration practices.
- Identified a gap in security considerations during AI component selection.
- Provided actionable recommendations for integrating security into AI component selection.
Notable insights
- Practitioners prioritize functional criteria over security when selecting AI components.
- The industry is repeating historical mistakes by neglecting security in AI integration.
Possible limitations
- Not stated in the abstract
Abstract
arXiv:2607.16660v1 Announce Type: cross Abstract: The increasing adoption of Large Language Models (LLMs) as AI components in modern software systems introduces distinct security risks to the software supply chain. While many considerations and safety mechanisms are in place for components of the traditional software supply chain, the recent rapid adoption of AI components and platforms has overlooked these hard learned lessons. Selecting and integrating AI models without clear guidance on how these choices affect system security may leave applications vulnerable to threats, such as malicious components, data leakage, and unintended behavior. The goal of this study is to understand practitioners' decision making process and security considerations in selecting and integrating AI components through an exploratory semi-structured interview study. Toward this goal, we conducted semistructured interviews with 22 software developers, architects, and AI practitioners across diverse organizations about how they integrate AI components into their software. Our analysis finds that practitioners' model selection is predominantly driven by functional criteria, including performance, accuracy, cost, and specific features, e.g., tool calling or multimodal support, while security is rarely considered as an evaluation criterion. We observe a consistent lack of security concern throughout the AI component integration process, with established software supply chain lessons overlooked or ignored. The industry is repeating the historically costly mistakes of early software dependency management, prioritizing rapid reuse and availability over security and provenance. We distill our findings into actionable recommendations for AI adopters, model providers, and researchers, advocating for a proactive, security-by-design approach that integrates security evaluation into component selection and sustains it throughout the software development lifecycle.