Align AI to Dynamic Human-AI Workflows
Valerie Chen, Cleotilde Gonzalez, Anita Williams Woolley, Michael Lee, Tongshuang Wu, Vincent Conitzer, Aarti Singh
Why It Matters
What makes this one worth your time
Understanding and implementing dynamic alignment can enhance the effectiveness of human-AI collaboration, making AI systems more responsive to real-world complexities.
This paper advocates for a shift to dynamic, interactive AI alignment based on evolving human preferences.
Summary
The paper critiques existing AI alignment approaches for their static nature and proposes a dynamic, interactive framework for aligning AI with human preferences that evolve through interaction.
Key contributions
- Formalization of the gap between static and dynamic alignment approaches.
- Outline of a research agenda integrating machine learning with social and decision sciences.
Notable insights
- The need for a trajectory-level view of human-AI interactions highlights the limitations of current static alignment methods.
- Interdisciplinary insights from social sciences can inform better alignment strategies in AI.
Possible limitations
- Not stated in the abstract.
Abstract
arXiv:2607.14240v1 Announce Type: new Abstract: Current alignment approaches typically focus on emulating human behavior using static representations of human preferences, failing to capture the dynamic, context-dependent nature of real-world human-AI interactions. In this paper, we argue for a shift from static and emulative to interactive and complementary alignment, where preferences emerge through interaction and alignment is defined not by satisfying preferences alone. We first formalize this gap by contrasting existing alignment with a trajectory-level view in which human and model behavior co-evolve over time. Because these interaction dynamics have not been adequately captured within existing ML formulations, we ground this perspective in insights from an interdisciplinary workshop. We draw on lessons from social-science accounts of human-human collaboration and then argue that human-AI systems amplify these dynamics, introducing new asymmetries that make reasoning about uncertainty harder and introduce new coordination challenges. Based on these lessons and new challenges, we conclude by outlining a research agenda for developing AI systems that align with humans in interaction, requiring an interdisciplinary synthesis of machine learning and the social and decision sciences.