Collaborate, Deliberate, Evaluate: How LLM Alignment Affects Coordinated Multi-Agent Outcomes

Using the theoretical lens of the modified-action MDP, we show that common alignment techniques that are typically developed under single-user settings do not account for the dynamics of long-horizon multi-party interactions. We use a roleplay simulation methodology to quantify how AI partner interventions affect the trajectory of collaborative task dialogues. We show that interventions that are robust to action modification significantly outperform standard alignment in collaborative task support.