Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance

Many researchers motivate explainable AI with studies showing that human-AI\nteam performance on decision-making tasks improves when the AI explains its\nrecommendations. However, prior studies observed improvements from explanations\nonly when the AI, alone, outperformed both the human and the best team. Can\nexplanations help lead to complementary performance, where team accuracy is\nhigher than either the human or the AI working solo? We conduct mixed-method\nuser studies on three datasets, where an AI with accuracy comparable to humans\nhelps participants solve a task (explaining itself in some conditions). While\nwe observed complementary improvements from AI augmentation, they were not\nincreased by explanations. Rather, explanations increased the chance that\nhumans will accept the AI's recommendation, regardless of its correctness. Our\nresult poses new challenges for human-centered AI: Can we develop explanatory\napproaches that encourage appropriate trust in AI, and therefore help generate\n(or improve) complementary performance?\n