Building Human-AI Reliance Through Cognitive Engagement and Exploratory AI Assistance

Abstract

AI assistance is increasingly used to improve human-AI collaborative decision-making. However, how domain experts integrate their knowledge with grounded constraints and formulate intent with AI systems remains underexplored. In this position paper, we argue for “cognitively aligned” AI assistance, where users engage interactively with symbolic (logic-based) and sub-symbolic AI to interpret, influence, and co-construct decisions. Through this lens, we believe that users can build effective reliance on AI assistance, iteratively anchoring their domain knowledge to adapt their mental models and AI assistance. We explore the current literature and emphasize the need for cognitive (analytical) engagement with AI assistance to improve semantic alignment and interactive affordances for domain experts. We outline a plan for a research study that explores users’ interaction with AI assistance and quantitative reasoning in business decision-making.