Quantitative Modeling of Dynamic Human-Agent Cognition

We explore cognitive models of the interaction between human and computational agents in networked information recommendation and analysis. Elements in an information workflow—including but not limited to queries, databases, algorithms, software solutions, virtual agents, and smart sensors—must vie for a user’s acceptance and attention. Intelligent technological artifacts, such as recommender systems, are often effective but opaque and unpredictable; in contrast, tool-like information systems, such as winnowing and query interfaces, are transparent but limited in their usefulness. System builders have responded by increasing transparency (via explanation) and customizability (via control parameters) of complex algorithms or by improving the effectiveness of simple information algorithms (such as adding personalization to keyword search). Unfortunately, requiring user input or attention requires cognitive bandwidth, which could hurt performance in time-sensitive operations. At the same time, improving the performance of algorithms typically requires making the underlying computations more complex, reducing predictability, increasing potential mistrust, and sometimes resulting in user performance degradation. When sufficiently complex, systems can also be perceived as being agents rather than tools, which further complicates interaction modeling. In this work, we use a novel statistical approach to conduct a quantitative investigation into the effects of cognitive factors on human interaction with intelligent systems under two different task paradigms. The resulting statistical comparisons of the two tasks expand our ability to quantify human-agent cognition and highlight future research challenges when extending to embodied multi-agent systems.