Modeling Interaction Between Large Language Models and Humans in Co-Creative Decision-Making as Distributed Bayesian Inference

Abstract

The advent of powerful Large Language Models (LLMs) enables an era where humans and AI agents collaboratively engage in dynamic discussions to make decisions together. While prior research has largely focused on LLMs as assistant tools for human decision-making, their potential to co-create decisions through iterative knowledge integration remains less explored. This study introduces a novel model of human-LLM co-creative decision-making, formulated within the distributed Bayesian inference framework. We specifically interpret the iterative cycle, where the LLM proposes options based on its knowledge and the human selects based on preferences, as an instance of the Sampling-Importance-Resampling (SIR) algorithm. To validate our model, two experiments were conducted using LLM agents as substitutes for human participants under controlled conditions: a cooperative card-guessing task and a travel brainstorming task. Findings from the card-guessing task revealed that iterative information integration from both agents leads to a progressively updated decision distribution, consistent with SIR dynamics. Furthermore, the brainstorming task demonstrated that our proposed co-creative model facilitates robust dynamic knowledge integration and adaptive convergence of decisions. By framing co-creative decision-making as the emergence of a shared posterior distribution from integrating diverse knowledge, this research establishes a theoretical foundation for advancing human-AI symbiotic societies.