Interacting with a humanized dual-LLM agent: an exploratory study of individual and collaborative modes in language proficiency and critical thinking awareness

The proliferation of Generative AI (GenAI) has profoundly transformed language learning paradigms. Although GenAI demonstrates great potential, its limited social presence, occasional AI hallucinations, and risk of overreliance raise concerns. A humanized GenAI agent powered by dual Large Language Models (LLMs) offers a viable solution. Nevertheless, how individual and collaborative interactions with a humanized dual-LLM GenAI agent affect foreign language proficiency, critical thinking awareness, as well as the underlying mechanisms remains underexplored. The study employed a quasi-experimental design involving 89 university students assigned to three different language learning conditions. Data on participants’ language proficiency and critical thinking awareness were collected, with their conversations with the GenAI agent being documented. While no condition proved statistically superior, exploratory analyses suggested a trend whereby collaborative interactions with the GenAI agent were associated with higher language proficiency, while only the individual interaction group showed significant within-group improvement in critical thinking awareness. Epistemic network analysis revealed that collaborative interactions fostered balanced and diverse interaction patterns, whereas individual interactions centered on cognitive monitoring and strategic learning behaviors. This study offers preliminary insights into the pedagogical value of a humanized dual-LLM GenAI agent and explores how interaction types relate to differentiated learning outcomes.