ELLMA-T: an Embodied LLM-agent for Supporting English Language Learning in Social VR

Many people struggle with learning a new language when moving to a new country, with traditional tools falling short in providing contextualized learning tailored to each learner’s needs. The recent development of large language models (LLMs) and embodied conversational agents (ECAs) in social virtual reality (VR) provides new opportunities to practice language learning in a contextualized and naturalistic way that takes into account the learner’s language level and needs. To explore this opportunity, we developed ELLMA-T, a design probe that integrates an LLM (GPT-4) with an ECA for English language learning in social VR (VRChat), informed by the situated learning framework. We conducted a feasibility study to explore the potential and challenges of LLM-based ECAs for language learning in social VR. Drawing on qualitative interviews (N=12), we reveal the potential of ELLMA-T to generate realistic, believable, and context-specific role plays for agent-learner interaction in VR, and LLM’s capability to provide initial language assessment and continuous feedback to learners. We provide four design implications for the future development of LLM-based language agents in social VR.