ChatFive: Enhancing User Experience in Likert Scale Personality Test through Interactive Conversation with LLM Agents

Abstract

Personality assessments provide insights into understanding individual differences. In HCI, personality assessments are used to model user behavior or tailor user interfaces. However, conventional Likert-scale personality tests face issues in user engagement and capturing comprehensive personality nuances. Building upon prior work using conversational user interfaces for personality prediction, we delve deeper into personalized personality tests. Through a formative study (n=4), we identified three design goals for user engagement. Informed by these goals, we propose a novel architecture integrating multiple large language model agents to support free-form conversation-based personality assessment. Our system, ChatFive, predicts users’ Big Five traits through real-time personalized dialogue. Evaluations from our user study (n=20) revealed that ChatFive significantly improved conveying true responses and felt more engaged, though requiring longer response times and different validation. We discuss the limitations on the validity of ChatFive and its implications.