CogMAS: A Cognitively-Grounded Multi-Agent Framework for Explainable and Consistent Open-Ended Student Response Scoring

Abstract

Automated scoring of open-ended questions continues to face significant challenges in modeling student cognition, ensuring scoring consistency, and providing interpretability. Although large language models (LLMs) have demonstrated substantial potential, single-model architectures exhibit structural limitations in handling complex reasoning and cognitive alignment. To address these issues, we propose CogMAS, a Cognitively-Grounded Multi-Agent Scoring Framework that incorporates three types of agents, i.e., student agents, teacher agents, and evaluation agents, to enable multidimensional and interpretable scoring of open-ended responses. CogMAS leverages Bloom’s taxonomy to construct a mapping between questions and cognitive dimensions, guiding teacher agents to perform dimension-aware scoring. A dual-stage semantic retrieval module is introduced to provide contextually relevant exemplars. Evaluation agents are responsible for detecting explanation path biases and deriving high-confidence reasoning chains and final scores. Teacher agents are further trained using Direct Preference Optimization (DPO) to improve the quality and consistency of scoring explanations. High-confidence score–explanation pairs are stored in a retrievable memory module to support continuous optimization in future tasks. Experiments on three public open-ended question scoring datasets demonstrate that CogMAS achieves state-of-the-art performance in both scoring accuracy and consistency, validating its effectiveness and generalizability.