This paper proposes a computationally efficient framework for simulating Socratic tutoring behavior using compact Large Language Models (≤3B parameters). Addressing the scarcity of specialized pedagogical datasets, we synthesized approximately one thousand structured student-teacher dialogues to drive a hybrid training pipeline that combines Supervised Fine-Tuning (SFT) with Low-Rank Adaptation (LoRA) and Direct Preference Optimization (DPO). We evaluated the pedagogical alignment of LLaMA 3 and Qwen model families using a rigorous protocol that integrates automated judging (via GPT-4o-mini) with human validation. Our results demonstrate that the LLaMA 3.2-3B model, when optimized via the sequential SFT+DPO pipeline, achieves 95.33% high-quality responses (Socratic score ≥ 4/5). Furthermore, the model exhibited robust generalization on unseen programming problems, maintaining an 87.6% high-quality rate. These findings confirm that compact models can effectively internalize complex pedagogical strategies through preference optimization, offering a scalable and privacy-preserving alternative to larger proprietary models in educational AI.
Paper
Full text
A Socratic Multi-Agent Framework using LLMs for Programming Education
Semantic Scholar · 2026
Abstract
This paper proposes a computationally efficient framework for simulating Socratic tutoring behavior using compact Large Language Models (≤3B parameters). Addressing the scarcity of specialized pedagogical datasets, we synthesized approximately one thousand structured student-teacher dialogues to drive a hybrid training pipeline that combines Supervised Fine-Tuning (SFT) with Low-Rank Adaptation (LoRA) and Direct Preference Optimization (DPO). We evaluated the pedagogical alignment of LLaMA 3 and Qwen model families using a rigorous protocol that integrates automated judging (via GPT-4o-mini) with human validation. Our results demonstrate that the LLaMA 3.2-3B model, when optimized via the sequential SFT+DPO pipeline, achieves 95.33% high-quality responses (Socratic score ≥ 4/5). Furthermore, the model exhibited robust generalization on unseen programming problems, maintaining an 87.6% high-quality rate. These findings confirm that compact models can effectively internalize complex pedagogical strategies through preference optimization, offering a scalable and privacy-preserving alternative to larger proprietary models in educational AI.