ETalk: An Intelligent Assessment System for Degree of Course Objectives Achievement in Medical Universities

The development of artificial intelligence provides the possibility to solve the problems of single dimensions and lagging feedback in medical education quality assessment. This work proposed ETalk, an intelligent assessment system based on generative dialog model. The system constructs a curriculum knowledge map by multimodal fusion technology, extracts three-dimensional feature vectors of teaching objectives, knowledge density, and teaching methods with hierarchical Transformer model. The Dynamic Question Generation Engine (DQGE) is constructed to combine Bloom's Cognitive Taxonomy and Item Response Theory to achieve the adaptive generation of personalized clinical context questions. The Bilateral Assessment Method (BAM) is constructed to quantify the achievement of teaching objectives based on the feedback of multi-round dialog interactions with students. Our results show the ETalk improved the matching of teaching objectives to 64.06%, knowledge mastery recognition is 80.28%, and medical thinking assessment to 74.16% in five medical courses. This work provides an interpretable and operationalized closed-loop assessment paradigm for quality monitoring in medical education within the context of artificial intelligence.