SterileAR: Exploration of Augmented Reality and Computer Vision Approaches for Real-Time Feedback in Sterile Compounding Training

Abstract

This paper presents preliminary work on the development of SterileAR, a software platform for providing real-time feedback to pharmacy students during training in sterile pharmaceutical compounding procedures. Simulation pedagogy requires that the learner receive immediate feedback to recognize procedural errors and internalize learning objectives. Instructors and task trainers are often utilized in tandem to allow deliberate practice for novices in specialized clinical procedures. In this work, we explore the possibility of augmenting sterile compounding procedure training with real-time feedback using augmented reality, machine learning, and computer vision technologies. We present our approaches to developing SterileAR in four iterations, including specific descriptions of successes and failures, and methodologies for capturing object spatial position, overlapping, and occlusion.