Systematic Clinical Evaluation of A Deep Learning Method for Medical Image Segmentation: Radiosurgery Application

We systematically evaluate a Deep Learning model in a 3D medical image segmentation task. With our model, we address the flaws of manual segmentation: high inter-rater contouring variability and time consumption of the contouring process. The main extension over the existing evaluations is the careful and detailed analysis that could be further generalized on other medical image segmentation tasks. Firstly, we analyze the changes in the inter-rater detection agreement. We show that the model reduces the number of detection disagreements by <inline-formula><tex-math notation="LaTeX">$\text{48}\%$</tex-math></inline-formula> <inline-formula><tex-math notation="LaTeX">$\text {(p < 0.05)}$</tex-math></inline-formula>. Secondly, we show that the model improves the inter-rater contouring agreement from <inline-formula><tex-math notation="LaTeX">$\text {0.845}$</tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX">$\text {0.871}$</tex-math></inline-formula> surface Dice Score <inline-formula><tex-math notation="LaTeX">$\text {(p < 0.05)}$</tex-math></inline-formula>. Thirdly, we show that the model accelerates the delineation process between <inline-formula><tex-math notation="LaTeX">$\text {1.6}$</tex-math></inline-formula> and <inline-formula><tex-math notation="LaTeX">$\text {2.0}$</tex-math></inline-formula> times <inline-formula><tex-math notation="LaTeX">$\text {(p < 0.05)}$</tex-math></inline-formula>. Finally, we design the setup of the clinical experiment to either exclude or estimate the evaluation biases; thus, preserving the significance of the results. Besides the clinical evaluation, we also share intuitions and practical ideas for building an efficient DL-based model for 3D medical image segmentation.