Benchmarking of Deep Learning Methods for Generic MRI Multi-Organ Abdominal Segmentation

Abstract. Purpose Recent advances in deep learning have led to robust automated tools for segmentation of abdominal computed tomography (CT). Meanwhile, segmentation of magnetic resonance imaging (MRI) is substantially more challenging due to the inherent signal variability and the increased effort required for annotating training datasets. Hence, existing approaches are trained on limited sets of MRI sequences, which might limit their generalizability. Approach To characterize the landscape of MRI abdominal segmentation tools, we present a comprehensive benchmarking of three state-of-the-art and open-source models: MRSegmentator, MRISegmentator-Abdomen, and TotalSegmentator MRI. As these models are trained using labor-intensive manual annotation cycles, we also introduce and evaluate ABDSynth, a SynthSeg-based model purely trained on widely available CT segmentations (no real images). We assess accuracy and generalizability by leveraging three public datasets (not seen by any of the evaluated methods during their training), which span all major manufacturers, five MRI sequences, as well as a variety of subject conditions, voxel resolutions, and fields-of-view. Results Our results reveal that MRSegmentator achieves the best performance and is most generalizable. By contrast, ABDSynth yields slightly less accurate results, but its relaxed requirements in training data make it an alternative when the annotation budget is limited. Conclusions We perform benchmarking of four open-source models for abdominal MR segmentation on three datasets and demonstrate that models trained on real, heterogeneous, multimodal data yield the best overall performance. We provide evaluation code and datasets for future benchmarking at https://github.com/deepakri201/AbdoBench.