Benchmarking CNN on 3D Anatomical Brain MRI: Architectures, Data Augmentation and Deep Ensemble Learning

Deep Learning (DL) and specifically CNN models have become a de facto method\nfor a wide range of vision tasks, outperforming traditional machine learning\n(ML) methods. Consequently, they drew a lot of attention in the neuroimaging\nfield in particular for phenotype prediction or computer-aided diagnosis.\nHowever, most of the current studies often deal with small single-site cohorts,\nalong with a specific pre-processing pipeline and custom CNN architectures,\nwhich make them difficult to compare to. We propose an extensive benchmark of\nrecent state-of-the-art (SOTA) 3D CNN, evaluating also the benefits of data\naugmentation and deep ensemble learning, on both Voxel-Based Morphometry (VBM)\npre-processing and quasi-raw images. Experiments were conducted on a large\nmulti-site 3D brain anatomical MRI data-set comprising N=10k scans on 3\nchallenging tasks: age prediction, sex classification, and schizophrenia\ndiagnosis. We found that all models provide significantly better predictions\nwith VBM images than quasi-raw data. This finding evolved as the training set\napproaches 10k samples where quasi-raw data almost reach the performance of\nVBM. Moreover, we showed that linear models perform comparably with SOTA CNN on\nVBM data. We also demonstrated that DenseNet and tiny-DenseNet, a lighter\nversion that we proposed, provide a good compromise in terms of performance in\nall data regime. Therefore, we suggest to employ them as the architectures by\ndefault. Critically, we also showed that current CNN are still very biased\ntowards the acquisition site, even when trained with N=10k multi-site images.\nIn this context, VBM pre-processing provides an efficient way to limit this\nsite effect. Surprisingly, we did not find any clear benefit from data\naugmentation techniques. Finally, we proved that deep ensemble learning is well\nsuited to re-calibrate big CNN models without sacrificing performance.\n