Editorial for “Gradual Self‐Training via Confidence and Volume Based Domain Adaptation for Multi Dataset Deep Learning‐Based Brain Metastases Detection Using Nonlocal Networks on MRI Images”
Advances in artificial intelligence (AI) and machine learning (ML) have recently been demonstrated to be valuable in clinical practice by improving the quality of image reconstruction, correcting imaging artifacts, and increasing diagnostic accuracy. Although AI and ML have been introduced in the aforementioned clinical applications, applying AI/ML methods to new clinical environments has a few challenges. First, the performance of supervised learningbased techniques is highly dependent on the training data. Therefore, for each new application, it is usually difficult to apply directly the previously trained model to a new clinical environment. While transfer learning has been explored in image reconstruction, it is often more challenging for detection and segmentation tasks. Second, the amount of data needed in the development process is usually large. Meanwhile, the data collection and labeling process is usually time-consuming and requires domain knowledge. Finally, in MRI applications, there are multiple image contrasts available. A robust AI/ML technique needs to be generalizable to all available types of images. The present study aims at detecting brain metastases using a semi-supervised and transfer learning approach based on a 3D convolutional neural network (CNN) object detection method containing nonlocal blocks. The proposed method achieved state-of-the-art performance on the publicly available BrainMetShare dataset. Without clinically acceptable CAD tools for detection of brain metastases, practitioners need to manually delineate the lesion in all MRI slices to determine overall disease extent and burden as well as to plan treatments. The limited volume of images one practitioner can carefully analyze can limit the feasibility of quantitative determination of disease burden and the accuracy of interpretation; further, the manual process may introduce variances and errors to the delineation. At the same time, for detection of brain metastases, few techniques have been explored using recent deep learning approaches and concepts. This study proposed an effective self-training framework for training deep learning models with limited amounts of data by using multiple datasets, including a local dataset, the BrainMetShare set, and the BRATS set. To fully utilize these datasets, the proposed approach uses domain adaptation to gradually learn new features from each set and achieve a final model with high generalizability. Interestingly, this approach combines supervised learning with unsupervised learning and transfer learning in an iterative framework. The model was first trained using labeled data from the local dataset and then gradually learned from the unlabeled high confidence data and labeled data from another domain (primary gliomas). In this framework, selection of lesions with similar volumes enables an effective domain adaptation and leads to a more generalized model. This study also proposed to use nonlocal networks to achieve high robustness on multimodality data. The use of nonlocal blocks was originally proposed to capture the longrange dependencies in high-dimensional network features and was demonstrated to be effective in jointly reasoning spatial and temporal information in video classification tasks. This study uses the nonlocal blocks to capture long-range features in 3D volumes and provide more global context to the network; overall, the use of nonlocal networks for detection of brain metastases was demonstrated to be effective. A final valuable contribution from this study is the use of metrics reported on public datasets. These metrics enable direct comparisons in future studies on detection accuracy. At the same time, the authors presented a comprehensive set of metrics for detailed analysis of the proposed model architecture and training framework.
Paper
Full text
Editorial for “Gradual Self‐Training via Confidence and Volume Based Domain Adaptation for Multi Dataset Deep Learning‐Based Brain Metastases Detection Using Nonlocal Networks on MRI Images”
Semantic Scholar · Medicine · 2022
Abstract
Advances in artificial intelligence (AI) and machine learning (ML) have recently been demonstrated to be valuable in clinical practice by improving the quality of image reconstruction, correcting imaging artifacts, and increasing diagnostic accuracy. Although AI and ML have been introduced in the aforementioned clinical applications, applying AI/ML methods to new clinical environments has a few challenges. First, the performance of supervised learningbased techniques is highly dependent on the training data. Therefore, for each new application, it is usually difficult to apply directly the previously trained model to a new clinical environment. While transfer learning has been explored in image reconstruction, it is often more challenging for detection and segmentation tasks. Second, the amount of data needed in the development process is usually large. Meanwhile, the data collection and labeling process is usually time-consuming and requires domain knowledge. Finally, in MRI applications, there are multiple image contrasts available. A robust AI/ML technique needs to be generalizable to all available types of images. The present study aims at detecting brain metastases using a semi-supervised and transfer learning approach based on a 3D convolutional neural network (CNN) object detection method containing nonlocal blocks. The proposed method achieved state-of-the-art performance on the publicly available BrainMetShare dataset. Without clinically acceptable CAD tools for detection of brain metastases, practitioners need to manually delineate the lesion in all MRI slices to determine overall disease extent and burden as well as to plan treatments. The limited volume of images one practitioner can carefully analyze can limit the feasibility of quantitative determination of disease burden and the accuracy of interpretation; further, the manual process may introduce variances and errors to the delineation. At the same time, for detection of brain metastases, few techniques have been explored using recent deep learning approaches and concepts. This study proposed an effective self-training framework for training deep learning models with limited amounts of data by using multiple datasets, including a local dataset, the BrainMetShare set, and the BRATS set. To fully utilize these datasets, the proposed approach uses domain adaptation to gradually learn new features from each set and achieve a final model with high generalizability. Interestingly, this approach combines supervised learning with unsupervised learning and transfer learning in an iterative framework. The model was first trained using labeled data from the local dataset and then gradually learned from the unlabeled high confidence data and labeled data from another domain (primary gliomas). In this framework, selection of lesions with similar volumes enables an effective domain adaptation and leads to a more generalized model. This study also proposed to use nonlocal networks to achieve high robustness on multimodality data. The use of nonlocal blocks was originally proposed to capture the longrange dependencies in high-dimensional network features and was demonstrated to be effective in jointly reasoning spatial and temporal information in video classification tasks. This study uses the nonlocal blocks to capture long-range features in 3D volumes and provide more global context to the network; overall, the use of nonlocal networks for detection of brain metastases was demonstrated to be effective. A final valuable contribution from this study is the use of metrics reported on public datasets. These metrics enable direct comparisons in future studies on detection accuracy. At the same time, the authors presented a comprehensive set of metrics for detailed analysis of the proposed model architecture and training framework.