Early detection of knee osteoarthritis using deep learning on knee magnetic resonance images

. An increasing number of deep learning techniques have been used to detect osteoarthritis (OA) and study its incidence and progression. Although valuable imaging-based biomarkers for OA are typically derived from magnetic resonance imaging (MRI), most of the deep learning techniques have been developed based on plain radiography (X-ray) images of patients and their demographics ( e.g. , age, gender, body mass index (BMI)). In this study, we aim to investigate the influence of MRI and patient data on the prediction of knee OA incidence using different deep learning architectures. Knee OA incidence within 24 months was predicted using the intermediate-weighted turbo spin-echo (IW-TSE) sequence of 593 patients from the Osteoarthritis Initiative. To extract a region of interest containing the knee joint from the IW-TSE sequence, a U-Net model was trained and used to segment bone on a dual echo steady state (DESS) sequence. Subsequently, IW-TSE and DESS sequences were registered and the DESS segmentations were transformed to the corresponding IW-TSE scans. The performance of MRI-based features in the prediction of knee OA incidence was tested using three different deep learning architectures: a residual network (ResNet), a densely connected convolutional network (DenseNet), and a convolutional variational autoencoder (CVAE). To evaluate the predictive performance of MRI-based features alone, the outputs of ResNet, DenseNet, and CVAE were coupled with patient data ( i.e. , age, gender, BMI) and used as input to a Logistic Regression (LR) Classifier. Knee OA was defined based on visual MRI and X-ray-based OA features. The performance of the segmentation was evaluated using the Dice similarity coefficient, while those of the OA detection algorithms were assessed using the area under the receiver operating characteristic curve (AUC) and the precision-recall curve (PR-AUC) metrics. Regarding the segmentation of tibial and femoral bones, the Dice similarity coefficients were 0.985 and 0.987, respectively. The ResNet and DenseNet showed similar results, with both methods having AUC values up to 0.6269. The best performing OA detection model was CVAE with an AUC of 0.6699 when combined with patient data and an AUC of 0.6689 when used alone as input to the LR classifier. All three detection algorithms yielded higher performance metrics when patient data were combined with MRI-based features. The results showed that three deep learning algorithms have similar metrics when using IW-TSE MRIs and their performance increased with the inclusion of patient data, which shows the strong influence of variables such as age, gender, and BMI on the detection of knee OA.