Attention-based Multi-scale Gated Recurrent Encoder with Novel Correlation Loss for COVID-19 Progression Prediction

COVID-19 image analysis has mostly focused on diagnostic tasks using single\ntimepoint scans acquired upon disease presentation or admission. We present a\ndeep learning-based approach to predict lung infiltrate progression from serial\nchest radiographs (CXRs) of COVID-19 patients. Our method first utilizes\nconvolutional neural networks (CNNs) for feature extraction from patches within\nthe concerned lung zone, and also from neighboring and remote boundary regions.\nThe framework further incorporates a multi-scale Gated Recurrent Unit (GRU)\nwith a correlation module for effective predictions. The GRU accepts CNN\nfeature vectors from three different areas as input and generates a fused\nrepresentation. The correlation module attempts to minimize the correlation\nloss between hidden representations of concerned and neighboring area feature\nvectors, while maximizing the loss between the same from concerned and remote\nregions. Further, we employ an attention module over the output hidden states\nof each encoder timepoint to generate a context vector. This vector is used as\nan input to a decoder module to predict patch severity grades at a future\ntimepoint. Finally, we ensemble the patch classification scores to calculate\npatient-wise grades. Specifically, our framework predicts zone-wise disease\nseverity for a patient on a given day by learning representations from the\nprevious temporal CXRs. Our novel multi-institutional dataset comprises\nsequential CXR scans from N=93 patients. Our approach outperforms transfer\nlearning and radiomic feature-based baseline approaches on this dataset.\n