Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer

Medical image registration and segmentation are two of the most frequent\ntasks in medical image analysis. As these tasks are complementary and\ncorrelated, it would be beneficial to apply them simultaneously in a joint\nmanner. In this paper, we formulate registration and segmentation as a joint\nproblem via a Multi-Task Learning (MTL) setting, allowing these tasks to\nleverage their strengths and mitigate their weaknesses through the sharing of\nbeneficial information. We propose to merge these tasks not only on the loss\nlevel, but on the architectural level as well. We studied this approach in the\ncontext of adaptive image-guided radiotherapy for prostate cancer, where\nplanning and follow-up CT images as well as their corresponding contours are\navailable for training. The study involves two datasets from different\nmanufacturers and institutes. The first dataset was divided into training (12\npatients) and validation (6 patients), and was used to optimize and validate\nthe methodology, while the second dataset (14 patients) was used as an\nindependent test set. We carried out an extensive quantitative comparison\nbetween the quality of the automatically generated contours from different\nnetwork architectures as well as loss weighting methods. Moreover, we evaluated\nthe quality of the generated deformation vector field (DVF). We show that MTL\nalgorithms outperform their Single-Task Learning (STL) counterparts and achieve\nbetter generalization on the independent test set. The best algorithm achieved\na mean surface distance of $1.06 \\pm 0.3$ mm, $1.27 \\pm 0.4$ mm, $0.91 \\pm 0.4$\nmm, and $1.76 \\pm 0.8$ mm on the validation set for the prostate, seminal\nvesicles, bladder, and rectum, respectively. The high accuracy of the proposed\nmethod combined with the fast inference speed, makes it a promising method for\nautomatic re-contouring of follow-up scans for adaptive radiotherapy.\n