XCAT-GAN for Synthesizing 3D Consistent Labeled Cardiac MR Images on Anatomically Variable XCAT Phantoms

Generative adversarial networks (GANs) have provided promising data\nenrichment solutions by synthesizing high-fidelity images. However, generating\nlarge sets of labeled images with new anatomical variations remains unexplored.\nWe propose a novel method for synthesizing cardiac magnetic resonance (CMR)\nimages on a population of virtual subjects with a large anatomical variation,\nintroduced using the 4D eXtended Cardiac and Torso (XCAT) computerized human\nphantom. We investigate two conditional image synthesis approaches grounded on\na semantically-consistent mask-guided image generation technique: 4-class and\n8-class XCAT-GANs. The 4-class technique relies on only the annotations of the\nheart; while the 8-class technique employs a predicted multi-tissue label map\nof the heart-surrounding organs and provides better guidance for our\nconditional image synthesis. For both techniques, we train our conditional\nXCAT-GAN with real images paired with corresponding labels and subsequently at\nthe inference time, we substitute the labels with the XCAT derived ones.\nTherefore, the trained network accurately transfers the tissue-specific\ntextures to the new label maps. By creating 33 virtual subjects of synthetic\nCMR images at the end-diastolic and end-systolic phases, we evaluate the\nusefulness of such data in the downstream cardiac cavity segmentation task\nunder different augmentation strategies. Results demonstrate that even with\nonly 20% of real images (40 volumes) seen during training, segmentation\nperformance is retained with the addition of synthetic CMR images. Moreover,\nthe improvement in utilizing synthetic images for augmenting the real data is\nevident through the reduction of Hausdorff distance up to 28% and an increase\nin the Dice score up to 5%, indicating a higher similarity to the ground truth\nin all dimensions.\n