Overcoming Barriers to Data Sharing with Medical Image Generation: A Comprehensive Evaluation

Privacy concerns around sharing personally identifiable information are a\nmajor practical barrier to data sharing in medical research. However, in many\ncases, researchers have no interest in a particular individual's information\nbut rather aim to derive insights at the level of cohorts. Here, we utilize\nGenerative Adversarial Networks (GANs) to create derived medical imaging\ndatasets consisting entirely of synthetic patient data. The synthetic images\nideally have, in aggregate, similar statistical properties to those of a source\ndataset but do not contain sensitive personal information. We assess the\nquality of synthetic data generated by two GAN models for chest radiographs\nwith 14 different radiology findings and brain computed tomography (CT) scans\nwith six types of intracranial hemorrhages. We measure the synthetic image\nquality by the performance difference of predictive models trained on either\nthe synthetic or the real dataset. We find that synthetic data performance\ndisproportionately benefits from a reduced number of unique label combinations.\nOur open-source benchmark also indicates that at low number of samples per\nclass, label overfitting effects start to dominate GAN training. We\nadditionally conducted a reader study in which trained radiologists do not\nperform better than random on discriminating between synthetic and real medical\nimages for intermediate levels of resolutions. In accordance with our benchmark\nresults, the classification accuracy of radiologists increases at higher\nspatial resolution levels. Our study offers valuable guidelines and outlines\npractical conditions under which insights derived from synthetic medical images\nare similar to those that would have been derived from real imaging data. Our\nresults indicate that synthetic data sharing may be an attractive and\nprivacy-preserving alternative to sharing real patient-level data in the right\nsettings.\n