Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective

Training generative adversarial networks (GANs) with limited real image data\ngenerally results in deteriorated performance and collapsed models. To conquer\nthis challenge, we are inspired by the latest observation, that one can\ndiscover independently trainable and highly sparse subnetworks (a.k.a., lottery\ntickets) from GANs. Treating this as an inductive prior, we suggest a brand-new\nangle towards data-efficient GAN training: by first identifying the lottery\nticket from the original GAN using the small training set of real images; and\nthen focusing on training that sparse subnetwork by re-using the same set. We\nfind our coordinated framework to offer orthogonal gains to existing real image\ndata augmentation methods, and we additionally present a new feature-level\naugmentation that can be applied together with them. Comprehensive experiments\nendorse the effectiveness of our proposed framework, across various GAN\narchitectures (SNGAN, BigGAN, and StyleGAN-V2) and diverse datasets (CIFAR-10,\nCIFAR-100, Tiny-ImageNet, ImageNet, and multiple few-shot generation datasets).\nCodes are available at:\nhttps://github.com/VITA-Group/Ultra-Data-Efficient-GAN-Training.\n