A Unified Generative Adversarial Network Training via Self-Labeling and Self-Attention

We propose a novel GAN training scheme that can handle any level of labeling\nin a unified manner. Our scheme introduces a form of artificial labeling that\ncan incorporate manually defined labels, when available, and induce an\nalignment between them. To define the artificial labels, we exploit the\nassumption that neural network generators can be trained more easily to map\nnearby latent vectors to data with semantic similarities, than across separate\ncategories. We use generated data samples and their corresponding artificial\nconditioning labels to train a classifier. The classifier is then used to\nself-label real data. To boost the accuracy of the self-labeling, we also use\nthe exponential moving average of the classifier. However, because the\nclassifier might still make mistakes, especially at the beginning of the\ntraining, we also refine the labels through self-attention, by using the\nlabeling of real data samples only when the classifier outputs a high\nclassification probability score. We evaluate our approach on CIFAR-10, STL-10\nand SVHN, and show that both self-labeling and self-attention consistently\nimprove the quality of generated data. More surprisingly, we find that the\nproposed scheme can even outperform class-conditional GANs.\n