ORDisCo: Effective and Efficient Usage of Incremental Unlabeled Data for Semi-supervised Continual Learning

Continual learning usually assumes the incoming data are fully labeled, which\nmight not be applicable in real applications. In this work, we consider\nsemi-supervised continual learning (SSCL) that incrementally learns from\npartially labeled data. Observing that existing continual learning methods lack\nthe ability to continually exploit the unlabeled data, we propose deep Online\nReplay with Discriminator Consistency (ORDisCo) to interdependently learn a\nclassifier with a conditional generative adversarial network (GAN), which\ncontinually passes the learned data distribution to the classifier. In\nparticular, ORDisCo replays data sampled from the conditional generator to the\nclassifier in an online manner, exploiting unlabeled data in a time- and\nstorage-efficient way. Further, to explicitly overcome the catastrophic\nforgetting of unlabeled data, we selectively stabilize parameters of the\ndiscriminator that are important for discriminating the pairs of old unlabeled\ndata and their pseudo-labels predicted by the classifier. We extensively\nevaluate ORDisCo on various semi-supervised learning benchmark datasets for\nSSCL, and show that ORDisCo achieves significant performance improvement on\nSVHN, CIFAR10 and Tiny-ImageNet, compared to strong baselines.\n