Target Layer Regularization for Continual Learning Using Cramer-Wold Generator

We propose an effective regularization strategy (CW-TaLaR) for solving\ncontinual learning problems. It uses a penalizing term expressed by the\nCramer-Wold distance between two probability distributions defined on a target\nlayer of an underlying neural network that is shared by all tasks, and the\nsimple architecture of the Cramer-Wold generator for modeling output data\nrepresentation. Our strategy preserves target layer distribution while learning\na new task but does not require remembering previous tasks' datasets. We\nperform experiments involving several common supervised frameworks, which prove\nthe competitiveness of the CW-TaLaR method in comparison to a few existing\nstate-of-the-art continual learning models.\n