Continual Backprop: Stochastic Gradient Descent with Persistent Randomness

The Backprop algorithm for learning in neural networks utilizes two\nmechanisms: first, stochastic gradient descent and second, initialization with\nsmall random weights, where the latter is essential to the effectiveness of the\nformer. We show that in continual learning setups, Backprop performs well\ninitially, but over time its performance degrades. Stochastic gradient descent\nalone is insufficient to learn continually; the initial randomness enables only\ninitial learning but not continual learning. To the best of our knowledge, ours\nis the first result showing this degradation in Backprop's ability to learn. To\naddress this degradation in Backprop's plasticity, we propose an algorithm that\ncontinually injects random features alongside gradient descent using a new\ngenerate-and-test process. We call this the \\textit{Continual Backprop}\nalgorithm. We show that, unlike Backprop, Continual Backprop is able to\ncontinually adapt in both supervised and reinforcement learning (RL) problems.\nContinual Backprop has the same computational complexity as Backprop and can be\nseen as a natural extension of Backprop for continual learning.\n