For some time, it has been thought that backpropagation of errors could not be implemented in biophysiologically realistic neural circuits. This belief was largely due to either 1) the need for symmetric replication of feedback and feedforward weights, 2) the need for differing forms of activation between forward and backward propagating sweeps, and 3) the need for a separate network for error gradient computation and storage, on the one hand, or 4) nonphysiological backpropagation through the forward propagating neurons themselves, on the other. In this paper, we present spiking neuron mechanisms for gating pulses to maintain short-term memories, controlling forward inference and backward error propagation, and coordinating learning of feedback and feedforward weights. These neural mechanisms are synthesized into a new backpropagation algorithm for neuromorphic circuits.
Paper
Full text
A Pulse-gated, Neural Implementation of the Backpropagation Algorithm
Semantic Scholar · Computer Science · 2019
Abstract
For some time, it has been thought that backpropagation of errors could not be implemented in biophysiologically realistic neural circuits. This belief was largely due to either 1) the need for symmetric replication of feedback and feedforward weights, 2) the need for differing forms of activation between forward and backward propagating sweeps, and 3) the need for a separate network for error gradient computation and storage, on the one hand, or 4) nonphysiological backpropagation through the forward propagating neurons themselves, on the other. In this paper, we present spiking neuron mechanisms for gating pulses to maintain short-term memories, controlling forward inference and backward error propagation, and coordinating learning of feedback and feedforward weights. These neural mechanisms are synthesized into a new backpropagation algorithm for neuromorphic circuits.