We propose the first practical scheme for end-to-end optical backpropagation in neural networks. Using saturable absorption for the nonlinear units, we find that the backward propagating gradients required to train the network can be approximated in a surprisingly simple pump-probe scheme that requires only passive optical elements. Simulations show that, with readily obtainable optical depths, our approach can achieve equivalent performance to state-of-the-art computational networks on image classification benchmarks, even in deep networks with multiple sequential gradient approximations. With backpropagation through nonlinear units an outstanding challenge to the field, this work provides a feasible path towards truly all-optical neural networks.