Rank-Constrained Neural Networks

This specification describes, among other things, a computer-implemented method. The method can include training a baseline neural network using a first set of training data. For each node in a subset of interconnected nodes in the baseline neural network, a rank-k approximation of a filter for the node can be computed. A subset of nodes in a rank-constrained neural network can then be initialized with the rank-k approximations of the filters from the baseline neural network. The subset of nodes in the rank-constrained neural network can correspond to the subset of nodes in the baseline neural network. After initializing, the rank-constrained neural network can be trained using a second set of training data while maintaining a rank-k filter topology for the subset of nodes in the rank-constrained neural network.