Data-driven emergence of convolutional structure in neural networks

Significance The interplay between data symmetries and network architecture is key for efficient learning in neural networks. Convolutional neural networks perform well in image recognition by exploiting the translation invariance of images. However, learning convolutional structure directly from data has proven elusive. Here we show how a neural network trained on translation-invariant data can autonomously develop a convolutional structure. Our work thus shows that neural networks can learn representations that exploit the data symmetries autonomously, by exploiting higher-order data statistics. We finally identify the maximization of non-Gaussianity as a guiding principle for representation learning in our model, linking discriminative vision tasks and unsupervised feature extraction.