On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Location

In this paper we challenge the common assumption that convolutional layers in\nmodern CNNs are translation invariant. We show that CNNs can and will exploit\nthe absolute spatial location by learning filters that respond exclusively to\nparticular absolute locations by exploiting image boundary effects. Because\nmodern CNNs filters have a huge receptive field, these boundary effects operate\neven far from the image boundary, allowing the network to exploit absolute\nspatial location all over the image. We give a simple solution to remove\nspatial location encoding which improves translation invariance and thus gives\na stronger visual inductive bias which particularly benefits small data sets.\nWe broadly demonstrate these benefits on several architectures and various\napplications such as image classification, patch matching, and two video\nclassification datasets.\n