RED : Looking for Redundancies for Data-Free Structured Compression of Deep Neural Networks

Deep Neural Networks (DNNs) are ubiquitous in today's computer vision\nland-scape, despite involving considerable computational costs. The mainstream\napproaches for runtime acceleration consist in pruning connections\n(unstructured pruning) or, better, filters (structured pruning), both often\nrequiring data to re-train the model. In this paper, we present RED, a\ndata-free structured, unified approach to tackle structured pruning. First, we\npropose a novel adaptive hashing of the scalar DNN weight distribution\ndensities to increase the number of identical neurons represented by their\nweight vectors. Second, we prune the network by merging redundant neurons based\non their relative similarities, as defined by their distance. Third, we propose\na novel uneven depthwise separation technique to further prune convolutional\nlayers. We demonstrate through a large variety of benchmarks that RED largely\noutperforms other data-free pruning methods, often reaching performance similar\nto unconstrained, data-driven methods.\n