The success of convolutional neural networks (CNNs) in computer vision\napplications has been accompanied by a significant increase of computation and\nmemory costs, which prohibits its usage on resource-limited environments such\nas mobile or embedded devices. To this end, the research of CNN compression has\nrecently become emerging. In this paper, we propose a novel filter pruning\nscheme, termed structured sparsity regularization (SSR), to simultaneously\nspeedup the computation and reduce the memory overhead of CNNs, which can be\nwell supported by various off-the-shelf deep learning libraries. Concretely,\nthe proposed scheme incorporates two different regularizers of structured\nsparsity into the original objective function of filter pruning, which fully\ncoordinates the global outputs and local pruning operations to adaptively prune\nfilters. We further propose an Alternative Updating with Lagrange Multipliers\n(AULM) scheme to efficiently solve its optimization. AULM follows the principle\nof ADMM and alternates between promoting the structured sparsity of CNNs and\noptimizing the recognition loss, which leads to a very efficient solver (2.5x\nto the most recent work that directly solves the group sparsity-based\nregularization). Moreover, by imposing the structured sparsity, the online\ninference is extremely memory-light, since the number of filters and the output\nfeature maps are simultaneously reduced. The proposed scheme has been deployed\nto a variety of state-of-the-art CNN structures including LeNet, AlexNet, VGG,\nResNet and GoogLeNet over different datasets. Quantitative results demonstrate\nthat the proposed scheme achieves superior performance over the\nstate-of-the-art methods. We further demonstrate the proposed compression\nscheme for the task of transfer learning, including domain adaptation and\nobject detection, which also show exciting performance gains over the\nstate-of-the-arts.\n