Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders

Major complications arise from the recent increase in the amount of\nhigh-dimensional data, including high computational costs and memory\nrequirements. Feature selection, which identifies the most relevant and\ninformative attributes of a dataset, has been introduced as a solution to this\nproblem. Most of the existing feature selection methods are computationally\ninefficient; inefficient algorithms lead to high energy consumption, which is\nnot desirable for devices with limited computational and energy resources. In\nthis paper, a novel and flexible method for unsupervised feature selection is\nproposed. This method, named QuickSelection, introduces the strength of the\nneuron in sparse neural networks as a criterion to measure the feature\nimportance. This criterion, blended with sparsely connected denoising\nautoencoders trained with the sparse evolutionary training procedure, derives\nthe importance of all input features simultaneously. We implement\nQuickSelection in a purely sparse manner as opposed to the typical approach of\nusing a binary mask over connections to simulate sparsity. It results in a\nconsiderable speed increase and memory reduction. When tested on several\nbenchmark datasets, including five low-dimensional and three high-dimensional\ndatasets, the proposed method is able to achieve the best trade-off of\nclassification and clustering accuracy, running time, and maximum memory usage,\namong widely used approaches for feature selection. Besides, our proposed\nmethod requires the least amount of energy among the state-of-the-art\nautoencoder-based feature selection methods.\n