Group variable selection via ℓp, 0 regularization and application to optimal scoring

Abstract

The need to select groups of variables arises in many statistical modeling problems and applications. In this paper, we consider the ℓp,0-norm regularization for enforcing group sparsity and investigate a DC (Difference of Convex functions) approximation approach for solving the ℓp,0-norm regularization problem. We show that, with suitable parameters, the original and approximate problems are equivalent. Considering two equivalent formulations of the approximate problem we develop DC programming and DCA (DC Algorithm) for solving them. As an application, we implement the proposed algorithms for group variable selection in the optimal scoring problem. The sparsity is obtained by using the ℓp,0-regularization that selects the same features in all discriminant vectors. The resulting sparse discriminant vectors provide a more interpretable low-dimensional representation of data. The experimental results on both simulated datasets and real datasets indicate the efficiency of the proposed algorithms.