Feature selection considering Uncertainty Change Ratio of the class label

Abstract

Abstract The topic of feature selection in high-dimensional data sets has attracted considerable attention. Feature selection can reduce the dimension of feature and improve the prediction accuracy of the classification model. Information-theoretical-based feature selection methods intend to obtain classification information regarding class labels from the already-selected feature subset as much as possible. Existing methods focus on the reduced uncertainty of class labels while ignoring the change of the remained uncertainty of class labels. In the process of feature selection, the large reduced uncertainty of class labels does not signify the few remained uncertainty of class labels when different candidate features are given. In this paper, we analyze the difference between the reduced uncertainty of class labels and the remained uncertainty of class labels and propose a new term named Uncertainty Change Ratio that considers the change of uncertainty of class labels. Finally, a novel method named Feature Selection considering Uncertainty Change Ratio (UCRFS) is proposed. To prove the classification superiority of the proposed method, UCRFS is compared to three traditional methods and four state-of-the-art methods on fourteen benchmark data sets. The experimental results demonstrate that UCRFS outperforms seven other methods in terms of average classification accuracy, AUC and F1 score.