Semi-Supervised Multi-Label Dimensionality Reduction Based on Dependence Maximization

Abstract

Like other machine learning paradigms, multi-label learning also suffers from the curse of dimensionality problem. Multi-label dimensionality reduction can alleviate the problem but they generally ask for sufficient labeled samples. Nevertheless, we often may only have scarce labeled samples and abundant unlabeled samples. In this paper, we propose a <inline-formula> <tex-math notation="LaTeX">${S}$ </tex-math></inline-formula>emi-supervised <inline-formula> <tex-math notation="LaTeX">${M}$ </tex-math></inline-formula>ulti-label <inline-formula> <tex-math notation="LaTeX">${D}$ </tex-math></inline-formula>imensionality <inline-formula> <tex-math notation="LaTeX">${R}$ </tex-math></inline-formula>eduction based on <inline-formula> <tex-math notation="LaTeX">${d}$ </tex-math></inline-formula>ependence <inline-formula> <tex-math notation="LaTeX">${m}$ </tex-math></inline-formula>aximization approach (SMDRdm in short). SDMRdm assumes the semantic similarity and feature similarity of multi-label samples are inter-depended. SMDRdm first applies label propagation on a neighborhood graph composed with labeled and unlabeled samples to obtain the soft labels of unlabeled samples, and then measures the semantic similarity between all the training samples (including unlabeled ones) based on these soft labels and available labels of labeled samples. Next, it measures the feature similarity between samples in the subspace projected by the target projective matrix, instead of the original high-dimensional feature space. After that, it maximizes the dependence between these two types of similarities and incorporates the dependence into linear discriminant analysis to optimize the target projective matrix. Experiments on publicly accessible multi-label data sets demonstrate that SMDRdm achieves more prominent results than other related approaches across various evaluation metrics. In addition, the empirical study also shows the semantic similarity between samples derived from soft labels works better than that derived from scarce available labels.