Feature selection is one of the important topics of machine learning, and it has a wide range of applications in data preprocessing. At present, feature selection based on <inline-formula><tex-math notation="LaTeX">$\ell _{2,1}$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mo>,</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq1-3297226.gif"/></alternatives></inline-formula>-norm regularization is a relatively mature method, but it is not enough to maximize the sparsity and parameter-tuning leads to increased costs. Later scholars found that the <inline-formula><tex-math notation="LaTeX">$\ell _{2,0}$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mo>,</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq2-3297226.gif"/></alternatives></inline-formula>-norm constraint is more conductive to feature selection, but it is difficult to solve and lacks convergence guarantees. To address these problems, we creatively propose a novel Outliers Robust Unsupervised Feature Selection for structured sparse subspace (ORUFS), which utilizes <inline-formula><tex-math notation="LaTeX">$\ell _{2,0}$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mo>,</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq3-3297226.gif"/></alternatives></inline-formula>-norm constraint to learn a structured sparse subspace and avoid tuning the regularization parameter. Moreover, by adding binary weights, outliers are directly eliminated and the robustness of model is improved. More importantly, a Re-Weighted (RW) algorithm is exploited to solve our <inline-formula><tex-math notation="LaTeX">$\ell _{p}$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mi>p</mml:mi></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq4-3297226.gif"/></alternatives></inline-formula>-norm problem. For the NP-hard problem of <inline-formula><tex-math notation="LaTeX">$\ell _{2,0}$</tex-math><alternatives><mml:math><mml:msub><mml:mi>ℓ</mml:mi><mml:mrow><mml:mn>2</mml:mn><mml:mo>,</mml:mo><mml:mn>0</mml:mn></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq5-3297226.gif"/></alternatives></inline-formula>-norm constraint, we develop an effective iterative optimization algorithm with strict convergence guarantees and closed-form solution. Subsequently, we provide theoretical analysis about convergence and computational complexity. Experimental results on real-world datasets illustrate that our method is superior to the state-of-the-art methods in clustering and anomaly detection tasks.
Paper
Full text
Outliers Robust Unsupervised Feature Selection for Structured Sparse Subspace
Semantic Scholar · Computer Science · 2024
Abstract
Feature selection is one of the important topics of machine learning, and it has a wide range of applications in data preprocessing. At present, feature selection based on <inline-formula><tex-math notation="LaTeX">$\ell _{2,1}$</tex-math><alternatives>mml:mathmml:msubmml:miℓ</mml:mi>mml:mrowmml:mn2</mml:mn>mml:mo,</mml:mo>mml:mn1</mml:mn></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq1-3297226.gif"/></alternatives></inline-formula>-norm regularization is a relatively mature method, but it is not enough to maximize the sparsity and parameter-tuning leads to increased costs. Later scholars found that the <inline-formula><tex-math notation="LaTeX">$\ell _{2,0}$</tex-math><alternatives>mml:mathmml:msubmml:miℓ</mml:mi>mml:mrowmml:mn2</mml:mn>mml:mo,</mml:mo>mml:mn0</mml:mn></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq2-3297226.gif"/></alternatives></inline-formula>-norm constraint is more conductive to feature selection, but it is difficult to solve and lacks convergence guarantees. To address these problems, we creatively propose a novel Outliers Robust Unsupervised Feature Selection for structured sparse subspace (ORUFS), which utilizes <inline-formula><tex-math notation="LaTeX">$\ell _{2,0}$</tex-math><alternatives>mml:mathmml:msubmml:miℓ</mml:mi>mml:mrowmml:mn2</mml:mn>mml:mo,</mml:mo>mml:mn0</mml:mn></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq3-3297226.gif"/></alternatives></inline-formula>-norm constraint to learn a structured sparse subspace and avoid tuning the regularization parameter. Moreover, by adding binary weights, outliers are directly eliminated and the robustness of model is improved. More importantly, a Re-Weighted (RW) algorithm is exploited to solve our <inline-formula><tex-math notation="LaTeX">$\ell _{p}$</tex-math><alternatives>mml:mathmml:msubmml:miℓ</mml:mi>mml:mip</mml:mi></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq4-3297226.gif"/></alternatives></inline-formula>-norm problem. For the NP-hard problem of <inline-formula><tex-math notation="LaTeX">$\ell _{2,0}$</tex-math><alternatives>mml:mathmml:msubmml:miℓ</mml:mi>mml:mrowmml:mn2</mml:mn>mml:mo,</mml:mo>mml:mn0</mml:mn></mml:mrow></mml:msub></mml:math><inline-graphic xlink:href="wang-ieq5-3297226.gif"/></alternatives></inline-formula>-norm constraint, we develop an effective iterative optimization algorithm with strict convergence guarantees and closed-form solution. Subsequently, we provide theoretical analysis about convergence and computational complexity. Experimental results on real-world datasets illustrate that our method is superior to the state-of-the-art methods in clustering and anomaly detection tasks.