Evaluating Meta-Feature Selection for the Algorithm Recommendation Problem

With the popularity of Machine Learning (ML) solutions, algorithms and data\nhave been released faster than the capacity of processing them. In this\ncontext, the problem of Algorithm Recommendation (AR) is receiving a\nsignificant deal of attention recently. This problem has been addressed in the\nliterature as a learning task, often as a Meta-Learning problem where the aim\nis to recommend the best alternative for a specific dataset. For such, datasets\nencoded by meta-features are explored by ML algorithms that try to learn the\nmapping between meta-representations and the best technique to be used. One of\nthe challenges for the successful use of ML is to define which features are the\nmost valuable for a specific dataset since several meta-features can be used,\nwhich increases the meta-feature dimension. This paper presents an empirical\nanalysis of Feature Selection and Feature Extraction in the meta-level for the\nAR problem. The present study was focused on three criteria: predictive\nperformance, dimensionality reduction, and pipeline runtime. As we verified,\napplying Dimensionality Reduction (DR) methods did not improve predictive\nperformances in general. However, DR solutions reduced about 80% of the\nmeta-features, obtaining pretty much the same performance as the original setup\nbut with lower runtimes. The only exception was PCA, which presented about the\nsame runtime as the original meta-features. Experimental results also showed\nthat various datasets have many non-informative meta-features and that it is\npossible to obtain high predictive performance using around 20% of the original\nmeta-features. Therefore, due to their natural trend for high dimensionality,\nDR methods should be used for Meta-Feature Selection and Meta-Feature\nExtraction.\n