Quantile Encoder: Tackling High Cardinality Categorical Features in Regression Problems

Regression problems have been widely studied in machinelearning literature\nresulting in a plethora of regression models and performance measures. However,\nthere are few techniques specially dedicated to solve the problem of how to\nincorporate categorical features to regression problems. Usually, categorical\nfeature encoders are general enough to cover both classification and regression\nproblems. This lack of specificity results in underperforming regression\nmodels. In this paper,we provide an in-depth analysis of how to tackle high\ncardinality categor-ical features with the quantile. Our proposal outperforms\nstate-of-the-encoders, including the traditional statistical mean target\nencoder, when considering the Mean Absolute Error, especially in the presence\nof long-tailed or skewed distributions. Besides, to deal with possible\noverfitting when there are categories with small support, our encoder benefits\nfrom additive smoothing. Finally, we describe how to expand the encoded values\nby creating a set of features with different quantiles. This expanded encoder\nprovides a more informative output about the categorical feature in question,\nfurther boosting the performance of the regression model.\n