Generalized stochastic orderings applied to the study of performance of machine learning algorithms for low quality data

Abstract

Usually, the expected loss minimization criterion is used in order to look for the optimal model that expresses a certain response variable as a function of a collection of attributes. We generalize this criterion, in order to be able to deal also with those situations where a numerical loss function makes no sense or is not provided by the expert. In a first stage, we consider the new framework in standard situations, where both the collection of attributes and the response variables are observed with precision. In a second one, we assume that we are just provided with imprecise information about them (in terms of set-valued data sets). We cast some comparison criteria from the recent literature on learning methods from low-quality data as particular cases of our general approach.