Training Well-Generalizing Classifiers for Fairness Metrics and Other Data-Dependent Constraints

Classifiers can be trained with data-dependent constraints to satisfy\nfairness goals, reduce churn, achieve a targeted false positive rate, or other\npolicy goals. We study the generalization performance for such constrained\noptimization problems, in terms of how well the constraints are satisfied at\nevaluation time, given that they are satisfied at training time. To improve\ngeneralization performance, we frame the problem as a two-player game where one\nplayer optimizes the model parameters on a training dataset, and the other\nplayer enforces the constraints on an independent validation dataset. We build\non recent work in two-player constrained optimization to show that if one uses\nthis two-dataset approach, then constraint generalization can be significantly\nimproved. As we illustrate experimentally, this approach works not only in\ntheory, but also in practice.\n