FairALM: Augmented Lagrangian Method for Training Fair Models with Little Regret

Algorithmic decision making based on computer vision and machine learning\ntechnologies continue to permeate our lives. But issues related to biases of\nthese models and the extent to which they treat certain segments of the\npopulation unfairly, have led to concern in the general public. It is now\naccepted that because of biases in the datasets we present to the models, a\nfairness-oblivious training will lead to unfair models. An interesting topic is\nthe study of mechanisms via which the de novo design or training of the model\ncan be informed by fairness measures. Here, we study mechanisms that impose\nfairness concurrently while training the model. While existing fairness based\napproaches in vision have largely relied on training adversarial modules\ntogether with the primary classification/regression task, in an effort to\nremove the influence of the protected attribute or variable, we show how ideas\nbased on well-known optimization concepts can provide a simpler alternative. In\nour proposed scheme, imposing fairness just requires specifying the protected\nattribute and utilizing our optimization routine. We provide a detailed\ntechnical analysis and present experiments demonstrating that various fairness\nmeasures from the literature can be reliably imposed on a number of training\ntasks in vision in a manner that is interpretable.\n