Summary
This paper studies fair prediction tasks where fairness is defined on some partition of the data points into groups (by gender or race, for examples). The paper studies a meta-learning framework that only needs access to group labels for the validation set rather than the entire training dataset. An outer hypergradient optimization of minibatch-level example weights is used to optimize for fairness (on the validation loss), interwoven with standard gradient optimization of the model parameters for the typical minibatch loss (given the weights).
Previous work has deployed this approach with various predetermined weighting schemes for the outer optimization. The current work identifies a challenge with the stability of such approaches — in particular, often during an outer optimization there may exist a group whose validation loss hypergradient is not aligned with the overall hypergradient, leading to potentially unstable learning and failure to converge to the Pareto front (that is, the undominated groupies loss frontier). To address this challenge, the current work proposes a first stage (conducted for several epochs) wherein the outer optimization is conducted by Nash bargaining between the groups: Roughly speaking, a hypergradient is selected which maximizes the product of alignment between the groups, relative to a default alternative of no change to the re-weighing hyperparameters. This is done for some time, ideally until the agents are able to converge to the Pareto front, at which time a secondary optimization for a particular fairness objective similar to prior work is pursued as a fine-tuning.
The method is validated in theory by demonstrating Pareto improvement under smoothness and boundedness conditions. Also, experiments are conducted on synthetic and real-world data. The benefit of the synthetic experiments is that the Pareto front can be explicitly computed, and that the method succeeds in converging to the front. The real world experiments show some improvements, though not universal or always dramatic, in fairness for predictive tasks including with unbalanced data, compared to previous meta-learning approaches that do not utilize Nash Bargaining.
Strengths
The paper does a good job of identifying a serious challenge to the learning stability of prior meta learning techniques for fairness. The challenges of hypergradient conflict is intuitive, coherent, and well described. The methods proposed, inspired by Nash Bargaining, are reasonable and original for resolving these conflicts.
The theoretical work is a nice contribution, providing both closed form solutions for the bargaining step and a well-grounded argument for Pareto improvement throughout the process.
The paper does a good job of presenting a lot of results in a clear way with attention placed on visualizations and comprehensive tables for the benefit of the reader.
Several different datasets are used and presented to do a good job of not overstating performance based on a single dataset.
In terms of impact and significance, group fairness-aware predictive models are of clear importance to the community, and the paper addresses one promising technique for dealing with the difficult case where group labels are not generally available for the entire training dataset. The technique proposed seems promising for continued development in contexts of fairness as well as simple problems of class imbalance.
Weaknesses
Lots of empirical results and observations on synthetic data and for a particular model architecture, but any specifics about the synthetic data and model architecture are hidden in the appendix. Of particular concern is the lack of discussion around limitations of the scale of the experiments, given that the proposed method introduces additional complexity into the training process designed to address instability of existing meta learning approaches. The real-world dataset examples use very small MLPs with a single 128 neuron hidden layer, and the image examples from CIFAR only use 5,000 or 500 training examples. Similarly, the real-world experiments use (I believe) at most 5 groups.
Discussion around line 275 “Our experiments show…does not deviate the model from the Pareto front…” I think it should be noted explicitly and emphasized that these experiments are on synthetic examples constructed for purpose of simplicity of analysis, so this trend should not be taken as a given in real-world data where one cannot necessarily even calculate the Pareto front to be able to verify the property.
It would be nice to see some evaluation of the frequency with which bargaining fails in the real-world data, as well as the relative importance for resolving hypergradient conflicts between (1) constraining optimization to the agreement set A, and (2) optimizing within A particularly along the Nash objective.
Typos/grammar:
1. “hypergradeint” instead of “hypergradient” on line 91.
2. Line 144-145, found the phrasing a little confusing
3. Line 271 “if encountered one”
Questions
Q1. Referring to the discussion around lines 151-160. What is the relative significance for the empirical optimization results for simply constraining the hypergradient optimization to the alternative set A [requiring improvement for all groups] versus the selection of the Nash objective in particular to optimize within A?
Q2. Also, how common is it for A to be empty, and would you expect this to become increasingly the case when looking at larger numbers of groups? It seems that in the experiments (referring to the Appendix A.6 details) there are only ever <= 5 groups, is it possible that this approach could stall or encounter more challenges for larger numbers of groups?