Summary
In this paper, the authors studied the tradeoff between user and item fairness in a recommendation setting. They proposed a constrained optimization problem that imposes user fairness as its objective and incorporates item fairness as its constraints. The authors also identified that (1) when user preferences are diverse, item fairness can be easily achieved; (2) when there is mis-estimation of user preference, imposing item constraint can lead to further cost for the users. Finally, the authors illustrated their findings using arXiv data.
Strengths
- The problem of multi-sided fairness is an important problem, but most existing literature focus on studying single-sided fairness. Understanding the price of fairness is a meaningful problem for decision-makers in practice.
Weaknesses
1. The model and framework used in this paper would require significantly more justification. More specifically,
- It is unclear why the authors choose to consider an optimization problem which maximizes normalized user preference as the objective subject to item fairness constraints. To me this is a rather counterintuitive choice and requires more motivation. Why not solving a dual-objective problem and treating user/item fairness in the same fashion? Alternatively, why not maximizing online platform's recommendation quality?
- The assumption that user and item share the same utility $w_{i,j}$ is too strong and can hardly hold in practice. In recommendation systems, a number of factors such as pricing, rankings, utility models, etc., could impact the item/user utilities in different ways. (See prior works such as [10, 11, 32], all of which use different definitions of user/item utilities.)
- The fairness notion that the authors adopt for users/items resemble a min-max type of fairness notion. This is again quite restrictive and requires more justification. Does your framework and results hold under alternative notions?
2. The theoretical results of this paper do not have sufficient technical contributions. For example,
- Proposition 2 basically uses the properties of BFS in LPs.
- Theorems 3 and 4 are only shown on a restrictive example (where there are only 2-3 types of users with opposing preferences in a pre-defined form). This raises the question of whether the theoretical results/insights can be extended to more general setups. Due to the restrictive setup and assumptions, the insights might also not have much practical relevance.
3. The price of fairness of multi-sided recommendations is a topic already studied in prior works. The insights provided in this work are not particularly surprising, nor distinguishable from prior works.
- For example, in [11] the authors also studied price of fairness and show that the price relates to the misalignment of platform/item/user objectives and designed an algorithm that resolves the issue of having unknown user/item data.
- The phenomena described in this work, such as more diverse user preferences naturally inducing item fairness, is also rather natural especially when min-max fairness is imposed, which encourages uniform item exposure. I'd expect such statement would fail to work if a different type of item fairness is considered.
4. No algorithm has been studied or introduced in this work. The empirical study of arXiv data merely serves to evaluate Problem (1) on arXiv data, but the model and framework itself already raises questions and requires extensive justification.
Overall, I think this work would require significantly more work in both its model and results and would recommend rejection.