Interpolating Item and User Fairness in Multi-Sided Recommendations

Summary

The paper addresses the challenge of balancing multiple stakeholder interests in online recommendation systems, which include platforms, items (sellers), and users (customers). To tackle this challenge, the authors introduce a novel fair recommendation framework, FAIR, formulated as a constrained optimization problem to balance these competing interests. They further explore this problem in a dynamic online setting, introducing FORM, a low-regret algorithm that simultaneously learns user preferences and enforces fairness in real-time recommendations. Through theoretical analysis and a real-world case study, the paper demonstrates that FORM can maintain platform revenue while achieving fairness for both items and users, contributing to more sustainable and inclusive digital platforms.

Strengths

The paper tackles the practically relevant and challenging problem of balancing platform revenue, user fairness, and item fairness The paper proposes an online algorithm to learn parameters of the problem while ensuring some given fairness constraints, not merely assuming that the parameters are given as done in most existing papers The paper provides sound and thorough regret analysis of the proposed algorithm The empirical results on real-world data shows that the proposed method effectively and flexibly control user-item fairness while maximizing the platform revenue

Weaknesses

This is not a weakness specific to this paper, but choosing the right fairness notion and the values of $\delta$ is non-trivial Even though the paper performs offline experiments on public real-world datasets, it fails to perform any online production A/B test; so it might be risky to be overly optimistic about the empirical results of the paper until the proposed approach is verified in an online environment and provides some tangible business benefits

Questions

How can we reasonably choose the right fairness notion and the values of $\delta$ in practice? Do the authors have any plan to deploy and evaluate the proposed algorithms in some real production environment?

Rating

6

Confidence

2

Soundness

3

Presentation

4

Contribution

3

Limitations

Yes

Summary

This paper focuses on an important and interesting research direction: achieving multi-sided fairness for recommendation. Specifically, the authors aim to answer two research questions: (1) What constitutes a fair recommendation within a multi-sided platform? and (2) How would a platform implement a fair recommendation in a practical online setting? The contributions of this work include a novel multi-sided perspective that achieves within-group and across-group fairness among users/items, and also an online recommendation algorithm for multi-sided fairness. Empirical and theoretical results show the advantage of the authors' contributions over prior baselines, especially in online settings.

Strengths

1. The motivation of this paper is clear, and it is well-written. The theoretical proof appears solid. 2. Prior works on fairness-aware recommendation generally focus on either a single side or an offline setting. This work combines both aspects and proposes a novel framework and algorithm, which is useful in real-world settings. 3. The authors compared their method with some well-studied offline and online fairness-aware recommendation methods, demonstrating that their proposed algorithm achieves a better tradeoff between fairness and revenue.

Weaknesses

1. After doing a quick search on "multisided fairness + recommendation", it seems some related works [1, 2, 3] are missing. Even though the setting might be different, the authors should mention and discuss them somewhere in the paper since the topics seem related. 2. An efficiency comparison on the algorithm running time is missing. Ref: [1] Naghiaei et al. CPFair: Personalized Consumer and Producer Fairness Re-ranking for Recommender Systems, SIGIR, 2022. [2] Wu et al. Joint Multisided Exposure Fairness for Recommendation. SIGIR, 2022. [3] Wu et al. A multi-objective optimization framework for multi-stakeholder fairness-aware recommendation. ACM Transactions on Information Systems, 2022.

Questions

1. Can the proposed method be applied to the scenario in which each user/item has multiple sensitive attributes? For instance, there may be multiple facets in describing a user, in terms of race, gender, occupation, age, income level, ... The number of facets here may be large sometimes, and it is hard to treat a specific combination of multiple facets as a single type. How does this algorithm deal with such "intersectional fairness" (e.g., to achieve fairness not only to "black"/"women"/"nurse"/"elder" separately, but also to the intersectional demographic groups)?

Rating

6

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

1. In the checklist, the authors claim that "We have clearly stated the assumptions needed for our theoretical results and the setups adopted for our numerical experiments" and "There is no negative societal impact of the work performed". However, these claims are too strong and I actually do not think the limitations and negative societal impact of this work are clearly or comprehensively discussed. For instance, one of the important challenges in recommendation is the "cold-start" problem, the author did not discuss whether their algorithm is capable of applying in this scenario, or it is also some limitation and future work. Also, one potential negative societal impact to me is the leakage of user/item sensitive information as the algorithm needs to use the "type" information. This is also not discussed or mentioned.

Summary

This paper presents a novel fairness recommendation framework called FAIR, and an online algorithm called FORM for solving multi-stakeholder fairness problems. The main contributions include 1. The FAIR framework is proposed to balance the platform revenue and the fairness of multi-stakeholders (items and users) through the form of constrained optimization problems. The framework can flexibly define fair solutions for items/users and adjust the trade-off between platform goals and stakeholder fairness.2. A FORM algorithm is designed for simultaneous learning and fair recommendation in online settings. The algorithm balances learning and fairness by relaxing the fairness constraints and introducing random exploration.3. Theoretical analyses demonstrate that the FORM algorithm achieves sub-linear regret in terms of both gain and fairness.4. A case study on Amazon review data verifies the effectiveness of the proposed method. The paper solves the fairness problem in multi-party recommender systems, and the proposed method can flexibly weigh the interests of multiple parties while maintaining the platform's revenue and demonstrates good performance in both theory and experiment. This is an important research direction, which is significant for the fairness of practical recommender systems.

Strengths

1. The needs of multiple stakeholders (users, items) are considered simultaneously and Fairness parameters can be adjusted as needed. 2. Theoretical bounds on algorithmic performance are effectively proven. 3. It has a degree of practicality, with case studies demonstrating application to real data.

Weaknesses

1. The computational complexity of the algorithm, especially its scalability in large-scale systems, is not discussed in detail in the paper. 2. The choice of δ_I and δ_U may require a lot of experiments to find suitable values. 3. The model assumes that the user type is known, which may not always hold in practical applications. 4. the paper mainly focuses on short-term fairness and does not delve into the impact of long-term fairness. 5. this paper mainly uses gain and fairness regret as evaluation metrics and more metrics may be needed to fully evaluate system performance.

Questions

1. The paper assumes that user types are fixed, but in practice, user preferences may change over time. 2. the universality of the fairness definition: is the definition of fairness proposed in the thesis applicable to all types of recommender systems? 3. the case study uses only one category of data from Amazon, is it sufficient to demonstrate the universality of the approach?

Rating

7

Confidence

3

Soundness

4

Presentation

3

Contribution

3

Limitations

please refer to "weakness"

Summary

The paper introduces a fair recommendation framework, FAIR, for balancing the interests of multiple stakeholders in recommendation systems, namely the platform, sellers, and users. The framework is formulated as a constrained optimization problem that addresses fairness concerns for both items and users in a dynamic online setting. The paper proposes a no-regret algorithm, FORM, which ensures real-time learning and fair recommendations. The effectiveness of the framework and the algorithm is demonstrated through theoretical analysis and a case studies on real-world data.

Strengths

The paper addresses the often-overlooked complexity of balancing the interests of multiple stakeholders in optimizing recommendation systems. The proposed FAIR framework offers a solution by ensuring fairness for both items and users while maintaining platform revenue. The extension to a dynamic online setting where data uncertainty is present is reasonable. The FORM algorithm's ability to learn and adapt in real-time is a merit for practical applications in environments where user behavior and preferences are nonstationary. This work also provides a robust theoretical foundation for the proposed framework and algorithm, including proofs of sub-linear regret bounds.

Weaknesses

While the paper presents promising results on a dataset with 30 items and 5 user types, it does not thoroughly address how the framework and algorithm would scale to much larger datasets with a large set of items and more diverse user profiles. My main concern is the proposed solution's scalability, a critical factor for real-world deployment but the proposed solution involves solving a constrained optimization problem at every time step, which could be complex and computationally intensive. This complexity might limit the feasibility of implementation for platforms with limited computational resources. I urge the author to include some discussion about the computational complexity of the proposed solution and also the limitations regarding the scalability.

Questions

1. do you think the regret upper bound given in Theorem 3.1 is optimal? If it is not, any idea how can we improve upon this result? 2. If the platform wants to tradeoff between user fairness and item fairness, how does it need to set the parameters delta^{I} and delta^{U}? The current fairness regret simply cares about the maximum of R^I and R^U, what if we care about both, e.g., R^I + \lambda R^U? Does the proposed method still applicable to this situation? 3. I'm wondering if there could be any intrinsic tradeoff between the revenue guarantee and fairness guarantee.

Rating

5

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes

Re: Rebuttal

I thank the authors for their detailed response, which addresses most of my concerns. I maintain my score and weakly lean towards acceptance.