User-Creator Feature Polarization in Recommender Systems with Dual Influence

Summary

This paper models dynamics of both users and creators in a recommender system. The user features shift in the direction of the content recommended to them. The creator dynamics are strategically motivated i.e. they try to align content to attract their audience, to increase profit. The authors then provide sufficient conditions for this model of dual dynamics to converge to polarization under a natural assumption that each creator has some non zero probability of being recommended to every user. The paper then discusses four real world recommendation designs, and whether they cause polarization or multiple clusters etc. They also provide results on synthetic and Movielens data complementing theory results and show that certain recommender designs do lead to polarization vs diverse clusters.

Strengths

- This paper is the first to consider dynamics of both users and creators in a recommender systems and provides sufficient analytic conditions for polarization - They apply this theory to 4 natural designs: (1) Top-k, (2) Truncation, (3)Diversity boosting and (4) Lower bounding probability. They show that rules (3, 4) lead to polarization and rule (1) leads to diverse clusters. This section is particularly insightful. - The experimental evaluation with synthetic and Movielens data is also insightful and complements the theory. The softmax probability leads to diminishing creator and recommendation diversity over time. They also study top-k probability and show how lower k is better for higher creator diversity, recommendation relevance.

Weaknesses

A criticism I had while reading the paper are gaps in literature for the discussion on dynamics in recommender systems. In addition to [Eilat & Rosenfeld] referenced in the introduction, [1,2,3,4,5,6] consider creator dynamics in recommender systems. These works assume static user features and provide results on content at equilibrium and user welfare. In the context of these works, it would be beneficial to highlight how your work is the first to consider both creator and user dynamics. [1] A Game-Theoretic Approach to Recommendation Systems with Strategic Content Providers (Ben-Porat and Tennenholtz) [2] Supply-side equilibria in recommender systems (Jagadeesan et al) [3] How Bad is Top-k Recommendation under Competing Content Creators? (Yao et al) [4] Modeling content creator incentives on algorithm-curated platforms (Hron et al) [5] Producers Equilibria and Dynamics in Engagement-Driven Recommender Systems (Acharya et al) [6] User Welfare Optimization in Recommender Systems with Competing Content Creators (Yao et al)

Questions

- I understand the motivation for the form of user update in equation (3) . In each recommendation step an item is recommended and the user preference shifts in that direction, this is like in [Dean and Morgenstern]. Can you motivate the update in Eq (4), is this myopically optimal for the creator to do, and how does it generalize [Eilat & Rosenfeld]? - Minor Typo? For Figure 6, larger $\rho$ seems to lead to higher creator diversity (green curve).

Rating

7

Confidence

5

Soundness

4

Presentation

2

Contribution

4

Limitations

The authors discuss limitations of their results in the paper.

Summary

The paper explores the dynamics between users and content creators in recommender systems, highlighting the dual influence where users’ preferences are shaped by recommendations and creators modify their content to align with what is more likely to be recommended. The study defines a model called user-creator feature dynamics to capture these interactions, demonstrating that such systems are prone to polarization, resulting in a loss of diversity. The paper then examines various approaches to mitigate polarization and improve diversity, finding that relevancy-optimizing methods, such as top-k recommendations, can prevent polarization more effectively than traditional diversity-promoting approaches.

Strengths

The paper provides an interesting perspective by addressing the mutual influence between users and creators in recommender systems. The theoretical results and experimental validation using both synthetic and real-world data look credible. The writing is overall easy to follow.

Weaknesses

1. There are two lines of works focusing on modeling content creator dynamics and user preference evolving dynamics that are neglected by the authors. I listed several representative works and it would be great to include a comprehensive literature review regarding these works in the related work section. 2. One of your main observation (larger $\beta$ leads to higher creator diversity and alleviated polarization) is actually pointed out in [1] under a similar model, where content creators compete for a fixed user population (see section 3.2 in [1]). And another main observation in section 5.3 that smaller $k$ improves diversity does not echo the result in [2], which shows that larger $k$ improves the total creator utilities. It would be better to include some detailed discussions regarding these two works. 3. The user/creator preference updating dynamics need more justifications and empirical evidence. 4. The dynamical model makes some sense to me, but it would be more interesting to understand whether the observations still hold in the presence of noise. If the noisy version is hard to analyze theoretically, additional simulation results could also be valuable. [1]. Modeling Content Creator Incentives on Algorithm-Curated Platforms [2]. How Bad is Top-K Recommendation under Competing Content Creators? [3]. Online recommendations for agents with discounted adaptive preferences [4]. Recommender systems as dynamical systems: Interactions with viewers and creators [5]. Learning from a learning user for optimal recommendations [6]. Supply-side equilibria in recommender systems

Questions

1. In theorem 3.3, how does the convergence rate depending on the temperature parameter $\beta$? I ask this because when $\beta\rightarrow +\infty$, the softmax recommendation strategy is equivalent to the top-1 recommendation strategy. In this case, Proposition 4.2 predicts that the top-1 recommendation lead to $n$ clusters rather than bi-polarization, which seems to contradict theorem 3.3. 2. I do not fully get why the specific forms of function $f$ and $g$ do not affect the analysis. Is it because your main results only depend on the range of $f$ and $g$? 3. In the experiments, the range of $\beta$ is quite conservative. I'm curious about the result under a larger range of $\beta$.

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

yes

Summary

This paper studies how recommendations become polarized over the long run when user and creator features dynamically change over time. The authors theoretically prove that, under the assumption that every creator can be recommended to every user with some non-zero probability, recommender systems will eventually converge to polarization. They also simulate some real-world models, including top-k recommendation, truncation, diversity boosting, and lower-bounding probabilities in a long-term setting. The key observation is that top-k recommendation (i.e., only recommending top-k items to users) can reduce polarization to some extent, while existing diversity-boosting methods will worsen polarization when user/creator features dynamically change over time in the system.

Strengths

1. The authors provide both theoretical and empirical evidence showing that relevance-focused recommendations (as opposed to diversity-focused recommendations), which harm diversity in a static setting, are actually effective in improving diversity in the long term. This observation is somewhat counter-intuitive to previous beliefs, making it very interesting. 2. The authors conducted simulations with both synthetic data and real-world data (i.e., Movielens) using four diversity and relevance-related measures. Additionally, the analysis with sensitivity parameters in softmax is insightful and supports the authors' main claim. 3. Studying diversity in a dynamic setting is novel.

Weaknesses

1. Despite the novelty and interestingness, I have concerns about the key assumptions of the theoretical and empirical analyses. The assumption that all items can be recommended to users is not realistic. In practice, almost all recommender systems rely on top-k recommendations for either effectiveness or resource constraints like screen size. For example, on platforms like Netflix or Amazon, customers can only see a certain number of items on the webpage (i.e., p=0 for items that users can't see). Even if they can scroll down and the system continually recommends new items, they cannot physically see all items in the system. Thus, I believe the top-k setting is the most realistic and natural for real-world scenarios, and this seems like a hole in the authors' analyses. In this sense, the measures for empirical analysis should also only consider top-k items, not all items. 2. For the real-world designs, it would be more extensive if users included trustworthiness-aware recommender systems that consider dynamic/continual settings. For example, [1] consider performance difference between two different user groups when the user/item features are continually updated over time in the systems. 3. For the analysis with Movielens, considering the interaction timestamp in the simulation would more accurately reflect real-world scenarios, for example, for determining the true labels. [1] Yoo et al., Ensuring User-side Fairness in Dynamic Recommender Systems, WWW'24

Questions

1. Please address the points I raised in Weaknesses. 2. (Minor) Are both consensus and bi-polarization conceptually polarization? 3. (Minor) How are the initial user/creator embeddings initialized in the Movielens experiment?

Rating

6

Confidence

4

Soundness

2

Presentation

3

Contribution

3

Limitations

The authors adequately addressed the limitations.

Acknowledging rebuttal

Thanks for the addressing my questions, I stand with my assessment of accepting this paper.

Re: Rebuttal

I thank the authors for their detailed response, which addressed most of my concerns. And I appreciate the summarized table which emphasizes the contribution. I decide to maintain my score, leaning towards acceptance.