Unveiling User Satisfaction and Creator Productivity Trade-Offs in Recommendation Platforms

Summary

This paper studies a model of content creation and consumption on arbitrary online user-generated content platforms (e.g., YouTube, TikTok). It focuses on a type of Cournot competition in which creators mainly modify their creation volume. The paper provides a description of this model, a theoretical analyses of the Pure Nash Equilibria in this setting, an analysis of how platform designers might use mechanism design to balance consumer and creator utility, a framing of this balancing problem as an optimization problem solvable via (approximated) gradient descent, and experiments using purely synthetic data (sampled "users" with Gaussian preferences) and empirical data (users with preferences from the MovieLens dataset, popular in recommender systems).

Strengths

Overall, this paper provides a strong overall contribution and number of results and insights that will be of interest to a number of different communities -- researchers interested in UGC and online communities, mechanism design, ML for social media, etc. The clarity is high throughout. The paper begins with strong and well argued motivation, the organization is helpful, and in general the overall narrative of the paper is clear. In terms of novelty, this paper directly builds on a previous modelling work, but is very upfront about highlighting what the main differences and additions are in terms of contribution. The experiments seem to especially build off the design of [40] (esp. in terms of the synthetic data + MovieLens combination), which might be worth mentioning if that is intentional. Overall, the potential significance of this work seems potentially high.

Weaknesses

Overall, I expect readers won't have any major concerns with the theoretical results or experiments (see some minor questions below in the Questions section). Rather, the main threat to the significance of this paper is making the case that that a Cournot-style is actually common in the UGC platforms being invoked here. Of course, even if only a few platforms really end up being well-described by the model, the contribution is still very meaningful. That said, a few specific concerns with the current draft: - a number of specific platforms are mentioned by name: YouTube, TikTok, Netflix, Spotify, and MovieLens. - Only data from MovieLens is used (which is very reasonable -- it's a very popular dataset for academic work for good reason). - However, the named platforms vary quite a bit in terms of their actual creator competition, i.e. one would expect the incentives of a platform like Netflix (which also acts a creator agent, sometimes with substantially higher budget than other creators) to differ quite a bit from TikTok See "Questions" section below for some specific questions about this concern that I think are likely to be in scope of a revision. With this critique in mind -- that certain platforms might violate the assumptions needed for the model to work well -- I think the current draft may overstate the generality of the conceptual insight.

Questions

A few very specific questions about the model (with the caveat that of course anything along the lines of using empirical data from major platforms and/or trying to frame this model as predictive for an entire spectrum of platform types is probably out of scope) - What is the strongest evidence that any of these major platforms follow Cournot competition like dynamics? - What is the impact of platform-as-creator dynamics, such on Netflix? - More generally, it would be helpful to explicitly state how resource heterogeneity amongst creators or budget heterogeneity amongst consumers may or may not cause issues for the use of this model. - To what extent would we expect results to hold if we did have access to MovieLens-style observational data from e.g. YouTube? Overall, these are not "existential" questions per se, but some attempt to clarify could strengthen the draft quite a bit.

Rating

7

Confidence

4

Soundness

3

Presentation

4

Contribution

3

Limitations

I do think the current draft could do more to justify the strength of the conceptual claims and/or hold a bit more space to explicitly discuss limitations (primarily, how well requisite assumptions hold across the platforms of interest). See above (Questions).

Summary

This paper studies the problem of the tradeoff between users’ satisfaction and creators’ engagement. Authors first define the traffic competition of creators on user-generated content platforms as a Cournot Content Creation Competition (C4) and establish corresponding PNEs. Based on PNEs, this work identifies the tradeoff between users’ and creators’ engagement and proposes the offline optimization solution to achieve the maximum social welfare by adjusting the exploration level of matching. Theoretical and empirical results are provided to support the effectiveness.

Strengths

1. Authors theoretically model the traffic competition among content creators as a C4 game, identify the tradeoff of user and creator engagement based on their theory, and finally find the optimal platform intervention to maximize the social welfare with the optimization method. Necessary proofs are provided with details. 2. Based on the synthetic and real-world datasets, authors validate the phenomenon of the user-creator tradeoff (Figure 1) and the benefit of optimizing \beta (Figure 2). Authors also provide the results in Appendix with different $\lambda$ in the objective $W_\lambda$ to investigate the sensitivity of their solution when the target is changed. 3. The manuscript is well-organized and easy to follow.

Weaknesses

1. Some assumptions are too strong, including (a) basic setups: Creators are producing contents with the same frequency and the same cost (only relate to the frequency) all the time. (b) platform intervention: all users contribute one unit amount of traffic, neglecting the dominant position of active users. 2. Although the effectiveness is guaranteed by the theory and empirical study on small datasets, authors should also present the potential of the solution to be applied in the practical scenarios, e.g., how is the efficiency of the optimization, how to conduct the daily update of intervention strategy. 3. Existing works have studied the C3 game. Authors may declare their unique contribution and improvement by considering “Cournot Competition” in their theory establishment and compare with previous methods in empirical validation.

Questions

1. How is the efficiency of the proposed solution when the dataset includes millions of users? 2. What are the differences between C4 and previous C3 game? How does C4 benefit from the additional “Cournot” setting?

Rating

4

Confidence

3

Soundness

2

Presentation

3

Contribution

2

Limitations

1. Limited practical value. The assumption is too strong, and the experiments are constrained on small dataset with 1,000 users. 2. Unclear distinct contributions compared with previous works.

Summary

The authors propose a new game-theoretical model Cournot Content Creation Competition ($C^4$), that studies the relation between the matching strategy of user-generated content (UGC) platforms and the production willingness of the platform’s content creators. Under certain assumptions, the authors show that the game has a unique Pure Nash equilibrium, and show that increasing matching accuracy elevates user satisfaction but also decreases the overall volume of content creation. Building on this tradeoff, the authors propose an optimization approach that balances the two objectives, providing both theoretical analysis and empirical simulations.

Strengths

- Overall well-written. - Interesting insight on the tradeoff between user satisfaction and creator engagement shown by theoretical analysis.

Weaknesses

- (Main) Model might be too simplistic - authors assume users consistent produce work of same topic & quality and only changes the production volume. - (Minor) The authors associate user satisfaction as a short-term goal for the platform and creation volume as a long-term goal for the platform. The authors make an argument for this in line 188-197, although I’m still not fully convinced: - The main imbalance that I feel comes from the fact that when I think of *long-term* goals of a platform, it's fundamentally intertwined with the ability for platforms to attract new and keep existing users, which comes from a user standpoint and not from a "content volume" standpoint. I get the authors argument when they mention how content creation frequency might harm user satisfaction (line 192 “users can hardly be satisfied by their previously consumed material”). However, given individual's limited attention span, I think this only happens when the number of creators are quite limited, and that it's unclear that a decrease of production frequency from say 2 weeks -> 3 weeks will result in a significant harm to the *long term* viability of a platform causing users to drop out in the long-term. - In general, this seems to point to an alternative model where the content volume *comes in* the user's utility model, where users’ utility are not only determined by how they liked the recommended content (which is the utility considered in the paper) but also by the availability of content on the platform, and they might drop out of the platform when their utility falls below a certain level. From this lens, it's less clear that this is a short-term v.s. long-term issue. - Typo: i =1 -> j=1 in line 92, third and fourth -> second and fourth in line 347

Questions

- Can the authors elaborate on the point on short v.s. long-term goals in the weakness section above? Specifically, why is user satisfaction short-term concern and content generation frequency long-term concern. - In practical scenarios when an increase in content comes with quality degradations and topic differentials, are there intuitions on whether and under what scenarios the result can continues to hold? - Is collusion between content creators a potential problem here?

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors adequately addressed the limitations.

Response

I thank the authors for their response. Most of my concerns are addressed, and I'm happy to keep my score, leaning towards acceptance of the paper.