Summary
The authors study strategic content creation in recommendation systems, focusing on the induced game's social welfare. The authors assume that the provider's rewards are entirely determined by the platform's payments, not clicks/engagements. This separation between the ranked results and the creators' incentives facilitates analyzing an expressive game in which the recommender system recommends a list of items (most prior work considers one item) with position bias. Notably, the authors assume that the rewarding mechanism is a mapping from a vector of relevance scores to reward vectors, i.e., $[0,1]^n \rightarrow [0,1]^n$.
The paper considers two classes of merit-based mechanisms, monotone and BRM (and also the BRCM subclass). It showcases evidence against monotone mechanisms, highlighting that in a particular class of games (TvN), the POA of any monotone mechanism is at most $\frac{K}{K+1}$, where $K$ is the length of the list, plus a small $\frac{1}{n}$ factor. Later, for BRCM mechanisms, the authors show that the welfare function is the potential function; thus, the global optimum of the welfare is a PNE (despite the POA could still be $<1$ for some mechanisms).
Finally, the authors describe how to optimize over BRCM mechanisms in the presence of data, and conduct synthetic and semi-synthetic experiments to demonstrate their approach.
Strengths
1. The paper deals with a timely and important topic, and well-connects to previous literature.
2. The paper non-trivially extends previous literature, suggesting new theory and experimental validation.
3. The optimization problem the paper suggests is exciting and new to this literature.
Weaknesses
1. Due to the abundance of mathematical objects and notations, the paper is non-trivial to follow. Perhaps this is inevitable, but I see this as a weakness.
2. The case against monotonicity focuses on a relatively small class of TvN games tailored to the authors' argument. Arguing against monotonicity in (more) general games would be much more convincing.
3. The empirical evaluation lacks proper benchmarks.
Questions
1. An assumption of the model is that impressions/clicks of users and creators' rewards are separated and orthogonal. In reality, the utility function of creators is not always "set" by the platform (as argued in 168,170), but rather determined by the interaction the platform allows. For instance, in Medium, blog traffic means more ads, regardless of relevance. Can the platform decide that a blog is irrelevant and thus, despite its heavy traffic, not share the ad revenue with the blog's creator? As written in my summary, I see this separation as the element facilitating a much more expressive game than those considered in prior work. So my question is: Can the authors justify this modeling assumption?
2. Relating to the previous point, something disturbs me in the explanation in Lines 241-250. Say the $K$ contents creators attract impressions, clicks, etc., and get high rewards. Then, the $K+1$ creators improves their content's relevance, and suddenly the $K$ selected creators suddenly get almost none. I think that this is hard to digest and probably will be deemed unfair from the creators' perspective. Could the authors justify why BRCM mechanisms benefit creators? After all, there are other unfair mechanisms that dictate the content each creator creates, which receives optimal welfare.
3. The paper advocates using BRCM mechanisms and not monotone ones. Noticeably, the intersection between these classes is not empty. But Theorem 1 suggests that the optimal BRCM mechanism that exists due to Corollary 1 is *not* monotone. This brings the question of how natural this mechanism is (the one guaranteed by Corollary 1). Do we want to sacrifice simplicity (and arguably fairness) for a proportion of $\frac{1}{K+1} of the welfare? Since advocating for BRCM is not equivalent to advocating against monotonicity (as the intersection is non-empty), I think the reader could benefit from a more elaborated discussion. Ideally, this paper would have presented evidence beyond social welfare in TvN games, but perhaps this is saved for future work.
4. The welfare function comprises the user welfare, the creators rewards and costs, and the mechanism's payments. But the mechanism's payments are precisely the creator rewards, so they cancel. Consequently, by offering unbounded rewards the platform does not change the welfare, but obviously this will never happen in real life. What part of the modeling forbids that? This seems like a loophole in the modeling. Could the authors justify this welfare modeling? Even if we care about the welfare function defined in this paper, one would expect the platform to minimize its payments for any given welfare level. Could this be injected into the model?
5. In the experimental part, the authors compare BRCM mechanisms with monotone mechanisms. However, for the monotone mechanisms, the authors do not optimize over the hyperparameter $\beta$. I believe an apples-to-apples comparison requires optimizing for both classes of mechanisms. Why didn't the authors do that in their analysis? Additionally, what would be other monotone baselines?
6. In Figure 1(a), the optimized BRCM scores better than the optimal BRCM. How could that be possible?
Minor:
• 23: “benefits\cite{}” -> add space
• Lines 139-140 say "Our objective is to design mechanisms M that: 1. guarantee the existence of PNE, thereby ensuring a stable outcome, and 2. maximize social welfare at the PNE." While Line 135 says that this is not the focus of [1,12,13], I think that [1] primarily addresses the same objectives.
• 155: the parameter $n$ seems to be out of context, as it doesn't appear in the $M^3$ notation.
• 210: ".." -> "."
• 254: $BRCM \in BRM$ -> $BRCM \subset BRM$
Rating
8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Limitations
No limitations