Improved Bayes Risk Can Yield Reduced Social Welfare Under Competition

Summary

This study highlights the inconsistencies of scaling laws in machine learning when model providers compete with each other in the market. The researchers conducted experiments with varying data representations and found that improving data representation quality may actually decrease the overall predictive accuracy, which is modeled as social welfare, across users in the competitive market. They also generalized experiment to unfair markets and study the market with 2 model providers only. The study raises awareness about the importance of evaluating machine learning models in a market where organizations are competing.

Strengths

This paper presents several strengths, including a clear and well-written introduction that guides the reader to understand their interesting findings and the awareness behind it. Approaches to vary data representation are comprehensive and practical. They provide basic setups to study and offer results empirically and theoretically. Notably, the paper's conclusion offers an interesting insight into why enhancing model accuracy may not necessarily lead to significant social benefits in real-world applications.

Weaknesses

Here are some potential improvements to the original content: 1. The available training resources of model providers in the market are unclear, as well as the basis on which they build their models. If models are trained solely on user-provided datasets, their performance may not vary significantly, which may make the best response trivial. 2. The experiment only covers CIFAR-10 and reduces it to a binary classification task, which warrants further exploration with datasets of higher cardinality. 3. The finding about inefficient translation to social welfare is interesting, but it lacks proper and thorough explanation.

Questions

If the market is competitive, when and why do people care about the social welfare? Organizations are selfish as they only care about earning more share in the market as much as possible. Is there any scenario in practice in machine learning that people care about social welfare more than their own preference or happiness.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

This paper presents a novel finding that challenges the conventional understanding of scaling laws in machine learning in the context of organizational competition. The framework proposed is foundational and can be applied to various scenarios.

Summary

The paper explores the impact of competition on machine learning models and social welfare. It investigates how scaling trends can be altered by competition and how improving data representation quality may not always lead to better predictive accuracy. The authors provide a theoretical framework for analyzing the impact of competition on machine learning models and demonstrate their findings through experiments on real-world datasets. The paper makes contributions to the understanding of the relationship between competition, data representation quality, and predictive accuracy in machine learning models.

Strengths

- The paper provides a comprehensive review of related work and connects to multiple research threads on the welfare implications of algorithmic decisions and competition between data-driven platforms. - The authors apply their framework to three concrete binary classification setups that vary representation quality along different axes, demonstrating the usefulness of their approach. - The paper is well-written and organized, making it easy to follow the authors' arguments and contributions.

Weaknesses

- The paper would benefit from a more thorough evaluation of the proposed framework, including experiments on real-world datasets and comparison to existing methods. - The paper could benefit from a more detailed discussion of the implications of their findings for real-world decision-making scenarios.

Questions

- How does the quality of data representation affect the equilibrium social welfare in the authors' model, and how does this compare to existing models?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

Yes.

Summary

This paper looks and trends that arise in predictive utility across a population as model providers adapt to improving representations. At the core of the paper is an analysis that characterizes the population utility of equilibria as the representation varies in quality, noise, and dimension, which shows that overall utility may not be monotone along these axes. The paper additional contains an empirical study, over synthetic data and CIFAR, which addresses some of the simplifications made in the analysis while showing qualitatively similar trends in overall utility.

Strengths

Topically, this paper addresses an interesting and potentially important question: what is the right way to model the effect that continued improvements to predictive accuracy have on social welfare? Some examples of prior work have pointed to a misalignment between predictive accuracy and social welfare, but the results in this paper are original and distinct, the the best of my knowledge. Namely, "model monoculture" can emerge when competing providers behave rationally, and representations are sufficiently good, leading to a subpopulation that isn't served by any provider. Many of the concerns that I had about the simplified setting when going through the theoretical analysis were addressed to some extent in sections 3.3 and 4, and it seems like the results may be fairly robust to these choices. The writing is polished, clear, and concise.

Weaknesses

A central assumption of the work is that all model providers have access to the same global representation. This makes some sense, but is not all that realistic: despite using on the same underlying techniques and relying on the same public-knowledge advances in learning, different model providers show very different performance metrics. The market reputation factor incorporated in 3.3 does not quite address this, as it is independent of model performance. While the experimental analysis is a nice demonstration that the theoretical results hold in some more realistic settings, there are still some simplifications that weren't explicitly motivated (only linear classifiers, binarized CIFAR). These choices make sense in that they are consistent with the rest of the paper, but the point of the experimental analysis is to show that the results are robust, so they do limit the significance of the work.

Questions

1. If you are able, please explain the rationale behind the global representation used in your work. To what extent do you believe that your results depend on this assumption? 2. Do you have any insight into the likely outcomes if your experiments were to relax some of the conditions that I mentioned above?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

The section 5 at the end of the paper would be a good place to add discussion of the fact that there is certain risk that the results are overly-sensitive to choices made in modeling and experiment design. It currently does not, and limitations are not discussed in much detail elsewhere.

Summary

This paper shows that "social welfare" might (under some settings) be worse-off as the representation quality of machine learning models increase when there are multiple strategic machine-learning-model providers. The paper focuses the analysis on binary classification tasks. The paper assumes that users choose the model provider which gives the most accurate prediction. And multiple strategic machine-learning-model providers compete for the number of users choosing them. The social welfare is defined as the difference between the most accurate model prediction and the true label in expectation. The representation quality is defined as the Bayes risk. The paper shows that under a stylized setting, increasing the representation quality might decrease the social welfare. Then many empirical evaluations are conducted to show that this also happens in several relaxed settings.

Strengths

The paper is well-written and easy to follow. The paper tries to answer an interesting question of what is the effect of representation quality on social welfare when there are competing machine-learning-model providers. The paper provides theoretical analysis on a stylized setting which shows an intriguing phenomenon: when the representation quality increases, the defined social welfare can decrease. And it also provides empirical evaluations to show this phenomenon beyond the restricted setting. This calls for further discussion on whether this happens more broadly and how to address this problem.

Weaknesses

My major concern about this paper is that all the theoretical analysis and empirical evaluation are conducted on settings where all model providers have the same representation. In reality, competing providers do not share their models and they rarely have the same representations. So the conclusion of this paper is based on evidence on a very restricted setting. I am not sure whether binary classification is the right problem to analyze the concerned question. First, if there are only two options, the users can simply look at the two options and make a decision. They do not even need to hop between model providers. Second, in the examples given in the paper, there is no single right choice of prediction (e.g. there might be multiple relevant webpages and songs, routes, and chat messages). Minor: In the description of Figure 1, "in the left plot than the plot plot" -> "in the left plot than that in the right plot".

Questions

See weaknesses.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

3 good

Limitations

To me, the authors adequately addressed the limitations.

Summary

The authors consider a setting where there exist multiple provides of a machine learning model, and all together choose to increase their representation quality. The authors show, via both theory and experiment, that this improvement in data representation does not necessarily translate to improved social welfare. In particular, the authors shows that, due to their model of competition, when data representation qualities increases, there is less specialization in that the models are unlikely to differ significantly in providing strong performance for a certain group of users, and the equilibrium achieved may be worse off in terms of overall loss each user received. The authors present empirical results on CIFAR-10 using varying pretrained representations, as well as a generalization of their theory to situations where users break tied based on model-provider reputation.

Strengths

Overall, I think the paper studies a very interesting setting (multiple machine learning providers in competition), that is increasingly relevant these days (though I think any model of this should at least strive to capture the more realistic setting in that some players, like Google/OpenAI/Apple are clearly very dominant in their respective spaces, and at times might be seen more as monopolies, and not just from a market-reputation `tie-breaking' angle). The paper is also well-written and easy to follow, and has a good mix of theoretical and empirical results.

Weaknesses

My main concerns with the paper are lack of discussion on the assumptions needed, and confusion about the central takeaway of the work re: its relationship to the scaling law literature. W1 Scaling Laws / Takeaway: Even after a few reads, the framing of the work as a contrast to current works on scaling laws that only focus on one model feels a bit arbitrary, and it seems a bit unfair to say the focus on one-model is a "limitation" of work on scaling laws. I don't agree that the purpose of scaling law research is to make a statement on social welfare, and rather to precisely characterize phenomena that change as you increase model size / parameters. I think focusing on clean measures like loss as # parameters / depth / etc. are important to rigorously do, as for a lot of research in the field its purpose is to shed some empirical insights on the model and training processes themselves. Whereas this work opens up so many questions about the right way to model social welfare, user choice, model provider decisions, etc., I feel like it is a big departure from the intention of most research studying scaling trends. I think at minimum, the authors should try to soften claims where they draw this contrast, such as the first two sentences of the abstract, that almost seem to imply that the finding that accuracy improves with scale is only due to the one-model, when what the authors measure (social welfare) is anyways different from accuracy. I am mainly concerned that this paper runs the risk of misleading general readers who might just get the takeaway "we shouldn't try to increase model size". That might be an attention-grabbing claim, but I don't think the small-scale setting the authors study warrants such a claim - at best, it opens up the need for more research on analyzing competition in the "large-scale machine learning models" space. Therefore, I think the authors should also think carefully what the central takeaway of the paper really is. Even if the authors stick with the contrast with scaling trends as a broad framing, I'm still not sure its too strong/surprising of a finding. W2: Why is keeping the data constant a realistic assumption to make? None of the cited examples to motivate the paper (chatGPT vs. Claude, Spotify v. Pandora) have this property, particularly when you consider that their data is reliant on the users who use their serve, as well as methods like instruction-tuning and RLHF that are more proprietary and key to the mass deployment of these models. in the LM space, we can see this with the development of products like CharacterAI. In a competitive setting, if model providers could change the data they use, wouldn't that be an action they could take to provide unique services that would break the setting studied by the authors? It seems like the purpose of the assumption is only to be able to focus on the scaling trends, which goes back to W1. But at the least, this assumption should be discussed. W3: I think the reasons behind why increasing data quality representations leads to models that are "likely to agree with each other on most data points" should be clearly stated. Isn't one aspect important for social welfare whether it is a particular kind of user subgroup that is hurt by all models (related to the cited work on homogenization)? The data representation methods studied (e.g. varying the representation dimension, representation noise) are all pretty synthetic that don't result in semantically meaningful improvements (e.g. over a subgroup) in a way more realistic improvements (increase deep NNs model parameters) might. W4. While I think the generalization to asymmetry in the market where model-provider reputation matters is a great addition, this still feels somewhat limited (only executed through the form of weighting when breaking ties). People choose models based on a variety factors including ease of service, its performance over the generally distribution of inputs the user expects to query with, etc. -- why pick just reputation in a limited way to study? Again, I think these limitations should at least be stated. Minor - I think more experiment details should be included in the main paper, such as calculating the Bayes risk using gradient descent from pretrained representations for the CIFAR10 experiments. Since the paper's message seems intended to capture a setting that is only really arising due to large, pre-trained models that afford high-quality data representations, I think these experimental results are important to be clear to the community about, and I might suggest even including another experiment in the language domain. - The abstract could hint at the punchline "why" for the decrease in social welfare (the fact with low quality representations there is a more heteregenous set of choices) - typo in line 162

Questions

- What are the bars on Figure 3? Standard error or CIs? - Did the authors try more values of m? Is there a trend to observe over m? Currently, it is hard to make an overall statement just comparing m=8 and 12. - Line 263: Should this say initialize randomly? How important is this choice?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

There is essentially no limitations section in the paper, which I feel is important given the many assumptions used for overall toy setting that aren't reflected in the real world the authors motivate the work with. While the authors did address prior work limitations by generalizing to a setting where users are influenced by model provider reputation, even this is only accounted for in the tie-breaking scheme and I think the overall user choice setting is quite idealized. Furthermore, in practice users would likely have several different inputs, and pick the model that provides low loss on the first few inputs they try with - this suggests consider the distribution of inputs different users care about, which the authors' model doesn't study. I think the authors should state their assumptions very clearly, and explain why running additional experiments / analyzing settings without these assumptions are out of scope for this work.

Thanks for your detailed clarification!!

Dear Authors, Thanks for your clarifications and additional experimental results. The new cifar10 results look good and comprehensive. Your further explanations on social welfare have addressed my concerns. About the "inefficient translation to social welfare", I have read this again in the paper and corrected my understanding. Thanks for response overall. I have raised my score by 1 and support this paper to be accepted.

Thank you for your rebuttal

Thank you for putting together a very detailed rebuttal! The additional data on CIFAR10, and the generalized version of Proposition 1, do address some of my concerns about the setting, and your experiments with local representations seem to illustrate the need for additional work. I look forward to seeing the final version of this paper, and will maintain my score and support for it.

Response

Thank you for your response -- most of my questions were answered. Regarding specific sentences that relate this work to the scaling law literature -- I believe even the abstract and introduction frame this work as addressing a limitation of scaling laws (i.e. "However, these scaling laws typically take the perspective..." , "in reality providers often compete.."). One could instead frame the introduction entirely about studying multi-platform settings, and then motivate why it would be useful to study the effect of scale in this setting. I still believe the current framing is a bit misleading -- I think this paper holds more value to a researcher focusing on multi-platform markets than scaling laws (as this paper wouldn't fundamentally change the core research question people studying the latter focus on), so in the revised version I encourage the authors to increase clarity of their paper in this regard. It would be great if the authors provided in their rebuttal the limitations they intend to expand on, instead of just stating they will include a section!