Learning to Mitigate Externalities: the Coase Theorem with Hindsight Rationality

Summary

This work considers a two player game with externalities imposed from one agent (“upstream”) to the other (“downstream”) agent. Equilibrium in the absence of taxation or payments between players would result in heavy efficiency loss due to the externalities. The Coase theorem tells us that in these situations we should expect to see payments from the downstream agent to the upstream agent in order to restore the socially optimal outcome. This work formulates this scenario as a multi-armed bandit problem, in which the payment term forms a part of the players payoffs and options. Namely, the downstream agent can put a conditional payment on the arms played by the upstream bandit. In this situation, the paper asks whether or not normal bandit learning algorithms can learn the optimal outcome. The paper puts forwards an assumption that must be satisfied on the upstream agents learning behavior, which is satisfied for many common bandit learning algorithms. A translation is given for the downstream agent to go from the raw bandit problem to a transformed bandit problem. With these, sublinear regret is realized relative to the socially optimal outcome.

Strengths

The work takes a well-known economic theory and asks whether a given type of ML can recover the expected phenomenon without knowing the specific externality a prior. Rather than showing an exact algorithm for recovering it, mappings are done so that any suitable bandit learning algorithm can successfully learn the results. This is a nice addition over prior work, and helps in suggesting that this is something with broad learnability. In the process, it is interesting to see the manner in which the externality needs to make it into the payoffs and how that space needs to be structured. Overall the paper is well written and has good fit at NeurIPS. Relative to prior work, it is able to show that as long as the upstream agent is using a suitable approach, any bandit learning algorithm will work. This is a much stronger guarantee for the downstream agent as they are less dependent on exactly what the upstream agent is doing.

Weaknesses

The paper and situation is quite interesting, but is one specific (classic) example scenario so applicability is good because it is a classic example, but still limited. A natural followin question is how broad do we expect these situations to go - can we recover general externalities in multi agent situations?

Questions

How broad do you expect the BELGIC routine to be applicable? In a general $M$ agent situation with less obvious structure on the externalities would you expect optimal transfers to still be learned efficiently?

Rating

7

Confidence

2

Soundness

4

Presentation

4

Contribution

3

Limitations

Yes the authors have adequately addressed limitations of their work.

Summary

The paper explores the field of mitigating externalities in economic interactions by applying the Coase Theorem with hindsight rationality. The Coase Theorem, a fundamental concept in economics, suggests that in the presence of externalities, property rights, and bargaining strategies can be utilized to achieve an optimal outcome for all parties involved. However, traditional applications of the Coase Theorem assume that players have perfect knowledge of the game, which may not hold true in real-world scenarios. In this paper, the authors introduce the concept of hindsight rationality, where players learn from past interactions to improve their decision-making processes. By incorporating hindsight rationality into the Coase Theorem framework, the paper aims to provide a mechanism through which players can adapt their strategies over time to maximize social welfare, even in the presence of uncertainty. The theoretical foundation of the paper is built upon a series of theorems supported by assumptions and proofs. Overall, the paper offers a novel perspective on addressing externalities in economic settings by integrating hindsight rationality into the Coase Theorem framework. Through a rigorous theoretical analysis, ethical considerations, and a discussion of limitations, the paper contributes to the ongoing discourse on optimizing social welfare in the presence of externalities.

Strengths

S1. One of the main strengths of the paper is its solid theoretical foundation. The authors create a clear framework based on well-known economic principles, like the Coase Theorem, and expand it by adding the idea of hindsight rationality. The thorough theoretical analysis, backed by detailed proofs and assumptions, strengthens the credibility and robustness of the proposed approach. S2. By introducing the concept of hindsight rationality within the context of mitigating externalities, the paper offers a novel perspective on addressing economic inefficiencies.

Weaknesses

W1. One significant weakness of the paper is the lack of empirical validation or experimental results to back up the proposed theoretical framework. Although the theoretical analysis is good, empirical evidence would improve the practical applicability and real-world relevance of the research findings. W2. It would have been insightful to see a comprehensive discussion on the potential challenges and complexities associated with implementing the proposed approach in practical economic settings. Addressing implementation hurdles could provide valuable insights for policymakers and practitioners. W3. The setting (extension of Coase theorem in a two-player bandit setting where the actions of one of the players affect the other player) considered in the paper is narrow and limited. The results are also exemplified for a simple setting of two firms, an upward and a downward. Do the observations apply to potentially broader settings? W4. The mathematical notations provided in the paper sometimes become too difficult to follow and correlate with the text.

Questions

Please clarify the points in the Weaknesses section.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes, the limitations of the proposed method are discussed in the paper.

Summary

The paper presents a two-player sequential game with bandit feedback where one player's actions create externalities that affect the other player's outcomes. In this model, the utility of only one player (the downstream player) is impacted by the collective actions of both the upstream and downstream players. They consider two scenarios: one where players act independently to maximize individual utility, and another where players can interact via transfers. In the first scenario (without transfers), the authors show that individual utility maximization can lead to suboptimal social welfare in some instances. In the second scenario, where the downstream player can choose a transfer, they demonstrate that social welfare can reach its optimum. They propose an algorithm for the downstream player that achieves this outcome, provided the upstream player follows any no-regret policy. This effectively presents an online version of the Coase theorem, establishing that even with imperfect information, the two-player sequential game converges to the global optimum outcome when transfers are allowed.

Strengths

1. The algorithm does not assume a specific policy for the upstream player and achieves sublinear regret for any no-regret upstream policy (with regret scaling as $T^\kappa$). 2. Lemma 1 provides a clean problem formulation, showing that bounds on regret in individual utility (with transfers) lead to a bound on the social welfare regret. 3. For the specific instance where the upstream player follows the Upper Confidence Bound (UCB) policy (or any other algorithm with $\sqrt{T}$ regret) the proposed algorithm achieves regret scaling as $T^\frac{3}{4}$.

Weaknesses

Please see the questions section.

Questions

1. It is not clear whether the regret incurred by the downstream player is optimal. Even if the upstream player is assumed to follow UCB, it's not evident whether a regret scaling as $\sqrt{T}$ policy is achievable for the downstream player as well. 2. It is stated (on lines 111-112) that the assumption that the rewards are bounded is "without loss of generality" (WLOG). Can you please explain why? Would this work even in the case of other standard reward distributions in bandit literature, such as sub-Gaussian rewards? 3. In the current setup, it is assumed that transfers occur only on a single action, with the transfer function taking a value of 0 for all but one action. Is there a particular reason for assuming such a transfer function? Can we work with more general transfer functions where the downstream player can choose to make transfers on multiple actions? How would this affect the regret of the downstream player?

Rating

7

Confidence

2

Soundness

3

Presentation

4

Contribution

3

Limitations

This paper does not have any negative social impact.