Summary
This paper examines the connection between the notions of consensus and equilibrium in a multi-agent system where multiple interacting sub-populations coexist, and it aims at answering below two central research questions in network (population) games scenario:
[1] Are there natural multi-agent learning models that can achieve the best of both worlds—reaching consensus as well as convergence to equilibrium—in diverse settings?
[2] How does the consensus formation process affect equilibrium selection in multi-agent learning?
The authors argue that consensus and equilibrium, the fundamental notions of these two fields, can be both understood as stability concepts in a multi-agent system where there co-exist multiple interacting sub-populations. In particular, consensus can be seen as an intricate component of intra-population stability, whereas equilibrium can be seen as encoding inter-population stability. The authors show that SFP (smooth fictitious play) algorithm can achieve consensus as well as convergence to equilibrium in a wide range of network population games (and unlike previous literature, here a coordinative reward structure is not a prerequisite for achieving
consensus). They also empirically shows that consensus formation process plays a crucial role in the seminal thorny problem of equilibrium's election in multi-agent learning (e.g., starts with the same initial mean belief, a large variance of initial beliefs results in a more desired equilibrium). Experiments were conducted for the scenario of Equilibrium Selection in Two-Population Stag Hunt Games.
Strengths
1. The paper is organized and presented well and clearly, I found the manuscript is reader friendly.
2. This work extend existing literature in multiple directions and presented a couple of nontrivial novel theoretical results, which is quite beneficial to the research community. E.g., SFP in network (population) games has not been previously explored until this work, unifying consensus formation and learning in multi-agent games, etc., proving consensus without assuming a coordinative reward structure, etc..
3. There are quite some helpful/valuable elaborations/explanations about comparing this work with relevant literature and discussing the differences and advantages.
Weaknesses
1. I'd like to see some discussions about the future research directions, and how this work could inspire/benefit other future research.
2. Regarding the figure 1 about impacts of variances of initial beliefs, it seems to be just plot of one example setting, and the empirical conclusion is not very convincing to me and I'd like to see some more clarification/elaboration/justification. For example, consider such a scenario where the starting mean belief is already the optimal value of a final desired steady state/equilibrium, it seems that if the staring variance is 0 (extreme case of small variance), then convergence to optimal results is already achieved, which is obviously better than a larger initial variance with the same starting mean belief, which is a counter example of the empirical conclusion of the paper saying that a larger initial variance (given same starting mean belief) is preferred. I would like to see either some rigorous mathematical proofs about such conclusion, or very comprehensive empirical studies on this before drawing the conclusions on this.
3. In page 4, it was mentioned that "This paper formally shows that the probability distribution over initial conditions can eventually degenerate to a point mass, and leveraging on this, presents a novel technique for proving the convergence of learning dynamics." It would be good to see some more elaborations on how this "novel technique" could be used for proving the convergence of learning dynamics in other relevant problems.
Questions
Please see my questions/comments/suggestions in above section when talking about weakness.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.