Summary
The paper provides a geometric perspective on weak-to-strong generalization [[Burns et al., 23](https://arxiv.org/abs/2312.09390)]. Specifically, the authors show that if the set of strong model-representable functions the following holds:
$$MSE(\phi^*, \phi^{ws}) \le MSE(\phi^*, \phi^w) - MSE(\phi^w, \phi^{ws}),$$
where $\phi^*$ is the ground truth labeling function, $phi^{w}$ is the the weak model labeling function and $\phi^{ws}$ is the weak-to-strong model labeling function. By labeling function I mean a mapping from inputs $x$ to real-valued labels $y$.
The authors use this result to argue that the gain in MSE loss in weak-to-strong generalization is at least equal to the _misfit_ term $MSE(\phi^w, \phi^{ws})$. There are some experiments verifying this result in practice.
Strengths
1. The core result is in fact very simple and intuitive: it's a geometric argument about a projection on a convex set.
2. To the best of my knowledge, this main result was not reported previously in the context of weak-to-strong generalization. It provides a toy model where it's trivial to see that weak-to-strong generalization will hold.
3. The authors show some simple experiments where their theory is applicable: synthetic regression tasks and molecular property prediction.
Weaknesses
W1. My main concern with the paper is that in my opinion the presentation is very confusing. Specifically, the core result is very simple, but it is presented in a way that took me a while to understand.
My first issue is that the authors use notation $d_{\mathcal{P}}$ to denote the mean-squared distance, i.e. $\mathbb{E}(f(x) - g(x))^2$ and refer to it as distance (line 133). If $d_{\mathcal{P}}$ was in fact a distance, then Eq. (2) is the inverse of the triangular inequality. This confused me for a while. Moreover, Fig 1 shows a triangle with sides labeled as $A$, $B$, $C$, and states $C \ge A + B$. The resolution is that $d_{\mathcal{P}}$ is a square of a distance, not a distance.
Then, Theorem 1 is just a combination of (1) law of cosines and (2) and the fact that the angle between the vectors connecting a point to its projection on a convex set, and a vector connecting its projection to any point in the convex set is $\ge 90$ degrees.
The proof and the statement of the theorem are in my opinion significantly complicated by the explicit use of representations $h$ throughout. I don't understand what value they add: throughout the paper, the authors always use the same compositions e.g. $f^s \circ h^s$. Denoting the entire mapping with one letter would simplify the presentation. Currently, the authors also state that they are "employing a representation-theoretic perspective" (line 308), but I don't think there is anything added by the parameterization of the model as a function on top of some representations. Possibly the only thing is Claim 3, but it doesn't affect other parts of the paper much.
W2. The result of the paper holds specifically for the MSE loss. Indeed, if $d_{\mathcal{P}}$ is a distance metric, then Eq.2 is the opposite of the triangle inequality and doesn't hold. Moreover, for the qualitative conclusion
> Thus, the 148 inequality directly says that the weakly-supervised strong model improves over the weak model by (at least) an amount equal to the misfit.
clearly doesn't hold for the classification settings considered by [Burns et al., 23]: it is possible for the strong model to differ from the weak model on datapoints where the weak model is correct, which would not make it more accurate with respect to the ground truth labels.
W3. In fact, [Burns et al., 23] provide a relevant discussion of _simulation difficulty_ in Appendix E (especially E.1). They also note that in order for the strong model to improve upon the weak supervisor, the strong model should be unable to simulate the weak labels well. A related result is also reported in Fig 8 of that paper. I think it would be good for the authors to comment on how their results connect to these results.
W4. The practical proposition of using the weak model with the highest misfit will not work without further assumptions. This idea is again quite related to the result in Appendix E.1 of [Burns et al., 23]: if the errors of the weak model are not simulatable by the strong model, but the signal is easy to simulate, then we will get very good weak-to-strong generalization. But in general, high weak-to-strong disagreement will not always imply good weak-to-strong generalization.
Even under the assumptions of convexity etc of theorem 1, I believe it is not true that for a given $d_{\mathcal{P}}(\phi^*, \phi^w)$ higher $d_{\mathcal{P}}(\phi^w, \phi^{ws})$ always lead to lower $d_{\mathcal{P}}(\phi^*, \phi^{ws})$. Is that correct?