Joint Training of Deep Ensembles Fails Due to Learner Collusion

Ensembles of machine learning models have been well established as a powerful method of improving performance over a single model. Traditionally, ensembling algorithms train their base learners independently or sequentially with the goal of optimizing their joint performance. In the case of deep ensembles of neural networks, we are provided with the opportunity to directly optimize the true objective: the joint performance of the ensemble as a whole. Surprisingly, however, directly minimizing the loss of the ensemble appears to rarely be applied in practice. Instead, most previous research trains individual models independently with ensembling performed post hoc. In this work, we show that this is for good reason - joint optimization of ensemble loss results in degenerate behavior. We approach this problem by decomposing the ensemble objective into the strength of the base learners and the diversity between them. We discover that joint optimization results in a phenomenon in which base learners collude to artificially inflate their apparent diversity. This pseudo-diversity fails to generalize beyond the training data, causing a larger generalization gap. We proceed to comprehensively demonstrate the practical implications of this effect on a range of standard machine learning tasks and architectures by smoothly interpolating between independent training and joint optimization.

Paper

References (66)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer a4xj7/10 · confidence 3/52023-06-22

Summary

The authors aim to answer why joint training of ensembles does not work as well as separately training the members of the ensembles, before ensembling them. This is a well-known empirical fact, but the authors attempt to give a theoretical understanding of it, which is novel to my knowledge. To reach their objective, they consider an expression for diversity (DIV) for probability averaging, which they motivate thru a (known) regression example. Then they generalize that expression to any twice differentiable loss function. The authors study DIV in more depth by (upper) bounding it, then using it as a tunable regularization tool to interpolate between joint training and separate training of ensembles. Then the authors hypothesize that jointly training the ensemble may artificially bump up DIV without yielding proper generalization. They say that this happens because of "learner collusion" a phenomenon where two learners are displaced artificially by a constant bias term without changing the overall performance of the ensemble. They devise empirical measures to test that hypothesis (by studying generalization gaps, diversity explosion, learner codependence etc.)

Strengths

S1: The paper is really well written. Thanks to the authors for building my reading experience so well. S2: The explanation of the phenomenon (to my knowledge) is novel, and gives me a new way of thinking about the well-known failure case of jointly training ensembles. S3: It is great that the authors have theory that is paralleled with experiments on real-world data.

Weaknesses

W1: Not sure I understand what happens when we vary $\beta$. I think the paper can improve in a more detailed discussion about a more thorough study of the effect of collusion in the intermediate values of $\beta$. See my Q2 in the questions below. W2: Typically, we expect to be able to devise better models after we have a better understanding of an underside phenomenon. The paper lacks a discussion about this. It's fine to leave it as future work, but it would be useful to discuss some directions a bit.

Questions

Q1: In Figure 2, why does Facebook seem to be different from the overall trend? Q2: If we get $\beta \approx 1$, then great, we have increased diversity and this gives us some hints as to why joint training might fail. However, as we vary $\beta$, I do not gain any intuition. Is the collusion effect a monotonic one? It seems so from Figure 6. If there is some $\beta > 0$ then do we always have that effect? Shouldn't intuition tell us that there should be a "sweet spot" for $\beta$? If not, then why would we be interested in training learners jointly to any extent? For example, could we do something more explicit to prevent the learner's collusion?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The authors have adequately addressed the limitations.

Reviewer pZUZ7/10 · confidence 3/52023-07-04

Summary

Ensembles are a simple but powerful way to improve model performance. Typically, ensembles are used to improve performance by training each model independently and then using them jointly. However, unlike previous ML methods that require ensemble members to be trained individually, Deep Ensemble, an ensemble of deep learning models, can also be trained jointly. Considering that joint performance is the true objective of Deep Ensemble, it seems more natural to train jointly rather than individually. However, in practice, joint training leads to poor performance and poor generalization. The authors show theoretically and empirically that this is due to ensemble diversity. From this, they propose an augmented objective $\mathcal{L}^\beta$ that allows for a linear interpolation between joint and independent training. Finally, they hypothesize that this is due to learner collusion, a phenomenon where each model has a bias when jointly training an ensemble, and show through experiments that this is indeed the case.

Strengths

- The notation is well organized and the formulas are easy to follow. - The problems covered in previous studies are well summarized and theoretically explained in Sections 2, 4, and 5. - The reason why joint training of Deep Ensemble fails is effectively explained through *learner collusion* and is well supported experimentally.

Weaknesses

- The authors do not propose a way to prevent learner collusion while doing joint training, so there is not much contribution in terms of practicality. The possibility of partial joint training by weighting between independent training and joint training using an augmented objective is presented as a future work, which is already analyzed in various ways in Webb et al. (2020). - (Minor) The overall formatting of the paper is less polished and a bit difficult to read. - Figures are scattered throughout the page. If possible, the figure should be located at the top of the page. - The text in the figure is too small to read the results. Increase the size of the text so that it is not too different from the size of the main text. - The lack of titles in the references makes it very difficult to identify the relevant papers. **Please fix this.** ----- (Webb et al., 2020) [To Ensemble or Not Ensemble: When does End-To-End Training Fail?](https://arxiv.org/abs/1902.04422)

Questions

- Do you think it would be beneficial to drop a different fraction of base learners each step during the joint training of the ensemble to avoid learner codependency? - Minor comments: - As mentioned above, the formatting of the reference is incorrect. At least the title of the paper should be visible. - NeurIPS style rules do not allow vertical lines in tables. I would recommend removing the background color as well. - Overall, I like the paper, but you spend too much space organizing and formalizing previous research; I think it would be better to add the experiments in the appendix to the main text. - In Figure 6, it looks better to add the ImageNet title to the two graphs on the right for consistency.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

I see no potential negative societal impact from this paper.

Reviewer m1Sy4/10 · confidence 4/52023-07-06

Summary

This paper explores the joint training of deep ensembles, wherein the ensemble error is directly optimized during training. The authors find that joint training leads to poor performance, which they posit it due to phenomenon they call "learner collusion", where base learners artificially inflate their diversity, at the expense of test performance. The authors provide a new decomposition of a broad class of loss functions in terms of the average error rate, the ensemble error rate, and a generalized notion of diversity. The authors then perform an experimental evaluation into the prevalence of learner collusion during joint training, using an augmented objective function to probe the effect of explicitly encouraging diversity during training.

Strengths

Overall, the paper is very well-written and easy to follow, and in my view the topic is very relevant to the community. To me, the most compelling experimental evidence for the learner collusion phenomenon are those presented under "learner codependence"/in Figure 4. From these results it is very clear that something distinct is happening in the joint training regime, and that the individual models are exploitating the joint loss. The definition of the augmented objective $\mathcal{L}^\beta$ seems like a very useful tool for probing these and other related phenomena.

Weaknesses

Unless I am misunderstanding, the conclusions drawn from Figure 2 do not seem to strongly support the claim that joint training significantly harms ensemble performance. Indeed, for the Facebook task, it seems the method actually helps, while for others it does not. This is in contrast to the CIFAR results in Table 1/Figure 6, where the degredation in performance is significant. Could the authors provide some clarification as the intended conclusion from these experiments? There are a number of other papers in the literature that study some form of the gap (in the authors' notation) $\bar{\text{ERR}} - \text{ERR}$, and express and/or bound it in terms of various forms of "diversity", depending on the given loss function. For example Ortega et al., 2022, Abe et al., 2022 and Masegosa et al., 2020 give explicit expressions for the average - ensemble error gap in terms of diversity metrics that are easily interpretable. It's unclear to me that the form in Theorem 4.5 adds much to this literature, and indeed the stated expression for diversity doesn't seem to be used in the remainder of the paper. Overall, while I think the paper is well-written and the topic is relevant, I think there is 1) insufficient use/novelty in the theoretical results and 2) somewhat incomplete conclusions to be drawn from the empirical results to recommend accepting at the the current stage. In particular, while it certainly appears like some type of "learner collusion" is happening, it seems very important to understand why this only appears at $\beta=1$, and exactly what impact this could have on the ensemble error. As a minor point, there seems to be an issue with the references -- they don't seem to contain any titles of the referred papers. **References:** Taiga Abe, E Kelly Buchanan, Geoff Pleiss, and John Patrick Cunningham. The best deep ensembles sacrifice predictive diversity, 2022. Luis A. Ortega, Rafael Cabañas, and Andres Masegosa. Diversity and generalization in neural network ensembles, 2022. Andres Masegosa, Stephan Lorenzen, Christian Igel, and Yevgeny Seldin. Second order PAC- Bayesian bounds for the weighted majority vote, 2020.

Questions

I find the distinct change in behavior of the various metrics at $\beta=1$ very interesting; it seems the behavior is relatively benign for $\beta < 1$, and yet there appears to be a distinct transition at $\beta = 1$. Do the authors have any hypotheses as to why this occurs?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

2 fair

Limitations

N/A

Reviewer XMTw5/10 · confidence 4/52023-07-07

Summary

This paper mainly studies the reason behind the failure of jointly training deep ensembles. It discovers that joint optimization results in a phenomenon in which base learners collude to artificially inflate their apparent diversity. Both theoretical and empirical evidence are provided further to verify the hypothesis.

Strengths

(1) This paper attempts to study a seemingly under-explored question, that is, why joint training of ensembles fails to generalize better than individual training and ensemble. This research topic is very important and helps us to understand the foundations of the deep ensemble. (2) Both theoretical proof and empirical proof are provided.

Weaknesses

(1) One big issue of this paper is that all references do not contain titles, which to be honest, I have never seen in top-tier conferences like NeurIPS. (2) I do not understand why joint training is easier to train but results in poor generalization. What does "easier to train" refer to? (3) What joint training approaches are used in the paper? (4) The empirical evaluation isn't very convincing to me. For instance, how many ensemble members are we using for "Learner codependence"? It is not surprising that the performance degrades to me if we drop ensemble members during testing, especially if our overall ensemble members is not sufficient.

Questions

Please see my above weaknesses.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

1 poor

Contribution

3 good

Limitations

N/a

Reviewer pZUZ2023-08-12

Thank you for the answers and additional experiments. I find this finding very interesting and think it will be helpful for future research. I have decided to raise my review score.

Senior Area Chairs2023-08-18

@XMTw: Please engage with rebuttal

The authors have posted their rebuttal to your review—does it affect your opinion? It'd be helpful if you can at least acknowledge having read the rebuttal, even if you don't find it convincing. Thanks!

Senior Area Chairs2023-08-18

@m1Sy: Please engage with rebuttal

The authors have posted their rebuttal to your review—does it affect your opinion? It'd be helpful if you can at least acknowledge having read the rebuttal, even if you don't find it convincing. Thanks!

Senior Area Chairs2023-08-18

@a4xj: Please engage with rebuttal

The authors have posted their rebuttal to your review—does it affect your opinion? It'd be helpful if you can at least acknowledge having read the rebuttal, even if you don't find it convincing. Thanks!

Reviewer a4xj2023-08-19

Great rebuttal.

Thank you for your clarifications. I read the discussions with the other reviewers too. I would like to see the paper accepted.

Reviewer XMTw2023-08-20

Thanks for the response

I thank the authors for the detailed response. It addressed some of my concerns. I believe the empirical results can contribute to the community. However, I believe the primary finding of this paper, i.e., jointly trained ensemble leads to learner Collusion is somehow trivial and straightforward. It is natural for me that the jointly trained ensemble leads to collusion without adding any regularization. Therefore, I would like to increase my score to 5, not 6.

Authorsrebuttal2023-08-20

We thank this reviewer for their response and are glad that they have determined that the paper should be accepted. We agree that their suggested extension to our empirical results reinforces the claims of our work. Let us briefly address the point that our findings are "*somehow trivial and straightforward*" as this was not raised in the original review (indeed, there this reviewer stated that "*this research topic is very important and helps us to understand the foundations of the deep ensemble*"). Our goal was to present this phenomenon as clearly and intuitively as possible and we are glad it was perceived as such. However, despite being a natural explanation in retrospect, the limitations of jointly training ensembles has been a **reoccurring issue** throughout the literature and, in the rare cases where they have been investigated, they have been **misdiagnosed**. We therefore suggest that a clear investigation of the issue, as provided by this work, is a worthwhile contribution to the literature which can (a) prevent the need to constantly rediscover this degeneracy, (b) amend the previous literature, and (c) guide future research into the optimization of deep ensembles. $~$ * **Reoccurring issue** - It is likely that this issue is regularly and unknowingly rediscovered by practitioners as joint training is a natural objective to consider. For example: (a) [1] reports training an ensemble sequentially due to “unstable” training dynamics when training simultaneously, (b) [2] introduces a regularization term to prevent their ensemble from “collapsing to degenerate solutions", (c) [3] states that training for both loss and diversity on the same data may "render the convergence point of the training process uncontrollable", and (d) [4] mentions that joint training "reduces the accuracy of the ensemble and can easily lead to training instabilities". We hope that, at a minimum, our work can act as a canonical reference for this issue such that future works might avoid this common pitfall. * **Misdiagnosed** - A small number of works have previously hypothesized about this problem. We addressed these works in Sec. 2 “joint training” with a deeper analysis in App E. The key takeaway is that the existing attempts to address this issue are insufficient for explaining the general phenomenon or, in some cases, are objectively incorrect (e.g. [5] reported successful joint training under SoftMax averaging which was discovered to be a coding bug that resulted in unintentionally implementing independent training instead). Although this degeneracy might appear trivial to this reviewer, we believe that there has been sufficient confusion in the literature to warrant our comprehensive analysis. $~$ [1] Pagliardini, Matteo, et al. "Agree to Disagree: Diversity through Disagreement for Better Transferability." The Eleventh International Conference on Learning Representations. 2022. [2] Lee, Yoonho, Huaxiu Yao, and Chelsea Finn. "Diversify and disambiguate: Out-of-distribution robustness via disagreement." The Eleventh International Conference on Learning Representations. 2022. [3] Pang, Tianyu, et al. "Improving adversarial robustness via promoting ensemble diversity." International Conference on Machine Learning. PMLR, 2019. [4] Mehrtens, Hendrik Alexander, Camila Gonzalez, and Anirban Mukhopadhyay. "Improving robustness and calibration in ensembles with diversity regularization." DAGM German Conference on Pattern Recognition. Cham: Springer International Publishing, 2022. [5] Dutt, Anuvabh, Denis Pellerin, and Georges Quénot. "Coupled ensembles of neural networks." Neurocomputing 396 (2020): 346-357.

Reviewer XMTw2023-08-20

Follow-up

I thank the authors for the quick response. I really appreciate their hard work. I believe that increasing the score to 5 does not necessarily means that I "have determined that the paper should be accepted". Moreover, I don't think it is not common to add new comments that were not raised in the original review. Aftering reading the new response, I am still not convinced that the findings in this paper are novel and non-trivial enough.

Authorsrebuttal2023-08-20

Likewise, we sincerely appreciate this reviewer's engagement. Regarding the new point raised, we only meant to clarify that since this was a new comment it was not something we had already addressed in our original rebuttal and would therefore respond in that comment. We suggest that despite the intuitive simplicity of base learners collectively biasing their prediction to artificially inflate diversity (i.e. learner collusion), this work may be of significant interest to others in the community where this effect has not been previously identified and, in fact, the question of *why* and *when* joint training fails has been misdiagnosed. For *why* it occurs, we note that the previous work of [6] who also noticed issues in joint training claimed that it was due to the fact that "gradients back-propagated into all ensemble members are identical". At the time of writing, we note that this work has been cited 270 times -- indicating that this issue is relevant to a reasonable subset of researchers. Then, in Appendix E, we showed that this explanation cannot explain the issue as it is not true (as the authors claim) in the case of probability averaging. Furthermore for *when* it occurs, we unify the joint objective decomposition to all twice differentiable loss functions rather than just cross-entropy loss which connects the issues in the regression setting with those of the classification setting for the first time. This is in addition to our other contributions which include (1) a tighter upper bound on diversity for guaranteed divergence for diversity, (2) a general augmented diversity objective with connections made to individual training with a gradient-adjusted target (i.e. GAT objective in App D), and (3) extensive novel empirical analysis (i.e. comparisons of joint training and independent training, analysis of increasing diversity, analysis of training dynamics, analysis of ensemble size, score-averaging vs probability averaging, and consideration of $\overline{\text{ERR}}$ as a metric instead of accuracy). We sincerely believe that providing the first accurate characterization of why joint training leads to this degeneracy in addition to the other aforementioned contributions will be valuable to a subset of the community who are developing novel theory and methods for ensembling deep neural networks. Your comment has helped us appreciate that these contributions could be made clearer in our text for future readers and, should this work be accepted, we would endeavor to improve our description of the contributions of our work early in the text.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC