Training shallow ReLU networks on noisy data using hinge loss: when do we overfit and is it benign?

We study benign overfitting in two-layer ReLU networks trained using gradient descent and hinge loss on noisy data for binary classification. In particular, we consider linearly separable data for which a relatively small proportion of labels are corrupted or flipped. We identify conditions on the margin of the clean data that give rise to three distinct training outcomes: benign overfitting, in which zero loss is achieved and with high probability test data is classified correctly; overfitting, in which zero loss is achieved but test data is misclassified with probability lower bounded by a constant; and non-overfitting, in which clean points, but not corrupt points, achieve zero loss and again with high probability test data is classified correctly. Our analysis provides a fine-grained description of the dynamics of neurons throughout training and reveals two distinct phases: in the first phase clean points achieve close to zero loss, in the second phase clean points oscillate on the boundary of zero loss while corrupt points either converge towards zero loss or are eventually zeroed by the network. We prove these results using a combinatorial approach that involves bounding the number of clean versus corrupt updates across these phases of training.

Paper

References (36)

Scroll for more · 24 remaining

Similar papers

Peer review

Reviewer NX3e7/10 · confidence 3/52023-06-17

Summary

The paper studies overfitting when training shallow ReLU networks with the hinge loss using noisy data (i.e., with label flipping noise). They consider a setting where the data points consist of a direction $\mathbf{v}$ that determines the (clean) labels, and noise from a sphere orthogonal to $\mathbf{v}$. This distribution is similar to the two-clusters setting that has been studied in previous works [Frei et al. 2022, Xu and Gu 2023, Chatterji and Long 2021], except that the assumptions on the noise in the current work is stronger. The authors identify conditions on the signal strength that result in benign overfitting, harmful overfitting, and “non overfitting” (where only the clean training examples are classified correctly, and the test error on clean data is small). The results follow by analyzing the trajectory of gradient descent.

Strengths

Understanding the conditions where benign overfitting occurs is an intriguing question that has be studied extensively in recent years. While the primary motivation for studying benign overfitting is to better understand interpolation learning in neural networks, most of the known results consider linear models (or kernel regression). The current paper helps understanding benign/harmful overfitting in shallow fully-connected ReLU networks trained with the hinge loss. Given our limited understanding of benign overfitting in nonlinear networks, I believe that the contribution of the current paper is significant. I did not verify the correctness of the proofs, which are in the appendix.

Weaknesses

- Assumption 1 in the main text is different from Assumption 1 in the appendix. Specifically, condition 5 of Assumption 1 in the appendix includes a parameter $\rho$ that does not appear in the assumption in the main text. I think that the authors should discuss the relationship between these two versions of Assumption 1. - Related to the above: Note that in lines 221-222 in the main text, the authors claim that the condition on the correlations between $\mathbf{n}_i,\mathbf{n}_l$ follows from Assumption 1 by letting $\rho \leq \frac{\gamma}{C}$. This is true if we use Assumption 1 from the appendix, but it does not follow from Assumption 1 in the main text. Please explain how/whether the proof holds using the assumption from the main text. This issue appears also in the other proof sketches. - The authors claim that there are only two papers about benign overfitting in nonlinear fully-connected neural networks: Frei et al. 2022, and Xu \& Gu 2023. The authors provide a nice comparison between the current results and these prior results. It is worth noting that there is an additional prior work on benign overfitting in nonlinear fully-connected networks: “Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization” by Spencer Frei, Gal Vardi, Peter Bartlett and Nati Srebro. This paper appeared in arxiv only 2.5 months before the NeurIPS deadline, so it is reasonable that the authors were not aware of it when writing the current paper, but I think that a comparison to this paper would be helpful. - The writing is quite sloppy. Below I specify some typos and minor comments: - Line 130, in the definition of $\mathcal{F}^{(t)}$: should be $>0$ instead of $<1$. - Line 132: “j=th” - Line 155 — “constant that depends on $n,m,k,\gamma,d$”: if we think about all of these parameters as constants, then what isn’t a constant? - Line 156: (5) should be (4) - Line 227: is the notation $[m]_p$ defined? Also, you probably meant $[2m]_p$. - Line 385 in the appendix: “delta” - The word “overfitting” sometimes means “interpolating also the noisy examples” and sometimes means “harmful overfitting”. I would write explicitly “harmful overfitting” when talking about the latter.

Questions

See the “weaknesses” section.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The authors discussed the limitations.

Reviewer 8QyX7/10 · confidence 3/52023-06-29

Summary

The paper studies training a noisy dataset with label noise using GD on a 2-layer network, without biases, and where the second layer is fixed. The main results are, that under the data distribution defined in the paper (namely, linearly separable with orthogonal noise), different types of overfitting (or no fitting) occur when changing the margin.

Strengths

- I think the main three results are very interesting, and shed light on the role of the margin for determining the types of overfitting which occurs. - AFAIK the main three results are novel, as they study benign overfitting for neural networks under hinge loss, whereas previous works either studied classification losses (e.g. log loss) or kernel methods. - The proof techniques also seem interesting and employ combinatorial arguments about how the iterations change the activation pattern of the network. - The proof intuitions are very helpful and elaborative.

Weaknesses

I think overall this is a nice paper that should be accepted, however, there are some weaknesses/questions which I would be happy to see the author’s response about. - I think the main weakness of the paper is the limited and very specific setting to which it is applied. This is acknowledged by the authors, however, I still think it should be further addressed. - The network doesn’t contain biases and only the weights of the first layer are trained. Regarding the bias terms, I am not sure how adding them would change the results and would be happy to see the author’s comment about this. - Regarding the data distribution, would it be possible to achieve a similar result where the noise n_i are sampled from a Gaussian distribution? If so, I think this is a much more standard setting to study (signal plus Gaussian noise). I guess also here, the noise would be almost orthogonal to the signal v. If something breaks here (rather than just make the computations more tedious), then I think the result is very brittle since it requires the noise to be exactly orthogonal to the signal. - I think the paper “Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting” by Mallinar et al. should also be cited, as it gives a taxonomy of the different possible types of overfitting (which is studied also here). - It would be nice to see an upper bound for Theorem 3.6. Currently when the margin is very small then the overfitting is either catastrophic (i.e random guess) or tempered (something between benign and catastrophic). Can the authors provide such an upper bound, or at least remark on what they think it should be? I think that it could depend on k, which depicts the noise level in the labels. - Small note: Assumption 1, should it be k <= n/C ?

Questions

- How the results would change if the network had bias terms? or the second layer would also be trained? - Will the result still hold if the noise n_i are drawn from a Gaussian distribution? (or other symmetric distribution such as uniform over a sphere) - What should the upper bound on the loss in Theorem 3.6 be?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

I think the authors adequately addressed the limitations of their work.

Reviewer ZHgu8/10 · confidence 4/52023-07-07

Summary

In this work a simple model of linear classification is analyzed to study the phenomenon of benign overfitting and generalization of neural networks with noisy data. A theoretical analysis of the trajectory of GD shows that it can be in three regimes (1) benign overfitting (2) non-benign overfitting and (3) no overfitting. The analysis relies on a novel proof of the nonlinear dynamics which shows a bound on the number of non-trivial updates of GD until convergence.

Strengths

1. Strong theoretical results in a very challenging setup of learning with nonlinear NNs. 2. Novel proof. Most works consider the logistic loss without a thorough analysis of the trajectory. Here the hinge loss is considered and a detailed analysis of the trajectory is provided. The key result is a bound on the number of non-trivial updates which is independent of the total number of iterations. This is highly non-trivial: a “worst-case” dynamics can result in oscillating behavior or non-convergence. However, a clever argument is presented which derives inequalities of clean and corrupt updates that together imply convergence. Hopefully such reasoning can be helpful in more complex settings. 3. Full understanding of the dynamics. While the proof is lengthy, it provides a unique understanding of NN dynamics, their inner workings and the impact on generalization. 4. The paper is mostly clearly written.

Weaknesses

1. The theoretical result holds for a bounded number of samples (i.e., $n$ cannot go to infinity). 2. Simple experiments which show the theoretical predictions could strengthen the paper.

Questions

In Eq. (1) why is $\beta(i)$ needed in the definition of $\boldsymbol{x}_i$. Isn’t the distribution of $\beta(i)\boldsymbol{n}_i$ the same as the distribution of $\boldsymbol{n}_i$?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

4 excellent

Limitations

Limitations are discussed.

Reviewer dAPs6/10 · confidence 4/52023-07-15

Summary

This paper studies the benign overfitting (or not) of two-layer ReLU neural networks under the hinge loss. Under a new noisy (almost orthogonal) data setting without separable data setting, - Theorem 3.1 shows that benign overfitting exists occurs the signal-level parameter $\gamma$ is small in $\Theta(1/n)$; - catastrophic overfitting (the generalization error is bounded below by a constant) occurs when $\gamma$ is small enough at the order of $n^{-2/3}$. - Instead, if $\gamma$ is large, no overfitting occurs, i.e., training error cannot be zero but generalizes well.

Strengths

- provide solid analysis of generalization guarantees of two-layer ReLU neural networks under a new data setting - The results on benign overfitting, catastrophic overfitting, and no overfitting heavily depend on the choice of the signal-level parameter $\gamma$

Weaknesses

- The motivation of this paper is to fix the benign overfitting problem under the hinge loss. Since the analysis of previous work on the logistic loss is invalid, the authors focus on a new data generation setting and new proof framework. However, the motivation and significance of this research appear somewhat limited, particularly when considering the hinge loss. It would be beneficial to emphasize the differentiation between the results obtained with the logistic loss and the hinge loss, highlighting the theoretical advantages of employing the hinge loss. Additionally, exploring potential implicit biases related to benign overfitting and engaging in further discussions, particularly in comparison to the work conducted by Shamir (COLT2022) on the squared hinge loss, would strengthen the paper's arguments. - The problem setting and data assumption are a bit unrealistic (also as suggested by the authors). It’s ok for me but there are some assumptions missing, e.g., in the benign overfitting part, $n - k > 2*10^6$ in Lemma B.4 is needed. I think this should be clearly mentioned in the main text. Minior: - The early phase in line 224 is undefined.

Questions

- I understand the data setting with $d \geq n^2$ for the orthogonal condition, which is widely used in the benign overfitting literature for two-layer neural networks. But currently there exists some work that has fixed this issue: Frei et al. Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization. COLT2023. So I kindly ask the authors to think about whether their assumption can be relaxed or not. - Since I’m an emergency reviewer, I don’t have enough time to check the proof. I high level read the proof framework, and find that line 236-237 is they key proof idea:”every training point activates one neuron after the last training update”. Could you please detail this? The generalization performance (whether benign overfitting occurs) only depends on the signal-level parameter $\gamma$ in the data generation process. However, intuitively, the difference on generalization guarantees is based on the changes of model complexity. So how to understand this issue? - Continuing the above question, we can find that, if $\gamma$ is small (enough) at the $O(1/n)$ order, we have the benign overfitting. However, it means that the signal level $v$ is quite small and the noise level $n_i$ is quite large. That means, the model can fit such large random noise? If $\gamma$ is large in Theorem 3.8, the model cannot fix the noise to achieve zero training loss if the noise level decreases. When checking these two results, I find that there exists some contradictions here? Maybe I’m missing something. - The width $m$ is not required to be over-parameterization? More discussions are needed.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

N/A

Reviewer dAPs2023-08-11

I appreicate the authors' feedback. It addressed most of my concern. About the (nearly) orthogonal condition, I fully understand the authors' concern, and the following paper (it posted after NeurIPS ddl) would be benifical to this question though the problem setting is different and one exrtra assumption is also needed. [1] Chistikov, D., Englert, M., & Lazic, R. (2023). Learning a Neuron by a Shallow ReLU Network: Dynamics and Implicit Bias for Correlated Inputs. arXiv preprint arXiv:2306.06479. Besides, I suggest the authors to think about how to improve the significance/impact of this work in the introduction, not just Table 1. For example, why we focus on the hinge loss? Researchers outside benign overfitting will think the extension from exponential-tail type loss to hinge loss is neither significant nor novel. Though this is not true, we cannot ensure the readers are experts in benign overfitting.

Authorsrebuttal2023-08-20

We thank the reviewer for highlighting this work which as highlighted was released after the NeurIPS deadline. Although the setup is quite different, in particular this paper studies the implicit bias of gradient flow with MSE loss for a clean regression task instead of noisy classification with hinge loss, we agree with the reviewer that the approach for handling correlated data is of great interest. We will certainly investigate such techniques in future work. We will happily improve our discussion of the motivation of this work in future versions. We respectfully disagree with the statement *"Researchers outside benign overfitting will think the extension from exponential-tail type loss to hinge loss is neither significant nor novel”*. Indeed, until now, benign and tempered overfitting have only been theoretically analyzed for exponentially tailed loss, despite the fact that the phenomenon is more universal than this. The training dynamics we observe and thus the proof techniques we use are quite different to prior works. Our work compliments our nascent understanding of benign overfitting and provides a path for assessing the impact of using one loss function versus another.

Reviewer NX3e2023-08-12

Thanks for the response

Reviewer 8QyX2023-08-13

Re:Rebuttal

Thank you for the response.

Program Chairsdecision2023-09-21

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC