Towards Data-Algorithm Dependent Generalization: a Case Study on Overparameterized Linear Regression

One of the major open problems in machine learning is to characterize generalization in the overparameterized regime, where most traditional generalization bounds become inconsistent even for overparameterized linear regression. In many scenarios, this failure can be attributed to obscuring the crucial interplay between the training algorithm and the underlying data distribution. This paper demonstrate that the generalization behavior of overparameterized model should be analyzed in a both data-relevant and algorithm-relevant manner. To make a formal characterization, We introduce a notion called data-algorithm compatibility, which considers the generalization behavior of the entire data-dependent training trajectory, instead of traditional last-iterate analysis. We validate our claim by studying the setting of solving overparameterized linear regression with gradient descent. Specifically, we perform a data-dependent trajectory analysis and derive a sufficient condition for compatibility in such a setting. Our theoretical results demonstrate that if we take early stopping iterates into consideration, generalization can hold with significantly weaker restrictions on the problem instance than the previous last-iterate analysis.

Paper

References (77)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer YpbP6/10 · confidence 2/52023-07-04

Summary

This paper suggests a notion of compatibility, which is an algorithm-dependent and data-dependent term to bound the excess risk of overparameterized linear regression models. Using this notion, the authors come up with three regimes on the covariance matrix, and characterize the compatible region (that is the region of steps where we can train the model with GD and still converge in terms of excess risk). The authors show the connection between compatibility and benign overfitting and also compare to stability bound. Several examples are provided to demonstrate how the compatibility region depends on the covariance structure.

Strengths

I find the paper to be well-written and easy to follow. The overall topic of generalization of overparameterized models (even in the linear case) is important. The authors make a considerable effort to introduce existing bounds.

Weaknesses

1. I find the study of overparameterized linear model *together with* early stopping less motivating. In particular, early stopping for linear regression is equivalent to $\ell_2$ regularization. In this case, we can already apply uniform convergence bound for generalization error. In this case, is the motivation mainly to get a better bound? If so, I would suggest the authors move the comparison to stability bound into the main text. This is in contrast to the overfitting case since if we are overfitting, there is no uniform convergence bound at all. 2. One relevant work is missing, [1] (and its related citations) discussed cases when we have different kinds of overfitting. In particular, their theorem 3.1 (b) corresponds to your example 4.1. I would love to see how your results are related to theirs. [1] Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting

Questions

See weakness.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

N/A

Reviewer dysH6/10 · confidence 4/52023-07-06

Summary

This paper suggests a new framework for evaluating the generalization error called compatibility. Compatibility takes into account both the data and algorithm and says the two are compatible if there is an early stopping region, as $n \to \infty$ that has zero excess risk. The paper then considers the linear regression framework and provides spectral conditions under which full batch constant step size gradient descent is compatible.

Strengths

The main strength of the paper is the generality of the data distributions considered. Specifically, most prior theory work assumes we have isotropic data or some fixed transform of isotropic data. However, this paper considers general anisotropic data. In this setup, they provide mild conditions under which we see that early stopping is beneficial for generalization. The paper also provides clear evidence of the benefits of early stopping. Showing that the min norm solution is not necessarily the optimal solution for generalization. Further, the paper shows that this is true even in the large data limit when the min norm solution is benign.

Weaknesses

The are two main weaknesses (one major one minor) 1) The paper is missing the comparison to Madhu S. Advani, Andrew M. Saxe, and Haim Sompolinsky. High-dimensional Dynamics of Generalization Error in Neural Networks. Neural Networks, 2020 While the data assumptions in the above paper are more restrictive, that is isotropic data. The paper exactly characterizes the risk curve as a function of time and determines the optimal stopping time. Hence a proper discussion of this is crucial. 3) While the paper is well written, the paper sweeps technical details under the rug that would be important to know. For example, the constants $c_0$ and $c_1$ are opaque and play an important role. See my questions for more instances.

Questions

In Assumption 1 subpoint 2, should it $\lambda_p > 0$? Instead of $\lambda_1 > 0$. Are the Cs on lines 169 and 173 supposed to be the same? Why is $\boldsymbol{Y}$ called the noise vector? In Equations (4), (5), how do we know such an $\ell$ exists? What is $c$ in the statement of Theorem 5.1 in $k_0 \le n/c$?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

N/A

Reviewer ytRs7/10 · confidence 2/52023-07-06

Summary

This paper considers the benign overfitting phenomenon in overparameterized linear regression problems under early stopping type gradient descent. Specifically, this paper gives a time variant bound for the excess risk during the gradient descent training. By comparing the optimal time variant bound with the previous bounds for the minimum-norm interpolant, this paper finds some data distribution examples that the optimal early stopping satisfies benign overfitting (compatibility) but the minimum-norm interplant does not. Among all the examples, some of which outperform the previous early stopping type bounds in overparameterized linear regression problems.

Strengths

* The organization and presentation of this paper is smooth and clear, and it provides a better understanding on the benign overfitting phenomenon for early stopping type gradient descent. * The theory in this paper is solid. Although it mainly follows from the previous two works *Bartlett et al.* and *Zou et al.*, the proposed notion 'effective dimensions' for the feature covariance matrix in this paper provides an alternative way to analyze the variance term in the Bias–variance decomposition.

Weaknesses

There is no major weakness for me. --- A minior issue: For the examples and the numerical experiments in this paper, it would be better if the authors could provide the patterns to show the interplay between the bias term $B(\theta_t)$ and the variance term $V(\theta_t)$ during training. Especially when comparing with the bounds in previous works, it would be more intuitive to see why the compatibility happens but the benign overfitting does not, if the comparisons of the evolutions of $B(\theta_t)$ and $V(\theta_t)$ are shown separately.

Questions

* In table 2, for the distributions that $\lambda_i = \frac{i}{i \log^2(i+1)}$ and $\lambda_i = \frac{i}{i \log^3(i+1)}$, both cases are benign overfitting based on *Bartlett et al*, but the min-norm excess risk seems to be very large though, is this caused by the bias term or the variance term? * I am curious what the generalization bound for early stopping will be like if following the notations in *Theorem 4. [Bartlett et al.]* in terms of $r_0(\Sigma), k^*, R_{k^*}(\Sigma)$? I am trying to understand the difference between methods using the proposed effective dimension in this paper and using the effective ranks in *Bartlett et al.*, because understanding the differences may help to motivates other more refined controls for the generalization analysis.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

NA

Reviewer 5qZn7/10 · confidence 1/52023-07-26

Summary

The authors present a new mathematical framework for characterizing generalization of over-parameterized machine learning models. The new framework provides improved bounds from prior work, and improves on prior work by combining both data and algorithm information. To summarize their contributions: The paper formalizes a notion of compatibility, as a minimal condition for generalization based on the interaction between data and algorithm. They derive sufficient conditions for compatibility for over-parameterized linear regression with gradient descent, and then derive time-variant generalization bounds for the over-parameterized linear regression and show empirical results that verify the motivation of compatibility and demonstrate benefits of early stopping. Caveat: This paper falls outside my area of expertise to the level that in my available time I cannot vouch for the accuracy of the theorems provided in the paper.

Strengths

Theoretical proofs are given, along with bounds comparisons, and empirical results. The paper seems well-written and rigorous in its assessment. The problem is well motivated and extremely well-referenced in its citations and descriptions of the current work in this area. The ability and degree to which over-parameterized models can generalize is one of the main mysteries of deep learning, and where traditional theory in generalization breaks down, so the paper is extremely relevant.

Weaknesses

This paper requires substantial knowledge of generalization theory to understand and is not accessible to a broader audience. For example, the introduction makes the assumption that terms like excess risk are understood by the reader, before they are later defined. While the theory is claimed to be general, thorough treatment is only given for linear regression, and derivations of sufficiency conditions and bounds for other problems are left as future work. While deep learning was given as a primary motivation to understand generalization for over-parameterized models, application to these models was not addressed in the paper. The experimental and conclusions sections were very minimal. I understand that the theory was the primary focus of the paper, but would have liked to see more practical demonstration and discussion of implications for the theoretical results that can be drawn from your investigations.

Questions

Are there any roadblocks that would make application of this difficult for other models, such as non-linear regression problems including deep neural networks? Are there larger or broader implications of data-algorithm compatibility and your results that you can discuss?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

1: Your assessment is an educated guess. The submission is not in your area or the submission was difficult to understand. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

Societal impact does not seem relevant or needed to address for this paper. The paper acknowledges a limited scope to linear cases, but does not go into any additional weaknesses.

Authorsrebuttal2023-08-10

Calculations for A1

## Uniform Convergence on Ridge Regression We next show that ridge regression with regularization $1/t$ and uniform convergence cannot cover our results. We consider a case with data covariance $\lambda_i=\frac{1}{i^{1.5}}$, and show that ridge regression would return a bound with compatibility region shorter than $(0, \sqrt{n})$, which cannot cover our results with compatibility region $(0,n)$. The uniform convergence bound scales like $O\left(\frac{||\theta_\lambda||}{\sqrt{n}}\right)$ (see, e.g., Theorem 5.10 in [12]), where $\theta_\lambda$ denote the ERM solution of ridge regression with regularization $\lambda$. We calculate the excess risk of $\theta_\lambda$ as follows. The key insight comes from the fact that the noise matters a lot in the parameter norm. $$\mathbb{E}\left[||\theta_\lambda||^2\right]=\mathbb{E}\left[||X^\top (XX^\top+\lambda I)^{-1}Y||^2\right]=\mathbb{E}\left[||X^\top (XX^\top+\lambda I)^{-1}(X\theta^*+\varepsilon)||^2\right] =\mathbb{E}\left[||X^\top (XX^\top+\lambda I)^{-1}(X\theta^*)||^2\right]+\mathbb{E}\left[||X^\top (XX^\top+\lambda I)^{-1}\varepsilon||^2\right]\ge \mathbb{E}\left[||X^\top (XX^\top+\lambda I)^{-1}\varepsilon||^2\right] =\mathbb{E}\text{Tr}\left[X^\top (XX^\top+\lambda I)^{-1}\varepsilon\varepsilon^\top (XX^\top+\lambda I)^{-1}X\right]\ge \sigma_\varepsilon^2 \mathbb{E}\text{Tr}\left[XX^\top (XX^\top+\lambda I)^{-2} \right] =\sigma_\varepsilon^2 \sum_{i=1}^n \frac{\mu_i}{(\mu_i+\lambda)^2},$$ where we assume the noise vector is conditionally unbiased and has a lower bounded variance, as in Theorem B.2. We next assumes that $t = 1/\lambda > \sqrt{n}$. Then, according to Lemma A.1 and B.2 in our paper, it holds that $ \mu_n=\Theta(\frac{1}{\sqrt{n}})=\Omega(\lambda)$ for $\lambda_i=i^{\frac{-3}{2}}$, where $k_0$ denotes the effective rank in the paper. Therefore, it holds that $$\sigma_\varepsilon^2 \sum_{i=1}^n \frac{\mu_i}{(\mu_i+\lambda)^2}=\Omega\left(\sigma_\varepsilon^2 \sum_{i=1}^n \frac{1}{\mu_i}\right) = \Omega(\sqrt{n}),$$ which leads to a vacuous generalization bound in uniform convergence. Therefore, the related compatibility region must be shorter than $(0, \sqrt{n})$. However, in this case, our results show a compatibility region $(0,n)$, which is better. ## Benign Overfitting in Ridge Regression We additionally show that even if we apply benign overfitting in ridge regression, the returned compatibility region still cannot cover our results. To apply Theorem 1 in [2], we first calculate the effective rank $k^*$. Note that by plugging in $\lambda_i=\frac{1}{i^{1.5}}$ and $\lambda = 1/\sqrt{n}$, $$ r_k=\frac{1}{\lambda_{k+1}}\left(\lambda+\sum_{i>k} \lambda_i\right)=\Theta\left(\frac{k^{\frac{3}{2}}}{\sqrt{n}}+k\right), $$ which increase monotonically with $r_n = \Theta(n)$ and therefore $$ k^*=\min \lbrace\kappa: \rho_\kappa>b\rbrace=\min \lbrace\kappa: r_\kappa>b n\rbrace=\Theta(n) $$ According to Lemma 11 in [2], we have $\bar{k}=\Theta(n)$, $k=\min\lbrace k^*,\bar{k}\rbrace=\Theta(n)$ and the first term in the variance part $\frac{k}{n}=\Omega(1)$, which makes the bound vacuous for this setting.

Reviewer YpbP2023-08-17

Thanks for the clarification, my concerns have been addressed and I'm happy to bump up my score.

Authorsrebuttal2023-08-18

Thank you!

We thank the reviewer for the kind reply! We ensure to incorporate the discussions in the next revision, and remain eager to engage in further clarifications to assist the evaluation.

Reviewer ytRs2023-08-12

Rebuttal update

I thank the authors for the rebuttal. Having read the rebuttal and other reviews I decided to keep my score.

Authorsrebuttal2023-08-12

Thank you!

We thank the reviewer for the supportive feedback. We will remain eager to engage in any further clarifications if needed.

Reviewer dysH2023-08-12

Still confused about the c's

Thank you for the clarifications, but I am still confused about the cs. The Cs on line 169 and 173 now make sense to me. I am still unsure about the ones in equations 4 and 5 and the one for Theorem 5.1. Clarification would be appreciated.

Authorsrebuttal2023-08-13

Further Clarifications on the Constants

We apologize for any lack of clarity and would like to offer further clarifications below. The constant $c, c_0, c_1$ are mainly served to make the statements more rigorously. One can replace the constants with $\mathcal{O}(\cdot)$ notations to make it more clear. For example, the statements regarding $c$ in Theorem 5.1 is equivalent to $k_0 =\mathcal{O}(n), \log \frac{1}{\delta} =\mathcal{O}(n), \lambda =\mathcal{O} \left(\frac{1}{\sum_{i>0} \lambda_i}\right)$. In this equation, the first condition constrain the regularity of the problem class, the second condition lower bounds the error probability, and the third condition upper bounds the learning rate, which are all standard in related literatures. The constants $c_0,c_1$ can be interpreted similarly. Constants in our paper do not depend on input dimensions ($p$) or sample sizes ($n$) or input covariance ($\Sigma$). These constants may only be associated with other terms assumed to be of order O(1) in the paper, such as the subGaussian parameter ($\sigma_x$). This simplification aims to symplify our results and shift the focus towards generalization bounds concerning input dimensions and sample sizes. This technique is a common practice in the related literature [1, 2]. Therefore, one can select a suitably set of constants (without dependence on input dimensions or sample sizes) for the main Theorems to hold. Difference choices of constants may yield different generalization bounds, but these bounds all share the same rate with regard to sample size $n$, as established in Theorem 5.1 using the $\lesssim$ notation. We again express our gratitude to the reviewer for the feedback. We will make a clear explanation on the constants in the next revision. We are eager to engage in any further discussions to help the evaluation. Thank you! [1] Peter L. Bartlett, Philip M. Long, Gábor Lugosi, and Alexander Tsigler. Benign overfitting in linear regression. CoRR, abs/1906.11300, 2019. [2] Difan Zou, Jingfeng Wu, Vladimir Braverman, Quanquan Gu, and Sham M. Kakade. Benign overfitting of constant-stepsize SGD for linear regression.

Reviewer 5qZn2023-08-16

Rebuttal response

I have read the rebuttal presented by the authors, and appreciate their clarifications. I look forward to seeing the reorganization and improvements in clarity in the next revision.

Authorsrebuttal2023-08-16

Thank you!

We again thank the reviewer for the precious time and valuable feedback, and make sure to revise the current manuscript to improve its clarity. We remain eager to engage in any further clarifications to help the evaluation.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC