Approximate Heavy Tails in Offline (Multi-Pass) Stochastic Gradient Descent

A recent line of empirical studies has demonstrated that SGD might exhibit a heavy-tailed behavior in practical settings, and the heaviness of the tails might correlate with the overall performance. In this paper, we investigate the emergence of such heavy tails. Previous works on this problem only considered, up to our knowledge, online (also called single-pass) SGD, in which the emergence of heavy tails in theoretical findings is contingent upon access to an infinite amount of data. Hence, the underlying mechanism generating the reported heavy-tailed behavior in practical settings, where the amount of training data is finite, is still not well-understood. Our contribution aims to fill this gap. In particular, we show that the stationary distribution of offline (also called multi-pass) SGD exhibits 'approximate' power-law tails and the approximation error is controlled by how fast the empirical distribution of the training data converges to the true underlying data distribution in the Wasserstein metric. Our main takeaway is that, as the number of data points increases, offline SGD will behave increasingly 'power-law-like'. To achieve this result, we first prove nonasymptotic Wasserstein convergence bounds for offline SGD to online SGD as the number of data points increases, which can be interesting on their own. Finally, we illustrate our theory on various experiments conducted on synthetic data and neural networks.

Paper

References (27)

Scroll for more · 15 remaining

Similar papers

Peer review

Reviewer dWGg7/10 · confidence 3/52023-06-26

Summary

This paper investigates the approximate heavy-tailed behavior of stochastic gradient descent (SGD) in local minima in practical settings and its correlation with overall performance. It shows that the stationary distribution of offline SGD exhibits approximate power-law distribution, and the approximation error is controlled by how fast the empirical distribution of the training data converges to the true underlying data distribution in the Wasserstein metric. The main contribution is to fill the gap in understanding the underlying mechanism generating the reported heavy-tailed behavior in practical settings, where the amount of training data is finite. The paper also proves nonasymptotic Wasserstein convergence bounds for offline SGD to online SGD as the number of data points increases. The theory is verified with experiments

Strengths

- **Originality**: The paper investigates the heavy-tailed behavior of SGD in practical settings where the amount of data is finite, which has not been well understood before. It proves a nonasymptotic Wasserstein convergence bound for offline SGD which reduces to online SGD as the number of data points increases. The difference from a true heavy-tailed distribution with a finite number of data is given by the Wasserstein distance. - **Quality**: The paper provides rigorous theoretical analysis and proof to support its claims. It also conducts experiments to verify its theory. - **Clarity**: The paper is well-written and easy to follow, with clear explanations of technical terms and concepts. The authors also provide many clear schematic illustrations to help readers understand their concepts and results. - **Significance**: The theory of this paper is of both theoretical interest and direct practical significance. The results are easy to understand and bounds on the distance from a true heavy-tailed distribution are given explicitly with a clear form.

Weaknesses

- Possible evaluation method for the values of estimated tail indices is not provided. In Figure 5, the estimated tail indices depend on the step size $\eta$, the batch size $b$, or their ratio $\eta/b$, and of course the number of data $n$. It will be strong if an evaluation method for the tail indices, even approximately, can be provided.

Questions

- In Line 267, the authors state that the tail indices depend on the ratio of the step size $\eta$ to the batch size $b$. However, a mere dependence on the ratio $\eta/b$ has been found broken in online SGD with a large $\eta$ as discussed in Sec.6.6 of Ref.[1]. A modification that is linear in the batch size $b$ is found for a simple 1-dimensional example. Could the authors elaborate on this result? - Still in Sec.6.6 of Ref.[1], the tail index is estimated with an explicit expression in terms of $\eta$ and $b$ for a 1D example. Is it possible to compare the estimated tail indices predicted by your theorem with the explicit expression given in Ref.[1]? [1] **Strength of Minibatch Noise in SGD**, Liu Ziyin, Kangqiao Liu, Takashi Mori, Masahito Ueda, *ICLR 2022*

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

The authors adequately addressed the limitations.

Reviewer t9h86/10 · confidence 2/52023-06-30

Summary

This manuscript considers the problem of multi-pass stochastic gradient descent with a finite batch-size and strongly convex objective. The key result is to show that when the stationary distribution of the parameters is heavy-tailed in the infinite data (one-pass) limit, then the stationary distribution of the finite batch limit is approximately heavy-tailed, in the sense of having a heavy-tailed component plus a correction depending on the 1-Wasserstein distance between empirical and population data distribution (hence decaying at worst as $\sim n^{-1/2}$). The authors also present numerical experiments that suggest the relevance of this theoretical result to more practical settings.

Strengths

Understanding SGD in its different flavours is an important problem in theoretical machine learning, and results on multi-pass SGD are scarce compared to the one-pass case. Therefore, the contribution is timely and significant. The technical part is sound. Moreover, the paper is well-written and the thread is relatively easy to follow: the motivation and goals are clearly stated and the main result addresses them. Finally, the numerical simulations supporting a broader scope of applicability are nice.

Weaknesses

There are two immediate shortcomings of the work. First, the setting is rather restricted (strongly convex goals). Second, while the authors cite related literature connecting heavy-tails with generalization, this is not explicitly explored in the context of the paper. Overall, one question that remains in the end of the reading is: the stationary distribution of the weights are approximately heavy-tailed, but so what?

Questions

- **[Q1]**: As a reader who is not very familiar with this line of work ([HM21, HSKM22, MM], etc.) I wonder how to conciliate these results to the classical literature [Fabian 1968, Kushner 1981, Pflug 1986] showing asymptotic normality of the stationary distribution under certain conditions. For instance, considering online SGD in the simple least-squares setting. Since the gradient is proportional to the residual $r_{i} = y_{i}-\langle a_{i}, x\rangle$: $$ x^{k+1} = x^{k} + \gamma r_{k} a_{k} $$ if the residuals are Gaussian close to the minimum (e.g. $y_{i}=\angle a_{i}, x_{\star}\rangle + z_{i}$ for some $x_{\star}$ and $z_{i}\sim\mathcal{N}(0,\sigma^2)$), won't the stationary distribution be Gaussian? What I am missing? - **[Q2]**: Theorems 2 and 3 have an explicit dependence on the sample size $n$, but how do they depend on the batch size b? **Minor comments** - The following sentences are strong and misleading: > *Previous works on this problem only considered, up to our knowledge, online (also called single-pass) SGD, which requires an infinite amount of data.* > *However, all these results rely on exact heavy tails, which, to our current knowledge, can only occur in the online SGD regime where there is access to an infinite sequence of data points.* While it is true that in one-pass SGD the maximum number of steps is limited by the availability of data, this doesn't necessarily means that it requires an infinite amount of data. Note that the number of steps required generically depend on the task under consideration, and in some practical scenarios where data is abundant only a few epochs are required, see e.g. Table 2.2. of [[Kaplan et al. 2020]](https://arxiv.org/pdf/2005.14165.pdf). The authors even acknowledge this fact in L29-30. - Please add the details of the plots in the caption in Fig. 2 & 5: the reported tail indices correspond to what simulation? - L128: You mean $[n] = \{1,\cdots, n\}$? **References** [[Fabian 1968]](https://projecteuclid.org/journals/annals-of-mathematical-statistics/volume-39/issue-4/On-Asymptotic-Normality-in-Stochastic-Approximation/10.1214/aoms/1177698258.full) Vaclav Fabian. *On Asymptotic Normality in Stochastic Approximation*. Ann. Math. Statist. 39 (4) 1327 - 1332, August, 1968. https://doi.org/10.1214/aoms/1177698258 [[Kushner 1981]](https://epubs.siam.org/doi/10.1137/0319007) Harold J. Kushner and Hai Huang. *Asymptotic Properties of Stochastic Approximations with Constant Coefficients*. SIAM Journal on Control and Optimization, 19(1), 1981 [[Pflug 1986]](https://epubs.siam.org/doi/10.1137/0324039) Georg Ch. Pflug. *Stochastic minimization with constant step-size: Asymptotic laws*. SIAM Journal on Control and Optimization, 24(4):655–666, 1986

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

I have discussed some of the limitations in the "Weaknesses" part.

Reviewer rBYn7/10 · confidence 3/52023-07-04

Summary

In this paper, the authors study the heavy-tail distribution for the parameters in offline stochastic gradient descent algorithm (SGD). Theoretical results are provided for a quadratic loss and a strongly convex problem, while numerical results cover more realistic cases such as fully connected NN or CNN. In the theoretical part of the paper, the authors link the tail of the distribution of the offline SGD to the online one. To be more specific, the authors show that the qualitative difference between the online and the offline tail is bounded by the Wasserstein distance between the generating and empirical distribution of the data. The author further bound the Wasserstein distance between the distribution of the online and the offline parameters by the same distance. The theoretical result is qualitatively supported by the numerical experiment.

Strengths

* Originality and significance: the authors extend the analysis of heavy tail in SGD from the online settings to offline settings. Personally, I find it really interesting of having a quantitative non-asymptotic bound of the difference between the two cases. It could make the existing and future online SGD analysis more relevant. * Quality: In my personal opinion, the authors perform convincing theoretical and numerical analysis. * Clarity: The paper is well written. The authors present their results in a self-contained way, with multiple examples and intuitions helping illustrating the idea behind their main results. And I appreciate that the code is attached to the paper.

Weaknesses

It appears to me that the fig.1 and fig.3 are not as convincing as the rest of the paper. The axes are unlabeled in these figures. The fact that the distribution has a heavy tail is not very obvious in these figures. For example, to my eye, the n=250 histogram is more power-law comparing to the n=1000 histogram. The weaknesses and questions are well addressed by the authors in their rebuttal. The authors provided detailed explanations and additional figures.

Questions

I believe that it would be nice if the authors could include results, expressed in numbers, which characterizes the difference between the experimental tail and a real power-law tail.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors adequately addressed the limitations.

Reviewer VaqK8/10 · confidence 4/52023-07-07

Summary

This paper considers offline SGD and proves an approximate power-law tail behavior of the stochastic gradient for strongly convex objectives, confirming the heavy-tail heuristic encountered in practices. Explicit tail estimations are obtained, and as a intermediate result, nonasymptotic Wasserstein convergence results for offline SGD are proved.

Strengths

1. The paper provides a good analysis of the behavior of offline SGD, and contributes to our understanding of heavy-tail phenomena in SGD training. 2. I have gone over the proofs and believe it's basically correct. 3. The paper is well-written and comfortable to read.

Weaknesses

I do not see apparent weaknesses in this paper.

Questions

I have no additional questions.

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

Yes.

Reviewer rBYn2023-08-17

Thank you for the explanation and additional figures. I have raised my rating to "Accept".

Reviewer t9h82023-08-18

Thank you for the detailed answer and for welcoming my suggestions. > *If the step-size is large enough, $\alpha$ can be strictly smaller than 2, and the averaged iterates will converge to an $\alpha$-stable law, where the studies you mentioned cannot be applied. This has been proven in [1, Corollary 11].* How large is large enough in this simple least-squares example? To my knowledge for $\gamma\geq 2$ the mean-squared error $||x_{\star} - x_{k}||^{2}_{2} \to\infty$ as $k\to\infty$, so am confused on how this can correlate with generalization in this case.

Authorsrebuttal2023-08-18

Thank you for going over our rebuttal. This is indeed a good question; however, we shall mention that the question rather concerns prior work and it is quite orthogonal to our contributions. Nevertheless, let us try to provide an answer, which essentially relies on multiple "surprising" facts. We will try to summarize the picture as follows. 1. For the case of linear regression with Gaussian data, the step-size required for the iterates to converge to a distribution with infinite variance is given in arxiv:2006.04740 Proposition 5. For general loss functions, we are not aware such an explicit result. 2. You are right, when the step-size is large the iterates may diverge in L2 (as you mentioned); however, they can still converge in Lp with p < alpha < 2 (not to a point though, to a random vector instead), see arxiv:2006.04740 Theorem 8 and Theorem 6 (or you can see arxiv:2102.10346 Theorem 3 for a similar outcome in a different setting). 3. As we mentioned in our earlier response, the connection between heavy tails and generalization has been established in certain “sterile” settings. As an example for such a connection, let us consider arxiv:2206.01274 Theorem 4, where the authors showed that even when the original loss value diverges (e.g. your L2 example), the algorithm can generalize well under a "surrogate loss" (for instance the original loss can be L2 but the surrogate loss can be Lp -- this is what they considered in their result). There are other papers, which use bounded surrogate losses -- for instance using 0-1 loss (accuracy) as a surrogate loss whereas the algorithm tries to minimize another loss function (cross entropy etc). We hope that this answers your question. We remain at your disposal if there is any further questions.

Reviewer t9h82023-08-19

Thank you for the pointers and for the clarification. I will need some time to digest it. I will keep my score for now and might re-evaluate it after the discussion period with the other reviewers.

Reviewer dWGg2023-08-19

Your reply addresses my questions and comments. Thank you very much. I will keep my score as "Accept". I do not raise my score because a possible estimation of the tail index is beyond the scope of this paper. Anyway, I like your idea of using the Wasserstein distance.

Program Chairsdecision2023-09-21

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC