Unified Convergence Theory of Stochastic and Variance-Reduced Cubic Newton Methods

We study stochastic Cubic Newton methods for solving general possibly non-convex minimization problems. We propose a new framework, which we call the helper framework, that provides a unified view of the stochastic and variance-reduced second-order algorithms equipped with global complexity guarantees. It can also be applied to learning with auxiliary information. Our helper framework offers the algorithm designer high flexibility for constructing and analyzing the stochastic Cubic Newton methods, allowing arbitrary size batches, and the use of noisy and possibly biased estimates of the gradients and Hessians, incorporating both the variance reduction and the lazy Hessian updates. We recover the best-known complexities for the stochastic and variance-reduced Cubic Newton, under weak assumptions on the noise. A direct consequence of our theory is the new lazy stochastic second-order method, which significantly improves the arithmetic complexity for large dimension problems. We also establish complexity bounds for the classes of gradient-dominated objectives, that include convex and strongly convex problems. For Auxiliary Learning, we show that using a helper (auxiliary function) can outperform training alone if a given similarity measure is small.

Paper

Similar papers

Reviewer fBtY8/10 · confidence 3/52023-06-22

Summary

The authors propose a new way of solving minimization problems that are not necessarily convex. The main element of their approach is that they use some helper functions which give auxiliary information about the optimization problem at hand. That enables them to give improved optimization algorithms.

Strengths

Originality: The main approach is original. That is, combining stochastic and variance-induced second-order algorithms is a novel direction. Quality: Writing and results are of good quality; see questions. Clarity: Clear writing; see questions. Significance: Work is significant, as optimization is a central task in computer science.

Weaknesses

Not significant weaknesses found; see questions.

Questions

Line 28: Why is a saddle point not desirable? Line 30: Please explain what you mean by ill-conditioned. Line 58: What is the PL condition? Line 60: Can you please elaborate on the discussion regarding previous work? Line 70: Unclear. Line 83: Bullet is unclear. Why is it important in the parameter is smaller? Line 86: Bullet: Why are stochastic Cubic Newton algorithms with variance reduction for gradient-dominated case important? Because of their applicability? Line 98: Can you please discuss a bit Assumption 1? It is not trivial, right? Line 105: Can you please elaborate on the accuracy measure $\mu_c$? Line 120: Can you please cite some relevant work here? Line 157: Can you please elaborate on Assumption 2? Line 271: I do not understand the discussion before Equation (12).

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

Yes.

Reviewer bq1t5/10 · confidence 4/52023-07-06

Summary

In this paper, the authors introduce a meta-algorithm for second-order methods which they refer to as the ``helper framework''. Their helper framework unifies stochastic and variance-reduced versions of Newton's method, and the framework could also be viewed as auxiliary tasks when applied to first-order methods. The paper uses the helper framework to establish various convergence rates for objectives with Lipschitz Hessian and helper functions with two different similarity conditions (bounded similarity and Lipschitz similarity). Their algorithm (Algorithm 1) applies this framework to analyze cubic Newton. Different settings of Algorithm 1 (e.g. using different helper functions) replicate stochastic Newton, variance-reduced Newton, stochastic and variance-reduced Newton, lazy Hessian Newton, etc. They apply the helper framework to gradient-dominated objectives. They derive convergence rates for stochastic cubic Newton algorithms with variance reduction in the gradient-dominated case, which they claim to be the first to establish.

Strengths

1. Introduces a new framework to unify analysis of different variants of Newton's method. 2. Instead of assuming convexity or strong-convexity, extends framework to gradient-dominated functions in general. 3. Through this framework, the authors claim to achieve improved complexity guarantees. 4. Well-written paper with clear notation and good recap of related work.

Weaknesses

1. Unclear why we need a new framework to analyze the different variants of Newton's method. 2. Unclear if the meta-algorithm presented is implementable. 3. The improved complexity guarantees claimed in the paper are unclear. 4. The additional variables in the helper framework may make it harder to understand basic second-order methods. The reason why I only rated a 2 for Contribution of this paper are mainly due to the four points above. I may have misread or overlooked some details. Please see the questions below that address the perceived weaknesses in more detail.

Questions

Questions: 1. Is it useful to have a framework that unifies the different variants of Newton's method? What additional insights are revealed by using the helper framework? 2. Could you expand on the statement ``we recuperate the known rates of convergence'' on line 213? For readers unfamiliar with the convergence rates of various second-order methods, it could be useful to have a table that shows (method, previous known rate, rate from helper framework). 3. Is their main algorithm (Algorithm 1) implementable/ practical? For example, in one case (line 208), the input parameter $m$ in Algorithm 1 is value that minimizes the number of gradient and Hessian evaluations to find an $\epsilon$ stationary point. This seems to be a difficult optimization problem itself. Are there more practical ways of setting $m$? Another example is in Corollary 1 where $M$ is chosen to equal $L$. 4. The framework introduces some variables such as $\delta_1$ and $\delta_2$ in Assumptions 2 and 3. This is expected when a more general framework is introduced and we have additional concepts such as similarity. On the other hand, there is no closed-form for the values of these new variables in general and they also appear hard to estimate. Could the authors provide more insight/ theory/ experimental results for the new variables introduced so that the new complexity guarantees are more understandable? Minor questions: 1. The parameter $M$ is used in equation 2 but not defined until later at line 123? 2. What is $F_0$ in Theorem 1? 3. Line 169. Could you be more specific about which inequality is used so that Assumption 2 is satisfied? 4. Line 172. Is the result of Corollary 1 a complexity guarantee for a second-order method? If so, why is the comparison against first-order SGD? 5. Is there any difference between the definitions of $h_1$ and $h_2$ on line 215 and Equation 6? 6. There seems to be experimental results for the discussion of ``To be lazy or not to be?'' in the appendix. Why not add a link to this? 7. Line 244. The similarity measures in Assumption 2 and 3 only take in $\delta_1$ and $\delta_2$. What is the second variable in $(\delta_1,1)$ and $(\delta_2,2)$?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

Yes

Reviewer XQuJ6/10 · confidence 4/52023-07-10

Summary

The paper provides a unified analysis for the Cubic regularized Newton Methods with variance-reduced estimators to solve stochastic nonconvex optimization problems. The results rely on the helper framework which generalizes the use of stochastic estimators for first-order and second-order oracles. The analysis recovers best-known rates and improves in terms of Hessian computation via Lazy Hessian update. Convergence is considered in general nonconvex and gradient-dominated (PL condition) problems.

Strengths

The strengths of the paper lie within the unified convergence analysis that can recover best-known rates. This can be done via using the helper function which is a clever way to generalize the estimation of stochastic first-order amd second-order (possibly higher...) oracles. Although Algorithm 1 is similar to the original Cubic Newton method but slightly difference in the gradient/Hessian estimators and the update of snapshot points. All claims are backed by proof. I have gone through the proofs and I find no problems.

Weaknesses

The paper lists 2 options to update the snapshot but there is lack of discussion on why these options are chosen (not others). Other than that, I agree with the weaknesses pointed out by the authors, especially on the choice of the helper function. As helper function is a new concept, the paper does not provide much details on how to find a concrete choice of helper function given an application. Also note that details on the choice of helper function are not needed in the results due to the similarity assumption.

Questions

As we have freedom to choose helper function, is there a preferred choice of helper function and what properties that we are looking for in general? For the two options (4) and (5), the theoretical results only consider the first and I find nowhere else in the paper that uses the second option. Do experiments all use option (4)? If we use option (5) do we expect improvement in performance?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

I believe the authors have clearly pointed out the limitations of this work.

Reviewer zomT3/10 · confidence 4/52023-07-22

Summary

The paper studies stochastic Cubic Newton methods for solving possibly nonconvex minimization problems. The authors propose a flexible “helper framework” that unifies stochastic and variance-reduced second-order algorithms, offering global complexity guarantees. This framework allows arbitrary batch sizes, noisy gradient and Hessian estimates, and includes variance reduction and lazy Hessian updates. The work achieves state-of-the-art complexities for stochastic and variance-reduced Cubic Newton methods without artificial logarithms. It also introduces a new efficient lazy stochastic second-order method for large dimension problems.

Strengths

This paper proposes a new lazy stochastic second-order method.

Weaknesses

- The paper lacks a discussion of variance-reduced methods in the literature, neglecting numerous related papers on the subject. - The experiments are only in Appendix and weak with a “toy” dataset. - The loss function is just similar to “convex” although using the non-convex regularizer. - Many assumptions have been used in the paper. That could restrict the applications of the problems. - It is not clear about the results that could use for machine learning applications.

Questions

- What value of lambda for non-convex regularizer in line 411 did you choose? - Is it more suitable to submit this paper to an optimization journal?

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

NA

Reviewer bq1t2023-08-10

Many thanks to the reviewers for their detailed response. I appreciate the table of global complexities and the discussion of the interpretation of $\delta_1$ and $\delta_2$ and how to set $m$. I raised the score I gave for the paper although I still have some reservations on (a) the practicality of the algorithm given the need to estimate $m$ (I agree that the literature generally assumes $L$ is known), (b) the usefulness of convergence guarantees that rely on additional variables $\delta_1$ and $\delta_2$.

Authorsrebuttal2023-08-14

We would like to thank you again for your answer. We will try to address reservations and hope our answers are satisfying. - Regarding **(a)**, estimating $m$ is not problematic; it serves as a hyperparameter in our algorithm that we can determine using methods like grid search. In our article, we demonstrate that we can choose $m$ in a way that minimizes the total number of gradient calls. This number depends entirely on the knowledge of the count of gradients and Hessians utilized during each step. Surprisingly, this requirement only involves knowing the number of samples $n$ and the dimension $d$. To illustrate this, let's consider the case of the lazy Newton algorithm. In this scenario, we have $\delta_1=0$ and $\delta_2=L$, leading to $E[|\nabla f(x_{\text{out}})|^{3/2}]=\mathcal{O}( \frac{\sqrt{L}F_0}{S\sqrt{m}})$. To determine the optimal $m$ based on this rate, let's calculate the total number of gradient calls. It's evident that this count equals $n d S + n Sm = nS (d + m)$ because we used $S$ Hessians and $mS$ gradients. For a given precision $\varepsilon$, we have $S\sim \frac{\sqrt{L}F_0}{\varepsilon^{3/2}\sqrt{m}}$. Consequently, when minimizing $nS (d + m)$ with respect to $m$, we obtain $m=\sqrt{d}$. While this example is the simplest in the paper, the general idea is the same. - In response to **(b)**, to underscore the practical significance of our convergence guarantees hinging on parameters $\delta_1$ and $\delta_2$, we highlight instances (as discussed in the paper) where these parameters are precisely known. To be specific, Equation (10) in the paper presents the bound $\mathcal{O}\big(\frac{\sqrt{L}F_0}{Sm} + \frac{\sqrt{\delta_2}F_0}{S\sqrt{m}} + \frac{\sqrt{\delta_1}F_0}{S}\big)$, allowing for the direct substitution of known similarity parameter values. For example, in the case of cubic Newton's method, we have $\delta_1=\delta_2=0$. For Lazy-Newton, the values are $\delta_1=0$ and $\delta_2=L$. In the context of VRCN, $\delta_1=\mathcal{O}(\frac{L}{\sqrt{b_g}})$ and $\delta_2=\mathcal{O}(\frac{L}{\sqrt{b_h}})$, while for Lazy-VRCN, $\delta_1=\mathcal{O}(\frac{L}{\sqrt{b_g}})$ and $\delta_2=0$. This consolidated formulation encompasses all of these rates within a single formula. An insightful observation is that random batch sampling (performed in previous works), can be understood as an attempt to reduce similarity, and aligns with the objective of finding helpers with small similarities. However, it's worth noting that alternative methods might exist to achieve similar outcomes.

Program Chairsdecision2023-09-21

Decision

Reject

© 2026 NYSGPT2525 LLC