Stochastic Newton Proximal Extragradient Method

Stochastic second-order methods achieve fast local convergence in strongly convex optimization by using noisy Hessian estimates to precondition the gradient. However, these methods typically reach superlinear convergence only when the stochastic Hessian noise diminishes, increasing per-iteration costs over time. Recent work in [arXiv:2204.09266] addressed this with a Hessian averaging scheme that achieves superlinear convergence without higher per-iteration costs. Nonetheless, the method has slow global convergence, requiring up to $\tilde{O}(\kappa^2)$ iterations to reach the superlinear rate of $\tilde{O}((1/t)^{t/2})$, where $\kappa$ is the problem's condition number. In this paper, we propose a novel stochastic Newton proximal extragradient method that improves these bounds, achieving a faster global linear rate and reaching the same fast superlinear rate in $\tilde{O}(\kappa)$ iterations. We accomplish this by extending the Hybrid Proximal Extragradient (HPE) framework, achieving fast global and local convergence rates for strongly convex functions with access to a noisy Hessian oracle.

Paper

References (27)

Scroll for more · 15 remaining

Similar papers

Peer review

Reviewer TogL7/10 · confidence 3/52024-07-05

Summary

The paper develops an accelerated scheme for strongly convex problems. As feedback it requires a deterministic first order oracles, but only an inexact Hessian estimator –– the main assumption being that the Hessian noise is mean zero and sub-exponential (implied by e.g. Hessian subsampling and $\Vert\nabla^2 f_i(x)\Vert$ bounded or Hessian sketching with a Gaussian sketch). The main idea is in combining the Hessian averaging scheme [1] with the hybrid proximal extragradient framework [18,19].

Strengths

The paper is technically strong and very clearly/transparently written.

Weaknesses

The main weaknesses seems to be that: - It is unclear whether the scheme has relevance in practice, since the experiments seems to be run with a nonaccelerated version. For completeness I would suggest running the accelerated version, even if it performs worse (at least in the appendix). The theoretical results are strong and sufficient in themselves, so even though it would be better to find an instance that matches the theoretically guarantees, I don't see it as necessary. - there are no guarantees for convex as also pointed out by the authors. Neither are critical concerns – I mainly have a few remarks and questions regarding comparison with other methods and the experiments (see below).

Questions

Method comparison & theory: - It would be informative to compare theoretically with existing (accelerated) results using exact Hessian. Does the proposed method recover the existing complexities? - l. 369 can you really claim better condition number dependency when you choose $x_{k+1}=\hat{x}_k$, which is not covered by the theory? What convergence guarantees can be shown then, since the interpolation in Eq. 4 otherwise appears crucial for the acceleration? - How does the structure of the scheme compare with [1] when $x_{k+1}=\hat{x}_k$? It seems that the main difference is the stepsize selection (error condition vs Armijo style backtracking line search). If this is true, the experimental comparison can seem a bit synthetic. - If you have full gradients, is it not possible to use Pearlmutter's implicit Hessian-vector product to (inexactly) do the second order update while still maintaining a $\mathcal{O}(nd)$ like first-order methods? See e.g. https://www.cs.toronto.edu/~jmartens/docs/Deep_HessianFree.pdf - Is it possible to relax the requirement of exact gradients? For first order methods it is still possible to achieve exponential convergence under e.g. relative noise (see e.g. Thm 5.1 in https://arxiv.org/pdf/2102.02921). Experimentally: - What are choices of hyperparameters $\alpha, \beta$ of the method? How are the baselines tuned in comparison? - The proposed method can also be used with a deterministic Hessian. How does the stochastic version compare with an exact Hessian? This would show the influence of the first (slow) phase and provide an idealized baseline. Minor: - Is it possible to shave off the logarithmic factors hiding in the $\widetilde{\mathcal O}$-notation? - Figure 1: is it a labeling mistake or is SNPE-UnifAvg consistently better than SNPE-WeightAvg? - How come stochastic NPE suffers an addition $\Upsilon$ dependency in the superlinear rate (in comparison with stochastic Newton does not)?

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

See weaknesses.

Reviewer BRp15/10 · confidence 4/52024-07-12

Summary

This work proposes a novel algorithm called the Stochastic Newton Proximal Extragradient method. Authors claim that their method reaches a superlinear convergence rate after $\mathcal{O}(\kappa)$ iterations, in contrast to the $\mathcal{O}(\kappa^2)$ iterations proved in previous work.

Strengths

- This paper proposes a new algorithm called the Stochastic Newton Proximal Extragradient method and proves its convergence.

Weaknesses

- The assumptions are very restrictive. For example, assumptions 3 and 5 are very strong, and few machine learning applications can satisfy such assumptions. - I am confused about how parts (c) and (d) of Theorem 1 prove superlinear convergence. From superlinear convergence, I expected to see $\rho^{2^t}$ where $\rho < 1$. How do the results show superlinear convergence? - In Table 2, you mentioned the complexity of Damped Newton where the dependence on $\epsilon$ is $\log \log \epsilon$. However, for SNPE, the iteration complexity $\log \epsilon$ is similar to AGD. The iteration complexity of Damped Newton is called superlinear, while that of SNPE is called just linear. - In lines 342- 344, the dominating term in the complexity is the one that depends on $\epsilon$. The Damped Newton has better dependence than SNPE. - Section 7 needs more numerical experiments. I expect to see a comparison between SNPE and Damped Newton. In that comparison, putting **real-time (and not iterations)** on the $x$-axis will be fair, as SNPE does a line search in every iteration. - In lines 334-337, the authors ignore the complexity of line-search in algorithm 1 (BLS). In the BLS, you compute $(1 + \eta \tilde{H})^{-1}g$ several times, which is very expensive, and authors ignore this while computing the complexity of SNPE.

Questions

Check weaknesses.

Rating

5

Confidence

4

Soundness

2

Presentation

2

Contribution

2

Limitations

NA

Reviewer m5uq6/10 · confidence 4/52024-07-12

Summary

This paper uses the hybrid proximal extragradient framework to accelerate the convergence of Hessian average. The theoretical results significantly reduce the number of iterations to enter the linear phase, initial superlinear phase, and final superlinear phase when compared to the initial Hessian average method.

Strengths

The theoretical results of this paper are impressive. It improves the results of Hessian average [Na e.t.al 2022]. The idea of incorporating NPE framework and Hessian average is interesting. The paper is generally well-written and the results are easy to follow.

Weaknesses

This paper does not provide empirical results for the proposed methods against AGD, which is necessary. The proposed methods still require exact gradient oracle and its iteration complexity is $O({\kappa}+\log(1/\epsilon))$, while the classical AGD method require only $O(\sqrt{\kappa}\log(1/\epsilon))$. It is very important to use numerical results to exhibit the benefits of using second-order information.

Questions

Refer to weakness part.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

N/A

Reviewer qDEj7/10 · confidence 2/52024-07-31

Summary

Newton method is well-known for its local quadratic convergence. However, the use of Hessian introduces additional computation challenges. One approach to tackling this issue is an inexact approximation of the Hessian. In this paper, the authors consider the finite sum minimization problem. They propose a stochastic approximation of the Newton method, where instead of the full Hessian they use its subsample approximation. The idea of the proposed method is based on another known approach, that uses weighted average over a subsample of Hessians. In this paper, the authors improve this approach using the Hybrid Proximal Extragradient (HPE) framework for strongly convex problems. As a result, the proposed algorithm achieves both linear and superlinear convergence areas in fewer iterations by improving the dependence on condition number of convergence rate.

Strengths

This paper introduces a new type of stochastic inexact Newton method with a better convergence rate than existing analogs. The paper is written in a clear way, sketchily describing its main points, whereas most of the technical details are provided in the appendix. As for me, this is a good way to describe your idea in such a limited space. Authors provide an intuition or explanation after every lemma and theorem, which is also a good practice.

Weaknesses

Authors consider only a strongly convex setup. However they mention extending their approach to convex case, this seems to me as a major limitation of this work. Additionally, to check condition (3) authors employ line-search, which introduces an additional logarithmic factor in the convergence rate. However this can be the burden of the HPE framework, it seems not very significant, but still an issue. ## Minor remarks 1. Line 168: remove "follows" from the end 2. Line 168: not Step 5, but Step 6 3. Sometimes you write "stepsize", sometimes - "step size". Please, be consistent.

Questions

1. Why do you use $\sigma_{t+1} = \eta_t/\beta$? Please provide more details.

Rating

7

Confidence

2

Soundness

3

Presentation

4

Contribution

3

Limitations

No limitations.

Reviewer BRp12024-08-09

Thank you for your explanation of superlinear convergence and the plots. So SNPE performs better than Newton in the regime $n >> d$. Adding plots for other regimes to the appendix will be beneficial (just a suggestion to check the performance of SNPE compared to Newton). I will raise my score to 4. I am still not convinced about the iteration complexity in the table. You mentioned the iteration complexity of SNPE has dependence $\frac{\log(\varepsilon^{-1})}{\log \log(\varepsilon^{-1})}$. Is there a proof of this in the appendix that I can check? Else, can you add the proof here?

Authorsrebuttal2024-08-09

Thank you for reading our rebuttal and for the follow-up comments. **SNPE v.s. Newton in the regime $n = O(d)$.** Thank you for the suggestion. We will include the additional plots for the regime where $n = O(d)$ in the appendix of our revision. --- **The iteration complexity of $\log(\epsilon^{-1})/\log\log(\epsilon^{-1})$.** Thank you for the question. Due to space constraints, we did not include the proof in the rebuttal. However, we will provide a proof sketch below. Note that for $t \geq \mathcal{U}\_3 = \tilde{O}(\Upsilon^2 + \kappa)$, we have $\\|x_{t+1}-x^\*\\| = \tilde{O}(\frac{\Upsilon}{\sqrt{t}}) \\|x_{t}-x^\*\\|$ by Theorem 2. By unrolling the inequality, this implies that $$\\|x_{t+1}-x^\*\\| \leq \\|x_{\mathcal{U}\_3}-x^\*\\| \tilde{O}\left(\prod_{s = \mathcal{U}\_3}^t \frac{\Upsilon}{\sqrt{s}}\right) \leq \\|x_0 - x^\*\\| \tilde{O}\left(\prod_{s = \mathcal{U}\_3}^t \frac{\Upsilon}{\sqrt{s}}\right).$$ Further, for $t \geq 2\mathcal{U}\_3$, we can upper bound $\frac{\Upsilon}{\sqrt{s}}$ as follows: - $\frac{\Upsilon}{\sqrt{s}} \leq \frac{\Upsilon}{\sqrt{\mathcal{U}\_3}} \leq 1$ for any $s \in [\mathcal{U}\_3, t/2]$, - $\frac{\Upsilon}{\sqrt{s}} \leq \frac{\Upsilon}{\sqrt{t/2}}$ for any $s \in [t/2,t]$. Thus, we have $$\prod_{s = \mathcal{U}\_3}^t \frac{\Upsilon}{\sqrt{s}} \leq \prod_{s = t/2}^t \frac{\Upsilon}{\sqrt{s}} \leq \left(\frac{\Upsilon}{\sqrt{t/2}}\right)^{\frac{t}{2}}.$$ To derive a complexity bound, we upper bound the required number of iterations $t$ such that $ \left(\frac{\Upsilon}{\sqrt{t/2}}\right)^{\frac{t}{2}} = \epsilon$. Taking the logarithm of both sides and with some algebraic manipulation, we obtain $$\frac{t}{2\Upsilon^2} \log \frac{t}{2\Upsilon^2} = \frac{2}{\Upsilon^2} \log \frac{1}{\epsilon}.$$ Using the [Lambert W function](https://en.wikipedia.org/wiki/Lambert_W_function), the solution can be expressed as $ \log \frac{t}{2\Upsilon^2} = W(\frac{2}{\Upsilon^2} \log \frac{1}{\epsilon}) \Rightarrow t = 2\Upsilon^2 e^{ W(\frac{2}{\Upsilon^2} \log \frac{1}{\epsilon})}$. Finally, by applying the bound $e^{W(x)} \leq \frac{2x+1}{1 + \log (x+1)}$ for any $x \geq 0$, we conclude that $t = O\left(\frac{\log(\epsilon^{-1})}{\log(\log(\epsilon^{-1}))}\right)$. Note that in the above derivation, we ignore the additional logarithmic factor $\log (t)$ in our superlinear convergence rate. However, a more careful analysis will show that it does not affect the final complexity bound. We also refer the reviewer to a similar derivation in Appendix D.2 of Jiang et al. (2023), where the authors provide the same complexity bound for a similar convergence rate of $(1+ O(\sqrt{t}))^{-t}$. R. Jiang, Q. Jin, and A. Mokhtari. "Online learning guided curvature approximation: A quasi-Newton method with global non-asymptotic superlinear convergence." COLT 2023.

Reviewer BRp12024-08-09

Thank you for the details. Please add them in the updated version. I will raise my score to 5.

Reviewer TogL2024-08-09

I thank the authors for their response and have no further questions.

Reviewer m5uq2024-08-10

I thank the authors for their response and have no further questions.

Area Chair LBpu2024-08-11

Discussion

Dear Reviewers, Thank you for your effort in reviewing this paper. Please go over the authors rebuttal (and I suggest also other reviews and rebuttals to those) and see if it changes your mind or answers any of your concerns. In particular please engage in discussion with authors on points which have been clarified in the rebuttal or discuss why the rebuttal does not answer your concerns. Thanks you! AC

Reviewer qDEj2024-08-14

I thank the authors for the answers. I don't have any further questions and increased my rating to 7.

© 2026 NYSGPT2525 LLC