Summary
The work studies when a discretized Langevin dynamics under the Robbins-Monro-type stepsizes can converge to the Gibbs distribution. The paper obtains asymptotic results with very mild assumptions, and the framework not only includes Euler discretization, but many other sampling schemes as well, such as mirror Langevin, proximal, randomized mid-point and Runge-Kutta methods. The analysis builds upon constructing a continuous-time trajectory via interpolating the iterates, Wasserstein asymptotic pseudotrajectory and checking the stability condition by invoking the dynamical system theory.
Strengths
The paper is built upon solid mathematical analysis and it is a nice contribution to the vast literature of Langevin algorithms in machine learning. The builds provides a unified framework for the asymptotic guarantees under the Robbins-Monro scheme.
Weaknesses
Even though the mathematical theory is nice, the results may not have too much practical importance. The author(s) emphasized that this work is about asymptotic analysis instead of the non-asymptotic analysis. However, because the results are of asymptotic nature, it not clear to me what insights the results can provide concerning the schemes such as mirror Langevin, proximal, randomized mid-point and Runge-Kutta methods, because you only have asymptotic guarantees, it is impossible to use these results to compare these algorithms with more classic and basic Euler discretization of Langevin algorithms.
Also, I find the discussion of existing literature less satisfactory. Some of the claims and statements may not be that accurate. For example, on page 1, ``Existing guarantees suffer from the drawback of lacking guarantees for the last-iterates’’ ``the convergence is typically given on the averaged iterates instead of the more natural last iterates’’. To the best of my knowledge, there are numerous works on Langevin algorithms in the past decade and most of them are about last iterates guarantees in Wasserstein, KL or other distances; see e.g. Dalalyan and Karagulyan (2019), Dalalyan and Riou-Durand (2020), Ma et al. (2019), Ma et al. (2021), Raginsky et al. (2017), Gao et al. (2022). ``and little is known beyond the elementary schemes of stochastic gradient Langevin dynamics.’’ This is not accurate either. There have been many studies in the literature about underdamped Langevin, high-order Langevin, non-reversible Langevin and other variants of SGLD such as Dalalyan and Riou-Durand (2020), Hu et al. (2020), Ma et al. (2021), Mou et al. (2021), Gao et al. (2022).
Questions
(1) Page 4. ``The usual Lyapunov-type analysis for sampling algorithms focuses on bounding the change in relative entropy across iterations…” ``this makes the Lyapunov analysis applicable only to the simple Euler-Maruyama discreteization of (LD)’’ I am not too sure whether these two statements are accurate. Lyapunov functions are often used in analyis of Langevin algorithms, to show uniform bounds on the moments, e.g. Raginsky et al. (2017), in the coupling methods, e.g. Dalalyan and Riou-Durand (2020). It is definitely applicable beyond the Euler-Maruyama scheme, e.g. Dalalyan and Riou-Durand (2020) uses the disretization proposed in Cheng et al. (2018) to analyze kinetic Langevin dynamics.
(2) One technical point I would like to see more discussions is that in equation (1) in your Definition 1, it is for fixed $T>0$. Actually the dependence on $T$ can be exponential in $T$, which is quite common for weak approximation error in the literature. However, in order for the Langevin algorithm to converge to the Gibbs distribution, one often needs uniform-in-time guarantees, and would you need $T\rightarrow\infty$ in order to obtain Theorem 2?
(3) Theorem 2 is a very nice and clean result. But I am surprised that you only need assumption (10) which is an assumption on the discretized dynamics only. The reason I am asking is that it seems to me that Assumptions 1-3 alone do not guarantee that the continuous-time Langevin SDE has a unique stationary distribution. If Theorem 2 holds, that means assumption (10) can imply that the continuous-time Langevin SDE has a unique stationary distribution? The existence of $\pi$ is necessary for Theorem 2 to hold.
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Limitations
I did not see such discussions about limitations.