CSIAM Trans. Appl. Math., 6 (2025), pp. 711-759.
Published online: 2025-09
Cited by
- BibTex
- RIS
- TXT
Abstract. We establish a sharp uniform-in-time error estimate for the stochastic gradient Langevin dynamics (SGLD), which is a widely-used sampling algorithm. Under mild assumptions, we obtain a uniform-in-time $\mathcal{O}(η^2)$ bound for the Kullback-Leibler divergence between the SGLD iteration and the Langevin diffusion, where $η$ is the step size (or learning rate). Our analysis is also valid for varying step sizes. Consequently, we are able to derive an $\mathcal{O}(\eta)$ bound for the distance between the invariant measures of the SGLD iteration and the Langevin diffusion, in terms of Wasserstein or total variation distances. Our result can be viewed as a significant improvement compared with existing analysis for SGLD in related literature.
}, issn = {2708-0579}, doi = {https://doi.org/10.4208/csiam-am.SO-2024-0039}, url = {http://global-sci.org/intro/article_detail/csiam-am/24501.html} }Abstract. We establish a sharp uniform-in-time error estimate for the stochastic gradient Langevin dynamics (SGLD), which is a widely-used sampling algorithm. Under mild assumptions, we obtain a uniform-in-time $\mathcal{O}(η^2)$ bound for the Kullback-Leibler divergence between the SGLD iteration and the Langevin diffusion, where $η$ is the step size (or learning rate). Our analysis is also valid for varying step sizes. Consequently, we are able to derive an $\mathcal{O}(\eta)$ bound for the distance between the invariant measures of the SGLD iteration and the Langevin diffusion, in terms of Wasserstein or total variation distances. Our result can be viewed as a significant improvement compared with existing analysis for SGLD in related literature.