Improved Convergence in High Probability of Clipped Gradient Methods with Heavy Tails

In this work, we study the convergence \emph{in high probability} of clipped gradient methods when the noise distribution has heavy tails, ie., with bounded $p$th moments, for some $1

Stochastic Gradient Optimization Techniques Sparse and Compressive Sensing Techniques Statistical Methods and Inference

Paper

Similar papers

Improved Convergence in High Probability of Clipped Gradient Methods with Heavy Tailed NoiseTa Duy Nguyen, Thien Nguyen, Alina Ene et al.2023 · Neural Information Processing Systems · 56 citations In Library

High Probability Convergence of Clipped-SGD Under Heavy-tailed NoiseTa Duy Nguyen, Thien Hang Nguyen, Alina Ene et al.2023 · arXiv (Cornell University) In Library

Algorithms with Gradient Clipping for Stochastic Optimization with Heavy-Tailed NoiseМ. А. Данилова2023 · Doklady Mathematics In Library

High Probability Analysis for Non-Convex Stochastic Optimization with ClippingShaojie Li, Yong Liu2023 · arXiv (Cornell University) · 9 citations In Library

High Probability Analysis for Non-Convex Stochastic Optimization with ClippingShaojie Li, Yong Liu2023 · Frontiers in artificial intelligence and applications In Library

High probability convergence of distributed clipped stochastic gradient descent with heavy-tailed noiseYuchen Yang, Kaihong Lu, Long Wang2025 · Systems & control letters (Print) · 4 citations In Library

From Gradient Clipping to Normalization for Heavy Tailed SGDFlorian Hübler, Ilyas Fatkhullin, Niao He2024 · International Conference on Artificial Intelligence and Statistics · 65 citations In Library

High-probability Bounds for Non-Convex Stochastic Optimization with Heavy TailsAshok Cutkosky, Harsh Mehta2021 · arXiv (Cornell University) · 108 citations In Library

Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient ClippingZijian Liu, Zhengyuan Zhou2024 · International Conference on Learning Representations · 47 citations In Library

Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability GuaranteesAleksandar Armacki, Shuhua Yu, Pranay Sharma et al.2024 · arXiv.org · 3 citations In Library

Source of record

OpenAlex

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal