Incremental Learning in Mirror Flows

We study mirror flows generated by a convex quadratic loss and a general convex lower semicontinuous mirror potential. We show that, when initialized near the boundary of the domain of the mirror potential, their rescaled trajectories converge to a limiting mirror flow whose potential is the indicator function of the domain. In this limit, the primal variable minimizes the loss over a time-dependent hypothesis set: the subdifferential of the support function of the domain, evaluated at the dual variable. This characterization provides a general mechanism for incremental learning in mirror flows.

ML methods math.OC cs.LG stat.ML

View graph Source

Paper

References (33)

01Saddle-to-Saddle Dynamics Explains A Simplicity Bias Across Neural Network ArchitecturesYedi Zhang, Andrew M. Saxe, Peter E. Latham2025 · arXiv.org · 13 citations In Library

02Diagonal Linear Networks and the Lasso Regularization PathRaphael Berthier2025 · arXiv.org · 4 citations In Library

03Learning In-context n-grams with Transformers: Sub-n-grams Are Near-stationary PointsA. Varre, Gizem Yüce, Nicolas Flammarion2025 · International Conference on Machine Learning · 17 citations In Library

04On Learning Gaussian Multi-index Models with Gradient FlowA. Bietti, Joan Bruna, Loucas Pillaud-Vivien2023 · arXiv.org · 52 citations In Library

05Saddle-to-saddle dynamics in diagonal linear networksScott Pesme, Nicolas Flammarion2023 · Neural Information Processing Systems · 61 citations In Library

06SGD learning on neural networks: leap complexity and saddle-to-saddle dynamicsE. Abbe, Enric Boix-Adserà, Theodor Misiakiewicz2023 · Annual Conference Computational Learning Theory · 156 citations In Library

07Implicit Regularization for Group SparsityJiangyuan Li, THANH VAN NGUYEN, C. Hegde et al.2023 · International Conference on Learning Representations · 13 citations In Library

08Incremental Learning in Diagonal Linear NetworksRaphael Berthier2022 · Journal of machine learning research · 31 citations In Library

09Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror DescentZhiyuan Li, Tianhao Wang, Jason D. Lee et al.2022 · Neural Information Processing Systems · 45 citations In Library

10Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputsEtienne Boursier, Loucas Pillaud-Vivien, Nicolas Flammarion2022 · Neural Information Processing Systems · 86 citations In Library

11Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and SparsityArthur Jacot, F. Ged, Berfin cSimcsek et al.2021 · 85 citations In Library

12Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank LearningZhiyuan Li, Yuping Luo, Kaifeng Lyu2020 · International Conference on Learning Representations · 166 citations In Library

Scroll for more · 21 remaining

Similar papers

Implicit Bias of Mirror Flow on Separable DataScott Pesme, Radu-Alexandru Dragomir, Nicolas Flammarion2024 · arXiv (Cornell University) · 7 citations In Library

Sparse Q-learning with Mirror DescentS. Mahadevan, Bo Liu2012 · Conference on Uncertainty in Artificial Intelligence · 41 citations In Library

Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature LearningTom Jacobs, Guido Montúfar2026 · arXiv.org · 0 citations In Library

Adaptive Discretization of Bregman Gradient Flows: A Dynamical Systems Perspective on Mirror Descent Convergence GeometryHaitao Song, Yulei Wang2025 · Informatica In Library

Implicit Bias of Mirror Flow for Shallow Neural Networks in Univariate RegressionShuang Liang, Guido Montúfar2024 · arXiv (Cornell University) · 3 citations In Library

Primal-Dual Stochastic Mirror Descent for MDPsDaniil Tiapkin, Alexander Gasnikov2021 · arXiv (Cornell University) · 7 citations In Library

Target Mirror Descent: A Unifying Framework for Solving Monotone Variational InequalitiesYu-Wen Chen, Can Kızılkale, Murat Arcak2026 · arXiv (Cornell University) In Library

Non-convex online learning via algorithmic equivalenceUdaya Ghai, Zhou Lu, Elad Hazan2022 · Neural Information Processing Systems · 15 citations In Library

Deterministic Zeroth-Order Mirror Descent via Vector Fields with A Posteriori CertificationMasahito Hayashi2026 · Open MIND In Library

Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EMPierre-Cyril Aubin-Frankowski, Anna Korba, F. L'eger2022 · Neural Information Processing Systems · 56 citations In Library

Source of record

arXiv

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal