A History of Meta-gradient: Gradient Methods for Meta-learning

The history of meta-learning methods based on gradient descent is reviewed, focusing primarily on methods that adapt step-size (learning rate) meta-parameters.

cs.LG Machine Learning and Data Classification Computer Science

View graph Source

Paper

References (57)

01Discovery of Useful Questions as Auxiliary TasksVivek Veeriah, Matteo Hessel, Zhongwen Xu et al.2019 · Neural Information Processing Systems · 90 citations In Library

02Examining the Use of Temporal-Difference Incremental Delta-Bar-Delta for Real-World Predictive Knowledge ArchitecturesJ. Günther, Nadia M. Ady, Alex Kearney et al.2019 · Frontiers in Robotics and AI · 9 citations In Library

03Meta-learning for Predictive Knowledge Architectures: A Case Study Using TIDBD on a Sensor-rich Robotic ArmJ. Günther, Alex Kearney, Nadia M. Ady et al.2019 · Adaptive Agents and Multi-Agent Systems · 5 citations

04A general reinforcement learning algorithm that masters chess, shogi, and Go through self-playDavid Silver, T. Hubert, Julian Schrittwieser et al.2018 · Science · 4.1k citations

05Meta-Gradient Reinforcement LearningZhongwen Xu, H. V. Hasselt, David Silver2018 · Neural Information Processing Systems · 381 citations In Library

06Metatrace: Online Step-size Tuning by Meta-gradient Descent for Reinforcement Learning ControlK. Young, Baoxiang Wang, Matthew E. Taylor2018 · arXiv.org · 16 citations In Library

07TIDBD: Adapting Temporal-difference Step-sizes Through Stochastic Meta-descentAlex Kearney, Vivek Veeriah, Jaden B. Travnik et al.2018 · arXiv.org · 17 citations In Library

08Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement LearningAnusha Nagabandi, Ignasi Clavera, Simin Liu et al.2018 · International Conference on Learning Representations · 665 citations In Library

09Crossprop: Learning Representations by Stochastic Meta-Gradient Descent in Neural NetworksVivek Veeriah, Shangtong Zhang, R. Sutton2017 · ECML/PKDD · 12 citations

10Model-Agnostic Meta-Learning for Fast Adaptation of Deep NetworksChelsea Finn, P. Abbeel, S. Levine2017 · International Conference on Machine Learning · 15k citations In Library

11DeepStack: Expert-level artificial intelligence in heads-up no-limit pokerMatej Moravcík, Martin Schmid, Neil Burch et al.2017 · Science · 1.0k citations In Library

12Learning to learn by gradient descent by gradient descentMarcin Andrychowicz, Misha Denil, Sergio Gomez Colmenarejo et al.2016 · Neural Information Processing Systems · 2.2k citations In Library

Scroll for more · 38 remaining

Similar papers

One Step at a Time: Pros and Cons of Multi-Step Meta-Gradient Reinforcement LearningClément Bonnet, Paul F. Caron, Thomas D. Barrett et al.2021 · arXiv (Cornell University) · 6 citations In Library

A Lazy Approach to Long-Horizon Gradient-Based Meta-LearningMuhammad Abdullah Jamal, Liqiang Wang, Boqing Gong2021 · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · 9 citations In Library

Multi-step Estimation for Gradient-based Meta-learningJin-Hwa Kim, Junyoung Park, Yong-Seok Choi2020 · arXiv (Cornell University) · 1 citation In Library

Meta-Learning Optimal Learning Rate Policies: Towards Fully Adaptive Gradient DescentRevista, Zen, IA, 102025 · Zenodo (CERN European Organization for Nuclear Research) In Library

Adaptive Gradient-Based Meta-Learning MethodsM. Khodak, Maria-Florina Balcan, Ameet Talwalkar2019 · Neural Information Processing Systems · 397 citations In Library

Scalable Meta-Learning via Mixed-Mode DifferentiationIurii Kemaev, Dan A. Calian, Luisa M. Zintgraf et al.2025 · International Conference on Machine Learning · 5 citations In Library

Meta-Gradients in Non-Stationary EnvironmentsJelena Luketina, Sebastian Flennerhag, Yannick Schroecker et al.2022 · arXiv (Cornell University) In Library

Meta-Learned Adaptive Gradients: Discovering Optimal Descent Strategies for Non-Stationary EnvironmentsRevista, Zen, IA, 102025 · Zenodo (CERN European Organization for Nuclear Research) In Library

Patents near this paper

META-GRADIENT UPDATES FOR TRAINING RETURN FUNCTIONS FOR REINFORCEMENT LEARNING SYSTEMSUS 10,860,926 · DEEPMIND TECHNOLOGIES LIMITED · 2020 META-GRADIENT UPDATES FOR TRAINING RETURN FUNCTIONS FOR REINFORCEMENT LEARNING SYSTEMSUS 11,836,620 · DEEPMIND TECHNOLOGIES LIMITED · 2023 DEVICE AND METHOD FOR MACHINE-LEARNING STEP-SIZE ADAPTATIONUS 11,625,598 · ROYAL BANK OF CANADA · 2023 SYSTEM AND METHOD FOR INCREASING EFFICIENCY OF GRADIENT DESCENT WHILE TRAINING MACHINE-LEARNING MODELSUS 11,126,893 · INTUIT, INC. · 2021 SYSTEM AND METHOD FOR INCREASING EFFICIENCY OF GRADIENT DESCENT WHILE TRAINING MACHINE-LEARNING MODELSUS 11,763,151 · APPLIED MATERIALS, INC. · 2023

Source of record

arXiv