Rethinking the Discount Factor in Reinforcement Learning: A Decision Theoretic Approach

Reinforcement learning (RL) agents have traditionally been tasked with maximizing the value function of a Markov decision process (MDP), either in continuous settings, with fixed discount factor γ

cs.LG cs.AI Computer Science Mathematics

View graph Source

Paper

References (44)

01Source Traces for Temporal Difference LearningSilviu Pitis2018 · AAAI Conference on Artificial Intelligence · 21 citations In Library

02Occam's razor is insufficient to infer the preferences of irrational agentsS. Armstrong, Sören Mindermann2017 · Neural Information Processing Systems · 97 citations In Library

03Deep Reinforcement Learning from Human PreferencesP. Christiano, Jan Leike, Tom B. Brown et al.2017 · Neural Information Processing Systems · 5.7k citations In Library

04The Predictron: End-To-End Learning and PlanningDavid Silver, H. V. Hasselt, Matteo Hessel et al.2016 · International Conference on Machine Learning · 310 citations In Library

05The successor representation in human reinforcement learningI. Momennejad, E. Russek, J. H. Cheong et al.2016 · Nature Human Behaviour · 413 citations

06Unifying Task Specification in Reinforcement LearningMartha White2016 · International Conference on Machine Learning · 99 citations In Library

07Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic MotivationTejas D. Kulkarni, Karthik Narasimhan, A. Saeedi et al.2016 · Neural Information Processing Systems · 1.3k citations In Library

08Deep Recurrent Q-Learning for Partially Observable MDPsMatthew J. Hausknecht, P. Stone2015 · AAAI Fall Symposia · 2.0k citations In Library

09Playing Atari with Deep Reinforcement LearningVolodymyr Mnih, K. Kavukcuoglu, David Silver et al.2013 · arXiv.org · 14k citations In Library

10Policy Shaping: Integrating Human Feedback with Reinforcement LearningShane Griffith, K. Subramanian, Jonathan Scholz et al.2013 · Neural Information Processing Systems · 484 citations

11A Bayesian Approach for Policy Learning from Trajectory Preference QueriesAaron Wilson, Alan Fern, Prasad Tadepalli2012 · Neural Information Processing Systems · 202 citations

12Discounting axioms imply risk neutralityM. J. Sobel2012 · Annals of Operations Research · 12 citations

Scroll for more · 32 remaining

Similar papers

Multi-Discounting Reinforcement Learning Based on Reward DecompositionPengbin Chen, Qi Liu, Yanjie Li et al.2026 · IEEE Transactions on Emerging Topics in Computing · 0 citations In Library

Multi-Discounting Reinforcement Learning Based on Reward DecompositionPengbin Chen, Qi Liu, Yanjie Li et al.2024 In Library

Taylor Expansion of Discount FactorsYunhao Tang, Mark Rowland, R. Munos et al.2021 · International Conference on Machine Learning · 9 citations In Library

Decoupling Time and Risk: Risk-Sensitive Reinforcement Learning with General DiscountingM. Moghimi, A. Coache, Hyejin Ku2026 · arXiv.org · 0 citations In Library

Exploration of the Possibility for a Wider Range of the Discount Factor for Reinforcement LearningMonica Li2024 In Library

Adaptive Discount Factor for Deep Reinforcement Learning in Continuing Tasks with UncertaintyMyeongSeop Kim, Jung‐Su Kim, Myoung‐Su Choi et al.2022 · Sensors · 30 citations In Library

An Improvement for Value-Based Reinforcement Learning Method Through Increasing Discount Factor SubstitutionLinjian Hou, Zhengming Wang, Han Long2021 · IEEE International Conference on Computational Science and Engineering · 3 citations In Library

Reinforcement Learning with Non-Exponential DiscountingM. Schultheis, C. Rothkopf, H. Koeppl2022 · Neural Information Processing Systems · 18 citations In Library

Discount Factor as a Regularizer in Reinforcement LearningRon Amit, Ron Meir, Kamil Ciosek2020 · arXiv (Cornell University) · 90 citations In Library

Adaptive Multi-Horizon Reinforcement LearningManoosh Samiei, D. Precup, P. Masset2026 · 0 citations In Library

Patents near this paper

REINFORCEMENT LEARNING METHOD IN WHICH DISCOUNT FACTOR IS AUTOMATICALLY ADJUSTEDUS 10,581,885 · KOREA INTERNET & SECURITY AGENCY · 2020 DIRECT INVERSE REINFORCEMENT LEARNING WITH DENSITY RATIO ESTIMATIONUS 10,896,383 · OKINAWA INSTITUTE OF SCIENCE AND TECHNOLOGY SCHOOL CORPORATION · 2021 REINFORCEMENT LEARNING WITH A STOCHASTIC ACTION SETUS 11,615,293 · ADOBE INC. · 2023 INVERSE REINFORCEMENT LEARNING BY DENSITY RATIO ESTIMATIONUS 10,896,382 · OKINAWA INSTITUTE OF SCIENCE AND TECHNOLOGY SCHOOL CORPORATION · 2021 METHODS AND APPARATUS FOR GENERATING FAST COUNTERFACTUAL EXPLANATIONS FOR BLACK-BOX MODELS USING REINFORCEMENT LEARNINGUS 11,403,538 · ARTHUR AI, INC. · 2022

Source of record

arXiv