Model-Free Reinforcement Learning with the Decision-Estimation Coefficient

We consider the problem of interactive decision making, encompassing structured bandits and reinforcement learning with general function approximation. Recently, Foster et al. (2021) introduced the Decision-Estimation Coefficient, a measure of statistical complexity that lower bounds the optimal regret for interactive decision making, as well as a meta-algorithm, Estimation-to-Decisions, which achieves upper bounds in terms of the same quantity. Estimation-to-Decisions is a reduction, which lifts algorithms for (supervised) online estimation into algorithms for decision making. In this paper, we show that by combining Estimation-to-Decisions with a specialized form of optimistic estimation introduced by Zhang (2022), it is possible to obtain guarantees that improve upon those of Foster et al. (2021) by accommodating more lenient notions of estimation error. We use this approach to derive regret bounds for model-free reinforcement learning with value function approximation, and give structural results showing when it can and cannot help more generally.

Paper

References (34)

Scroll for more · 22 remaining

Similar papers

Peer review

Reviewer DHV36/10 · confidence 3/52023-07-05

Summary

The paper studies model-free RL algorithm using DEC as the complexity measure.

Strengths

Extending previous model-based algorithm to model-free algorithm is a nice contribution.

Weaknesses

I believe this paper provides a nice contribution for the DEC family of papers. But I would like the authors to discuss more on the implication side. Why we care about extending model-free algorithms to bilinear classes? There are too many complexity measures for RL function approximation that are proposed recently so I am concerned about how useful those theories are for practice. The algorithm for Bellman-Eluder dimension is model-free. Then can I ask why we need a new algorithm? I hope the authors could pay more attention to the computational side. So far, all the lines of DEC works are not computationally efficient and even lack any experiments.

Questions

See above.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

1. No discussion or solution on how to implement the algorithm. 2. The analysis seems to build on two existing works: Zhang [32] and Foster et al. [13]. Not very exciting about the technique.

Reviewer kpYc6/10 · confidence 4/52023-07-09

Summary

This paper studies a variant of the recently proposed framework of Foster et al. for sequential decision making, based on a concept called "decision estimation coefficient" (DEC). The variant proposed here is based on enhancing the estimation step in the "estimation-to-decisions" (E2D) with an optimistic bias inspired by the recent work of Zhang. The authors show that this biased estimation scheme, coupled with an appropriately adjusted decision-making rule, can satisfy very similar regret guarantees for a wide range of sequential decision problems, and can provide improvements in certain settings. In particular, the authors show that their approach can handle a large class of tractable models for reinforcement learning called "bilinear classes".

Strengths

The paper is well-written and the technical content is of excellent quality. The proposed approach is justified and explained well, and its analysis is also presented in an accessible way (at least on a high level). While the regret bounds for bilinear MDP classes is not novel in the sense that there exist other algorithms that achieve the same guarantees, I appreciate the conceptual contribution of showing that the DEC framework is also capable of tackling these (relatively) challenging problems. I appreciated the very careful comparison between all the relevant DEC variants in Section 3: the authors didn't just propose a technique and proved some bounds about it, but also explored a range of other opportunities and explained the differences between them in an accessible manner.

Weaknesses

On the negative side, the rates that the authors derive are not particularly great: the scaling goes from $T^{2/3}$ in the setting with the most stringent assumptions all the way to $T^{5/6}$ as more and more assumptions are dropped. While the authors discuss this limitation quite openly, I would have appreciated some more discussion as to where this relatively poor scaling comes from. One contributing factor is certainly the use of batched estimation steps. My understanding is that some further looseness may come from the optimistic bonuses added to the estimation procedure, which makes the total estimation error grow polynomially with the number of updates (as opposed to logarithmically, which would allow getting sqrt{T} rates after putting everything together). I wonder though if this intuition is correct, and I would appreciate it if the authors could clarify what rate they would get if they could afford to set n=1 (without paying for it). Altogether, it would have been nice if the to compare the various notions of estimation error with the same care as what the DEC variants have received. Overall, I am leaning towards suggesting acceptance, but I would feel more strongly about my support if the authors were able to address my questions above in a satisfying way.

Questions

See above.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

N/A

Reviewer ooU76/10 · confidence 3/52023-07-27

Summary

This paper contributes to a line of research on decision-estimation coefficients (and related algorithms). An optimistic variant of the original DEC is introduced and it is shown that this variant can lead to a new related meta-algorithm with optimism. This new structure and algorithm are shown to provide non-trivial regret bounds for model-free RL for bilinear classes (a known framework for sample efficient learning). It is also shown that the original DEC is insufficient to achieve the same. Finally it is shown that posterior sampling does not solve all the problems that E2D.Opt can solve.

Strengths

Overall this is a good paper that is presented clearly and makes an interesting contribution to a growing line of research. The application of E2D-style algorithms to model-free settings is important, as well as the ability to incorporate classes that were previously not handled. The discussion of where advantages exist for optimistic DEC are thorough and helpful. The additional results also shed light on the problems that can be handled by both coefficients and other algorithms.

Weaknesses

The new coefficient is used to reproduce an existing result. This leads to a natural new meta-algorithm and theorem, but since they are only shown in one specialized case, it’s unclear whether it’s useful beyond this one setting as a “meta-algorithm.” While it is apparent that one could swap out different divergences and oracles, it may be described better as a single algorithm, inspired by the original E2D. The regret bound appears to have a slightly worse rate than what would be achievable with the original BiLin-UCB, which I believe has T^{2 / 3} if converted properly to regret. It’s not clear how one would implement such an algorithm in practice.

Questions

Does optimistic DEC subsume all of the original DEC results? Are there any downsides compared to the original E2D? What benefit does the batching offer if the regret bounds still scale as poly(K)? Why not just use one sample per batch? Prop 3.4: shows that PS does not cover all the problems that E2D.Opt can solve. How about the other way? Prop 3.4: What is the expectation over? Is this an average regret over MDPs in the class and if so, is posterior sampling misspecified? Suggestions: (14) does not seem to point to an equation in the main paper. Lines 94-95: It may be more clear to say that “there exist algorithms/oracles” that do this.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

No, but this work is theoretical. The limitations are either apparent from the assumptions or discussed adequately. A conclusion to summarize would be helpful though.

Reviewer nAEr6/10 · confidence 3/52023-07-31

Summary

This paper adapted the DEC framework and E2D algorithm to incorporate model-free reinforcement learning. Specifically, this paper proposed the optimistic DEC and proved that it appears in the regret bound for the optimistic E2D algorithm with an optimistic estimation oracle. The example of bilinear class is provided. The paper discussed when the optimistic E2D algorithm will have benefit than E2D in its original form.

Strengths

This paper is well-written and provides concrete steps toward understanding the general framework of DEC. The original DEC framework is mostly model-based decision making, whereas the new framework can handle model-free RL.

Weaknesses

This paper is largely built upon [13] and [32]. The additional results over [13] and [32] are a bit incremental. While using the DEC framework to provide a regret bound for bilinear class is nice, this does not new/improved regret bound for bilinear class. In Appendix A, the authors discussed the relationship with constrained DEC [16], and mentioned that it would be interesting to explore whether optimistic estimation can be combined with the constrained DEC techniques. To make this paper stronger, one possible direction is to discuss the relationship with constrained DEC in details (either provide some results in this direction, or point out the technical difficulty to the combination).

Questions

What is the general picture of the lower bound side of optimistic DEC? Am I correct that oDEC is always smaller than DEC, and DEC serves as a lower bound, so it is not meaningful to discuss oDEC as a lower bound?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

na

Reviewer kpYc2023-08-15

Thank you for the response! I really appreciate the clarification. Perhaps it would be useful to include (at least a version) of this text in the final version of the paper. I especially wonder about the nature of the gap in the analysis of Agarwal and Zhang --- has this issue been publicly known? If so, an explicit pointer in the present submission would be appreciated. If not, it could make sense to provide a short description of the issue somewhere in the present submission (of course only after discussing with the original authors to make sure that they are fine with it).

Authorsrebuttal2023-08-15

Thank you for the suggestion! We have been in contact with the authors of Agarwal and Zhang (2022) and they are planning to update their paper with a fix under additional structural assumptions. We will be sure to include a pointer once the updated version is available.

Reviewer nAEr2023-08-17

Thanks for clarification

Thanks to the authors for clarification. I am wondering if the authors can also respond to the question: > In Appendix A, the authors discussed the relationship with constrained DEC [16], and mentioned that it would be interesting to explore whether optimistic estimation can be combined with the constrained DEC techniques. To make this paper stronger, one possible direction is to discuss the relationship with constrained DEC in details (either provide some results in this direction, or point out the technical difficulty to the combination). I agree that oDEC and constrained DEC are two orthogonal directions. The question is whether they can be naturally combined. If they can simply be combined, maybe these can be added to this paper. If not, some thoughtful discussions in this paper will be helpful.

Authorsrebuttal2023-08-20

Thank you for the interest! Recall that Foster et al. (2023) give variants of the constrained DEC for the PAC setting and the regret setting. For the PAC setting, we are optimistic that our techniques can be combined with those of Foster et al. (2023) to derive guarantees based on an optimistic variant of the constrained DEC (however, as shown in Foster et al. (2023), the constrained and offset DEC for PAC coincide up to lower-order terms, so the value of such an extension is unclear). For the regret setting, the algorithms in Foster et al. (2023) are quite tailored to hellinger distance, and make heavy use of the fact that it satisfies the triangle inequality and symmetry. Adapting their techniques to general divergences, even without the use of optimistic estimation, appears to be quite non-trivial, and is likely out of scope for this paper. Nonetheless, we are happy to include additional discussion around these issues in the final version of the paper.

Reviewer ooU72023-08-18

Response

Thank you for the helpful response! > it is straightforward to show that our lower bound construction also applies to classical posterior sampling in the Bayesian framework with a well-specified prior. We are happy to include this result if it is of interest to the reviewer. It may be worth mentioning in passing, but I see now it is clear from the original proof.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC