Achieving Constant Regret in Linear Markov Decision Processes

We study the constant regret guarantees in reinforcement learning (RL). Our objective is to design an algorithm that incurs only finite regret over infinite episodes with high probability. We introduce an algorithm, Cert-LSVI-UCB, for misspecified linear Markov decision processes (MDPs) where both the transition kernel and the reward function can be approximated by some linear function up to misspecification level $\zeta$. At the core of Cert-LSVI-UCB is an innovative \method, which facilitates a fine-grained concentration analysis for multi-phase value-targeted regression, enabling us to establish an instance-dependent regret bound that is constant w.r.t. the number of episodes. Specifically, we demonstrate that for a linear MDP characterized by a minimal suboptimality gap $\Delta$, Cert-LSVI-UCB has a cumulative regret of $\tilde{\mathcal{O}}(d^3H^5/\Delta)$ with high probability, provided that the misspecification level $\zeta$ is below $\tilde{\mathcal{O}}(\Delta / (\sqrt{d}H^2))$. Here $d$ is the dimension of the feature space and $H$ is the horizon. Remarkably, this regret bound is independent of the number of episodes $K$. To the best of our knowledge, Cert-LSVI-UCB is the first algorithm to achieve a constant, instance-dependent, high-probability regret bound in RL with linear function approximation without relying on prior distribution assumptions.

Paper

References (37)

Scroll for more · 25 remaining

Similar papers

Peer review

Reviewer YEKy4/10 · confidence 3/52024-07-03

Summary

The paper studies constant regret guarantees for linear MDPs. The main result is the first algorithm with this guarantee. The algorithm is a modification of LSVI-UCB with a careful quantization technique and some form of action elimination. This improves upon the regret of previous works by a factor of $\log K$ where $K$ is the number of episodes. Unlike previous works, the algorithm is robust to some degree of model mispecification and does not need to know it in advance.

Strengths

1. The method is robust to some model mispecification and does not need to know it in advance. 2. The regret guarantee is tighter compared to previous works. 3. The algorithm is computationally efficient in the sense that it runs in polynomial time in the problem parameters. 4. There are significant technical challenges in the work. In particular the careful quantization required to avoid dependence on $K$ and the refined analysis of the regret bound by the suboptimality gaps.

Weaknesses

1. While it's challenging to achieve regret that is independent of $K$, I do not see this as a significant improvement over the logarithmic dependence. This is especially true since it is not clear whether the correct polynomial dependence on the remaining parameters has been established. Moreover, this improvement is only relevant in the regime of a fixed success probability $\delta$, which is often dependent on $K$, e.g., if bounding the expected regret. 2. The improvement to the regret comes at a significant increase to the algorithm's complexity. It's not clear whether this is necessary. For example, are you certain that the quantization cannot be replaced by a more careful analysis of the value cover? Why do you need both bonuses and action elimination and sample rejection? The runtime complexity also seems higher (though this is a minor concern). If you think these modifications are indeed necessary, perhaps you could demonstrate this with experiments. 3. While I am not familiar with better results, the dependence on the worst case gap weakens the result. Do you think it is possible to achieve a better dependence? e.g. on the average inverse gap with respect to the optimal policy. 4. The writing is not very consistent with some places having phrasing and grammatical issues (e.g., quantification should be quantization and many more). 5. The algorithm needs to be better explained. While it seems the authors made significant efforts in this regard, many parts of the algorithm remain rather unclear. For example, the bonuses are not immediately apparent. As I understand it, you use a quantization of the standard bonus. It would be much easier for the reader if this is made explicit in the algorithm and explained. Other examples are the action elimination and sample rejection whose purpose is unclear.

Questions

See above.

Rating

4

Confidence

3

Soundness

2

Presentation

2

Contribution

2

Limitations

N/a

Authorsrebuttal2024-08-14

Thank you for your reply. We addressed your questions and concerns as follows. **Q1. When considering an infinite time horizon, it seems unlikely that it would not scale with the horizon as this means we can incur linear regret with constant probability.** **A1.** We would like to emphasize that the constant failure probability (e.g., $\delta = 10^{-6}$) is acceptable in practice since it is highly unlikely to enter the “bad events” in a realistic RL task. Moreover, the high-probability bounds are well studied and acknowledged in literature (Jin et al., 2020) and cannot be viewed as some “intermediate” result towards the expected result. Therefore, we believe our result is a direct improvement of these works (Jin et al., 2020, Vial et al., 2022) to achieve the constant regret in this regime. Secondly, we would like to highlight that the constant regret bound is important because it reveals a PAC-type (w.p. $1 - \delta$, gap $\Delta$, $\text{Regret} \le O(d^3H^5\Delta^{-1}\log (1 / \delta))$) guarantee of the regret distribution, which cannot be obtained from expected regret. --- **Q2. The correctness of the work.** **A2.** We would like to first acknowledge that it is indeed important to improve on factors $d$ and $H$, where we mentioned a few potential directions in the future work. However, we would like to point out that achieving a constant regret is as important as shaving off other dependence because the factors $d, H$ and $K$ are independent from each other. For the case that $d, H$ is small and the $K$ is large, removing $\log K$ is significant. We would also like to emphasize that the future potentials for shaving off $d, H$ factors does not imply our algorithm is **incorrect** because these results are all upper bounds. We have also double checked the proof before and during the rebuttal, and we are confident that our analysis is correct. --- **Q4. Is there a lower bound showing this is impossible?** **A4.** In tabular setting, Simchowitz et al. 2019, Xu et al., 2021 provided the gap dependent upper bound by (Theorem 1.1, Xu et al., 2021) $$\text{Regret}(K) = \tilde{O}\left( \left( \sum_{(s,a,h) \in \mathcal{S} \times \mathcal{A} \times [H]: \Delta_h(s,a) > 0} \frac{1}{\Delta_h(s,a)} + \frac{|Z_{\text{mul}}|}{\Delta_{\min}} + SA \right) \text{poly}(H)\log K \right), \ \ (1)$$ where $Z_{mul}$ is the set of state-action pairs (s, a)’s satisfying a is an non-unique optimal action for s. The upper bound (1) contains both the averaged inverse gap $\sum \Delta_h^{-1}(s, a)$ and the minimal gap $\Delta_\min$. Only when the optimal action for each state is unique (i.e., $Z_{mul} = 0$), (1) can be reduced to $$\text{Regret}(K) = \tilde{O}\left( \left( \sum_{(s,a,h) \in \mathcal{S} \times \mathcal{A} \times [H]: \Delta_h(s,a) > 0} \frac{1}{\Delta_h(s,a)} + SA \right) \text{poly}(H)\log K \right),$$ In the general case where the optimal action is not necessary unique, we would like to point out that Xu et al., 2021 proved a **lower bound** (Theorem 1.3) suggesting **no algorithm** `ALG` can achieve a regret such that for all MDPs $M$ and $K$ approaches infinity, $$\text{Regret}(K, M, \text{ALG}) = O\left( \left( \sum_{(s,a,h) \in \mathcal{S} \times \mathcal{A} \times [H]: \Delta_h(s,a) > 0} \frac{1}{\Delta_h(s,a)}\right)\log K \right), \ \ (3)$$ As discussed in Xu et al,. 2021, this suggests that it is **not possible** to achieve a regret bound that **solely** depends on the sum of the inverse of the gaps, even in the tabular MDP setting. Xu et al,. 2021 also mentioned that this is a separation between RL and contextual bandits since such a ‘averaged inverse gap’ can be achieved in contextual bandits (Bubeck et al, 2012, Lattimore et al., 2020) Therefore, in linear MDPs, such a lower bound presented as (3) also suggests that it is impossible to achieve a gap-dependent bound that is solely depend on the average inverse gap $\sum \Delta_h^{-1}$ since any tabular MDP can be translated into a linear MDP (Example 2.1, Jin et al. 2020). In short, given the results from literature, we believe that getting such a bound solely depends on the average inverse gap without other dependencies like minimal sub-optimal gap $\Delta_\min$ or $SA$ is impossible in linear MDPs. --- - Vial, Daniel, et al. "Improved algorithms for misspecified linear markov decision processes." AISTATS, 2022 - Jin, Chi, et al. "Provably efficient reinforcement learning with linear function approximation." COLT, 2020 - Xu, Haike et al. "Fine-grained gap-dependent bounds for tabular mdps via adaptive multi-step bootstrap." COLT, 2021 - Simchowitz, Max et al. "Non-asymptotic gap-dependent regret bounds for tabular mdps." NeurIPS, 2019 - Bubeck, Sébastien et al. "Regret analysis of stochastic and nonstochastic multi-armed bandit problems." Foundations and Trends® in Machine Learning 5.1 (2012): 1-122. - Lattimore, Tor et al. Bandit algorithms. Cambridge University Press, 2020

Reviewer 33Lb5/10 · confidence 3/52024-07-12

Summary

This paper proposed a constant regret learning algorithm for linear MDPs with approximation error (misspecification). Specifically, it proved an instance dependent regret that is independent of the number of episodes. In addition, the algorithm does not require prior knowledge of misspecification level or suboptimality gap.

Strengths

1. The background of the problem is stated clearly with well summarized review of the literature. 2. There is a comprehensive comparison of the new constant regret results with existing works.

Weaknesses

1. The proposed algorithm and analysis follow the algorithm in Vial et al. (2022) closely. The main differences seem to be the layer-dependent quantification in Algorithm 1 and the exit condition 3 in Algorithm 2. It would be better to discuss more clearly how these two modifications contribute to getting rid of the log(K) dependence. 2. Some of notations are used without explanation. For example, $\lambda$ in algorithm 1, $\gamma_l$ in Algorithm 2 and Theorem 5.1. Also, how the certified estimator is defined in Algorithm 2 is not clear. 3. Some analysis and results require more clarification: (a) The $\delta$ condition used in Theorem 5.1 does not match the one used in Lemma C12. (b) It is not very clear how the key results in getting rid of log(k) (Lemma C14 line 765) is derived using $x\leq a+\sqrt{bx}$.

Questions

1. The new algorithm does not require UniSOFT assumption made by Papini et al. (2021a). However, in Thm 5 of that paper, they showed that UniSOFT is a necessary condition for constant regret. Can you explain how this does not apply to the current setup? 2. It is not very intuitive to understand the constant regret with no dependency on the number of episodes. Are there any simple examples that can run the algorithm empirically to verify this behavior?

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

There is no potential negative societal impact. The authors addressed the limitations in Section 7 such as the tightness of the bound and uniform misspecification assumption.

Reviewer 8jf26/10 · confidence 3/52024-07-16

Summary

This paper gives a constant regret (total regret independent of the number of episodes $K$) algorithm for online reinforcement learning of linear MDPs. The environment is considered to be a $\zeta$-approxinate linear MDP as in (Jin et al 2019), with an assumption that the misspecification level ($\zeta$) is not too large. The algorithm is a more sophisticated variant of the original LSVI-UCB, which is a further improvement of earlier work that improves the regret of LSVI-UCB (but does not achieve constant regret). The regret is parametrized by the instance-dependent minimal suboptimality gap $\Delta := \min_{h,s,a} \\{V_h^\ast(s) - Q_h^*(s,a) : V_h^\ast(s) - Q_h^*(s,a) \neq 0\\}$, which also appears in earlier instance dependent regret bounds for linear MDPs (and also in general episodic MDPs). Their algorithm, Cert-LSVI-UCB, has a regret bound of $\tilde{O}(d^3 H^5 \Delta^{-1})$. The dependence on $\Delta$ is optimal, whereas the dependence on $d$ and $H$ may be suboptimal. This paper is also an improvement on the previous work that provides "constant regret" bounds for linear MDPs (Papini et al 2021) in that it does not require the "UniSoft" assumption on the feature mapping (only requires the standard assumptions of misspecified linear MDPs as in Jin et al 2019, plus that the misspecification level $\zeta \leq \tilde{O}(\Delta/\sqrt{d}H^2)$).

Strengths

* The algorithm and its analysis gives a substantial improvement over earlier algorithms with instance-dependent bounds for linear MDPs, in that it is the first algorithm with a constant regret bound that requires minimal assumptions, even for misspecified linear MDPs. * The algorithm is cleanly based on earlier linear MDP algorithms starting from LSVI-UCB, applied in more sophisticated ways (multiple regression phases per episode etc), and the improvements in the algorithm and challenges in the analysis have been highlighted in the main paper itself (especially, in avoiding the dependency of the regret on the number of phases).

Weaknesses

* The contribution of the paper is entirely theoretical, but most of the proof is in the appendix (which is unavoidable). * Some experimental results (even on synthetic data) would definitely enhance the paper.

Questions

NA

Rating

6

Confidence

3

Soundness

3

Presentation

4

Contribution

3

Limitations

* The limitations have been adequately addressed and there is no negative societal impact.

Reviewer jLch6/10 · confidence 2/52024-07-17

Summary

The paper presents improved regret bounds for Linear MDPs when the suboptimality gap \Delta is known. Most crucially (a bit surprisingly), the regret bounds are constant in a number of episodes K

Strengths

The contributions are broadly theoretical, and the removal of dependence on the number of episodes K seems to be a good contribution to the academic understanding of the gap-dependent (instance-dependent)learning bounds for reinforcement learning. The paper provides a good coverage of related works in this area and describes the setting well.

Weaknesses

1) b_h^\pi seems to be undefined in proposition 3.2 2) \kapp_\ell in Algorithm 1, line 8, as claimed by the author, helps to get rid of log(K). Can you explain more about how it helps to remove log(K)? is this the reason why the regret bounds do not have log(K) dependence? 3)I would urge the authors to provide a full expression of regret without O notation, including the log factors either in the main text or in the appendix (I could not find a full expression for the result in theorem 5.1). Specifically, it is surprising, perhaps I am missing something, that the regret is O(1) when Delta = 1/(d^3H^5) when K=1. I suspect some lower-order terms are hidden, or maybe the result is only valid for K \geq something. The asymptotic dependence can be right still.

Questions

Please see above, I am willing to engage during the rebuttal phase.

Rating

6

Confidence

2

Soundness

3

Presentation

2

Contribution

3

Limitations

NA

Reviewer jLch2024-08-11

Thank you for your detailed response. I am broadly satisfied. My only recommendation (for an updated version) is that authors focus a bit on explaining the key difference that allows them to have regret and not depend on $K$, as they have mentioned some of the points in the rebuttal here.

Authorsrebuttal2024-08-11

Thank you!

Thank you for your continuous support for our paper. As you suggested, we will expand the discussions on the technical challenges in Section 6 in the revision.

Reviewer YEKy2024-08-13

response

Thank you for the detailed response. [A1]: I am fully aware that "shaving" this log factor achieves independence of the time horizon. However, this is only with a fixed probability $1-\delta$. When considering an infinite time horizon, it seems unlikely that $\delta$ would not scale with the horizon as this means we can incur linear regret with constant probability. [A2]: The fact that previous works achieve the same dependence on $d, H, \Delta^{-1}$ **does not imply** that this is the correct dependence. Many works indeed focus on achieving improved dependence on $K$ as long as other parameters scale polynomially. However, I find this sensible only when the dependence on $K$ is polynomial. When it is logarithmic, I find it very likely that shaving a factor of $d$ or $H$ would be more significant than shaving a $\log K$ factor. [A3+5]: Thank you for the explanations. While I'm not fully convinced of the necessity of each part, I get the general picture. Regardless, this is a secondary concern. [A4]: Regarding gaps, is there a lower bound showing this is impossible in general? I am aware that this concern is not unique to this work but it is a weakness nonetheless.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC