Provably Safe Reinforcement Learning with Step-wise Violation Constraints

In this paper, we investigate a novel safe reinforcement learning problem with step-wise violation constraints. Our problem differs from existing works in that we consider stricter step-wise violation constraints and do not assume the existence of safe actions, making our formulation more suitable for safety-critical applications which need to ensure safety in all decision steps and may not always possess safe actions, e.g., robot control and autonomous driving. We propose a novel algorithm SUCBVI, which guarantees $\widetilde{O}(\sqrt{ST})$ step-wise violation and $\widetilde{O}(\sqrt{H^3SAT})$ regret. Lower bounds are provided to validate the optimality in both violation and regret performance with respect to $S$ and $T$. Moreover, we further study a novel safe reward-free exploration problem with step-wise violation constraints. For this problem, we design an $(\varepsilon,δ)$-PAC algorithm SRF-UCRL, which achieves nearly state-of-the-art sample complexity $\widetilde{O}((\frac{S^2AH^2}{\varepsilon}+\frac{H^4SA}{\varepsilon^2})(\log(\frac{1}δ)+S))$, and guarantees $\widetilde{O}(\sqrt{ST})$ violation during the exploration. The experimental results demonstrate the superiority of our algorithms in safety performance, and corroborate our theoretical results.

Paper

References (45)

Scroll for more · 33 remaining

Similar papers

Peer review

Reviewer 31rE7/10 · confidence 4/52023-06-24

Summary

This paper studies safe RL with step-wise violation constraints, different from the popular CMDP with an additive expectation cost constraint. The step-wise violation constraint is more suitable for safety-critical systems. The authors propose an algorithm that provides violation and regret bound. They then further develops an algorithm to learn a near-optimal safe policy and show its effectiveness in the experiments.

Strengths

1) This paper studies an important problem. The paper is well-written and easy to follow. The RL with step-wise violation constraint, as a formulation, is novel and more general than the popular CMDP with an additive expectation constraint. 2) The proposed approach looks correct and sound to me although I didn't check every proof. 3) The authors provide theoretical analyses of violation and regret bound. 4) The proposed safe RL algorithm achieves better performance than the existing baselines.

Weaknesses

1. This paper may miss some reference for safe RL with step-wise violations such as a) Wang, Y., Zhan, S. S., Jiao, R., Wang, Z., Jin, W., Yang, Z., ... & Zhu, Q. (2022). Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environments. arXiv preprint arXiv:2209.15090. The authors have to at least discuss this paper in this work as the timepoint-level "hard chance constraint" looks very similar to the step-wise violations proposed by the authors. 2. In the MDP model, it is not clear to me whether the paper solves safe RL with continuous/discrete deterministic/stochastic systems or environments. The transition set \Delta_h(s, a) could be an infinite set when considering continuous state space. 3. Essentially, my understanding of this paper is that they are dealing with discrete action and state space, and because of this, the RL agent can infer which state and/or future state is unsafe to visit by building the A_safe set via dynamical programming. which is the core of the proposed algorithms.

Questions

I am using a Windows laptop and I cannot open the full_paper file (format) in the supplementary folder, so I cannot see the limitations of the approach.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

I would like to authors to clarify the limitations of their formulation and approach in the response.

Reviewer ZGMh5/10 · confidence 3/52023-07-03

Summary

This paper formulates and studies a strict step-wise violation constraint reinforcement learning problem, where a non-negative state-dependent cost is cumulated at each step. They show lower bounds for regret and safety violations. A model-based algorithm that matches the lower bound is provided, while another O(1) gap-dependent violation is provided too. They also study the reward-free problem and provide an algorithm that has a sample complexity with a leading term matching the SOTA of a no-constraint case.

Strengths

The step-wise violation constraint problem is meaningful. This paper is technically sound. The proofs are correct as far as I can tell. The empirical result looks good.

Weaknesses

1. The idea of the unsafe set can only apply to state-dependent safety problems. I doubt this technique can apply to a (state, action)-dependent safety problem. Additionally, it is weird that the reward is (state, action)-dependent, but the cost is state-dependent only. 2. It is weird that the reward-free can give cost feedback. 3. Definition 6.1 is confusing. I guess from your proof that \pi in (4) means all feasible policies defined by \Delta and U? 4. line 545, '<' should be '>'?

Questions

See weakness.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

3 good

Limitations

Part of the limitations are shown. The authors don't discuss the limitations of state-dependent cost as well as the weird reward-free setting.

Reviewer ryzU6/10 · confidence 3/52023-07-04

Summary

The authors propose a new formulation for the safe RL problem Safe-RL-SW, whose violation constraints are step-wise, different from the existing work. A model-based general algorithmic framework SUCBVI is proposed and the theoretical guarantees on the upper bound and lower bound of its regret are provided. The authors also propose a similar reward-free safe RL problem with step-wise violation constraints (Safe-RFE-SW) along with an algorithm SRF-UCRL, the theoretical upper bound on whose regret is provided. It is claimed to be the first result on step-wise violation constraints in the RFE setting. Experiments are provided in support of the theoretical results.

Strengths

The authors propose new problem formulations for safe RL as well as algorithms under these formulations. Theoretical results are provided for the proposed algorithms. Overall, the new problem formulations seem original and the results seem significant.

Weaknesses

How the results in this work compare with the existing work is somewhat unclear, despite some discussion in the Related Work Section. In Section 2, the authors mention two closely-related work Amani et al. (2021) and Shi et al. (2023) that also consider step-wise violation and state that more detailed comparisons can be found in Appendix B, but I did not find any comparison with these two works in Appendix B. I strongly suggest that the authors compare with all related works and clearly state how this work differentiates from them in the revision. For example, do all CMDP works define violation with $C'(K)$ in Line 136? Can you compare with the CMDP works that do not use expected violation like $C'(K)$, if there is any? As for Amani et al. (2021) and Shi et al. (2023), please explain whether there is any difference in your formulations and how your results compare with theirs, if your setting is comparable with theirs.

Questions

I hope the authors can provide clarification for a few questions: 1. What is the role of $n$ in Theorem 5.2? Why considers $M_1, \cdots, M_n$ in this theorem instead of just focusing on the one MDP that establishes the lower bound? 2. Do all CMDP works define violation with $C'(K)$ in Line 136? Is there any CMDP work that do not use $C'(K)$ and have a violation definition that is more comparable to the step-wise violation in this work? 3. In the plot of "Step-wise Violation in Safe-RL-SW" of Figure 1, the cumulative step-wise violation curve for SUCBVI seems to grow much more slowly than $\sqrt{T}$. However, shouldn't we expect something $\Omega(\sqrt{ST})$, especially given your lower bounds in Section 5?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The limitation is currently discussed in Appendix A. I encourage the authors to move part of it in the conclusion so that it is more conspicuous.

Reviewer 83sH5/10 · confidence 4/52023-07-21

Summary

This paper considers online RL problem for an MDP with stage-wise constraints. The stage-wise constraints basically specify the set of unsafe states which must be avoided at all times. While there is a lot of recent work on CMDPs, this formulation, which is actually more relevant is less studied. The authors propose a variant of UCBVI algorithm, and show that it achieves O(\sqrt{SATH^3}) regret which is order optimal except in H. It also achieves sublinear constraint violation, which is independent of T when it is gap-dependent. They also present a reward-free exploration algorithms with sublinear regret and constraint violation.

Strengths

This seems one of the few results on Online RL for MDP with stage-wise constraints. Attempt at numerical evaluation is commendable.

Weaknesses

Unfortunately, the results are weaker than what have been achieved recently for CMDP problems: some of the formulations only consider clipped violation functions, and still achieve bounded constraint violation. The results for reward-free learning are actually much weaker as it only achieves sublinear constraint violation. Furthermore, novelty in the algorithm is limited based as it is on UCBVI, a well-trodden path. Recent work on online learning for CMDPs has several interesting algorithmic ideas which the authors could potentially use to propose algorithms for sharper results.

Questions

Could you use some of the recent work of Lei Ying and Dileep Kalathil to design algorithms that achieve even sharper results. I believe it is possible without too much extra work.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

N/A.

Reviewer RuZi6/10 · confidence 4/52023-07-26

Summary

The paper studies an episodic constrained reinforcement learning problem with step-wise constraints on states. The authors first extend the classical UCB-VI to step-wise constraints and prove the sub-linear optimality gap and step-wise constraint violation. A lower bound is also provided to show optimal dependence on the state space size and the number of episodes. Second, the authors present a reward-free algorithm that takes nearly optimal sample complexity and subliner step-wise constraint violation during exploration. Some numerical experiments are provided to show the effectiveness of the proposed algorithms.

Strengths

**originality** - The problem formulation follows the existing tabular constrained MDP with instantaneous constraints. The safety constraint used in this work is an inequality of a function of state. The learning objective is to minimize regret and step-wise violation. The instantaneous constraint is also studied in other papers, e.g., Safe Policy Optimization with Local Generalized Linear Function Approximations and Provable Safe Reinforcement Learning with Binary Feedback. - The only problem assumption is the existence of an initial state that is not in the unsafe set. This assumption seems to be the weakest in the literature. Since the feasible policy has to satisfy the constraint at every step, transition dynamics can't guarantee the feasibility of the next state no matter the actions taken. So, it might not be intuitive that the feasibility has nothing to do with transition dynamics. - In the first algorithm, the authors extend the existing UCBVI by incorporating the estimation of safe and unsafe sets, and prove sublinear regret and step-wise violation. It is useful if the authors could discuss how this work differs from previous applications of UCBVI to constrained MDPs, e.g., (Amani et al., 2021), which studies linear function approximation and offers zero violation. - In the second algorithm, the authors generalize the existing reward-free algorithm to step-wise constraints, and prove nearly-optimal sample complexity. **quality** - Most statements are clarified by remarks or proofs. - It is useful if the authors can make a more technical comparison with existing works since this paper is more theoretical. - In experiments, it might be not very fair to compare other methods that study value-based constraints. - It is less discussed about the weaknesses of the proposed method. It is useful to discuss if the proposed methods handle function approximation and provide zero constraint violation. **clarity** - The main results are delivered clearly, and the paper is organized well. **significance** - It is important to develop RL algorithms that can learn step-wise constraints. The authors have extended two known unconstrained RL algorithms to tabular constrained MDPs with step-wise constraints. However, step-wise constraints have been tackled in several other works, which warrant detailed comparison to show the significance of this paper. - The provided theoretical analysis and guarantees are more expected. It is less discussed new challenges of extending the existing unconstrained analysis. - The provided experiments do not seem to reflect the performance of the proposed algorithms. There is no variance in learning curves. Since all algorithms directly converge, exploration does seem to be an issue for convergence, which does not explains the adaptivity of online RL algorithms.

Weaknesses

- It is useful to make a more detailed comparison with existing works on constrained MDP with instantaneous constraints, especially problem assumptions. Discussing some related applications is also useful. - The technical contributions can be better explained by comparing the constrained and unconstrained ones. It is useful to highlight the main technical challenges. - It is useful to discuss extending to large problems using function approximation and providing better violation guarantees. - For experiments, it is important to make a fair comparison with existing works on constrained MDPs with instantaneous constraints. It is also important to experiment algorithmms in a correct way by showing convergence and exploration randomness.

Questions

Some questions are stated in Strengths and Questions. Here are some other questions. - How the confidence bonus in Algorithm 1 is constructed? - Is there a lower bound for Algorithm 2 that shows the hardness of step-wise constraints? - To see how the proposed algorithms deal with step-wise constraints, can the authors plot the instantaneous constraint violation?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

3 good

Limitations

Yes.

Reviewer 31rE2023-08-12

Thanks for the rebuttals. In your future revision, please make sure to clarify that this paper considers a tabular MDP in the problem formulation.

Reviewer ryzU2023-08-13

I appreciate the rebuttal from the authors. When you revise the paper, please make sure to discuss the comparison with other works and any implications of the proposed problem formulation sufficiently in the main text. I have raised my score.

Authorsrebuttal2023-08-16

Thank you very much for raising the score! We will discuss the comparison with other works and the implications of the formulation sufficiently in our revision.

Reviewer RuZi2023-08-14

Thank you for the clarification and additional experiments. By viewing this, I have increased my score.

Authorsrebuttal2023-08-16

Thank you very much for raising the score!

Reviewer ZGMh2023-08-16

Thank you for the rebuttals. I still have some concerns after reading the rebuttal and I will keep my rating. 1. If the framework is extended to (state, action)-dependent cost as defined by authors, then it seems no hope to handle the large action case even if function approximation is used since the definition of unsafe states always requires maximize over all actions. 2. I am still not fully convinced by the reasons that the cost feedback can be given in the reward-free setting. In some cases, the reward function can also depends on the environment, which does not change frequently.

Authorsrebuttal2023-08-16

Response to Reviewer ZGMh

Thank you for the response! **1.If the framework is extended to (state, action)-dependent cost as defined by authors, then it seems no hope to handle the large action case even if function approximation is used since the definition of unsafe states always requires to maximize over all actions.** If we extend to (state, action)-dependent cost, it indeed needs to explore all actions to receive the safety feedback of each action. One possible avenue to avoid this exploration is to posit that the cost function exhibits a certain favorable structure, (e.g. $c(s,a)= \theta^T \varphi(s,a)$ for some $\theta \in \mathbb{R}^d$ and known features $\varphi(s,a) \in \mathbb{R}^d$, as in existing works [1,8].) and then estimate the cost for all $(s,a)$ exploiting the structure without exploring all actions. This is an interesting future direction. **2. I am still not fully convinced by the reasons that the cost feedback can be given in the reward-free setting. In some cases, the reward function can also depend on the environment, which does not change frequently.** The RFE setting is motivated by the fact that in practice, the reward function can be difficult to specify and change frequently by manual crafting [2,3], or there can be multiple rewards of interest in the same environment [4]. In contrast, the cost function can heavily depend on the environment such as barrier placements [5] and the gesture of the robot [6]. Hence, it can be immutable and knowable. Previous safe RFE work [7] also receives the cost feedback during the exploration phase. Moreover, from a theoretical perspective, access to information about the cost function becomes imperative. Devoid of this information, ensuring safety constraints during the exploration phase becomes an impractical pursuit. We thank the reviewer again for the detailed and valuable comments. Please let us know if you have any further questions. We will be happy to answer them. If you find our response satisfying, we wonder if you could kindly consider raising the score rating of our work? Thank you very much! **References:** [1]. Amani S, Thrampoulidis C, Yang L. Safe reinforcement learning with linear function approximation, ICML, 2021. [2]. Leike J, Krueger D, Everitt T, et al. Scalable agent alignment via reward modeling: a research direction, preprint 2018. [3]. Fu J, Singh A, Ghosh D, et al. Variational inverse control with events: A general framework for data-driven reward definition, NIPS 2018. [4]. Wu J, Braverman V, Yang L. Accommodating picky customers: Regret bound and exploration complexity for multi-objective reinforcement learning, NIPS 2021. [5]. Thananjeyan B, Balakrishna A, Nair S, et al. Recovery rl: Safe reinforcement learning with learned recovery zones, IEEE Robotics and Automation Letters, 2021. [6]. Thomas G, Luo Y, Ma T. Safe reinforcement learning by imagining the near future, NIPS 2021. [7]. Huang R, Yang J, Liang Y. Safe exploration incurs nearly no additional sample complexity for reward-free rl, ICLR 2023. [8]. Wachi et al. Safe policy optimization with local generalized linear function approximations. 2021.

Reviewer ZGMh2023-08-16

Thank you for your reply. Even with the linear structure, it seems that the definition of safe set stills requires to maximize over the entire action space. I understand the requirement of cost information theoretically. However, in practice, this setting doesn't make sense in some cases. For example, there are multiple costs of interest in the wireless network, e.g., bandwidth and expense.

Authorsrebuttal2023-08-17

(a). We agree that we need to compute the maximum value $\max_{a \in A} c(s,a)$. In fact, in the realm of linear MDP, in terms of computation, all previous works need to calculate the maximization over the action space, e.g., [1,2,3]. However, there is no need to actually explore (i.e., actually take) all the actions. (b). In the wireless network example, even if there are multiple costs, e.g., bandwidth and expense, these costs and constraints can be known beforehand. Subsequently, we can treat this as a multi-constraint problem, and use a vector-valued cost function to simultaneously represent multiple cost functions [6]. Moreover, reward serves as a way for us to specify the objectives under different tasks, and can be changed frequently by manual crafting [4,5]. Thus, it becomes crucial to devise an algorithm that effectively learns the underlying model and attains commendable performance across diverse reward scenarios. On the other hand, cost often depends on the environment and is an inherent attribute of the environment, rather than being manually designated. Hence, it is reasonable to receive the cost feedback during the exploration phase. [1]. Jin C, Yang Z, Wang Z, et al. Provably efficient reinforcement learning with linear function approximation, COLT 2020. [2]. Hu P, Chen Y, Huang L. Nearly minimax optimal reinforcement learning with linear function approximation, ICML 2022. [3]. Amani S, Thrampoulidis C, Yang L. Safe reinforcement learning with linear function approximation, ICML, 2021. [4]. Leike J, Krueger D, Everitt T, et al. Scalable agent alignment via reward modeling: a research direction, preprint 2018. [5]. Fu J, Singh A, Ghosh D, et al. Variational inverse control with events: A general framework for data-driven reward definition, NIPS 2018. [6]. Yu T, Tian Y, Zhang J, et al. Provably efficient algorithms for multi-objective competitive rl, ICML 2021.

Authorsrebuttal2023-08-16

Thank you very much for increasing the score! We will clarify the tabular MDP setting in our revision.

Reviewer 83sH2023-08-21

I know the difference between stage-wise and time-averaged constraints. I understand [3], etc. handle constraint violation in expectation. Since your requirement it stage-wise, it effectively is tantamount to constraining the policy space at each stage so that the constraint is satisfied. I think it should be possible to guarantee, zero or bounded constraint violation in your space as well. I will keep my score.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC