Uncertainty-based Offline Variational Bayesian Reinforcement Learning for Robustness under Diverse Data Corruptions

Real-world offline datasets are often subject to data corruptions (such as noise or adversarial attacks) due to sensor failures or malicious attacks. Despite advances in robust offline reinforcement learning (RL), existing methods struggle to learn robust agents under high uncertainty caused by the diverse corrupted data (i.e., corrupted states, actions, rewards, and dynamics), leading to performance degradation in clean environments. To tackle this problem, we propose a novel robust variational Bayesian inference for offline RL (TRACER). It introduces Bayesian inference for the first time to capture the uncertainty via offline data for robustness against all types of data corruptions. Specifically, TRACER first models all corruptions as the uncertainty in the action-value function. Then, to capture such uncertainty, it uses all offline data as the observations to approximate the posterior distribution of the action-value function under a Bayesian inference framework. An appealing feature of TRACER is that it can distinguish corrupted data from clean data using an entropy-based uncertainty measure, since corrupted data often induces higher uncertainty and entropy. Based on the aforementioned measure, TRACER can regulate the loss associated with corrupted data to reduce its influence, thereby enhancing robustness and performance in clean environments. Experiments demonstrate that TRACER significantly outperforms several state-of-the-art approaches across both individual and simultaneous data corruptions.

Paper

Similar papers

Peer review

Reviewer vWB57/10 · confidence 3/52024-06-12

Summary

The authors introduce TRACER, a new Bayesian methodology to capture the uncertainty via offline data for robustness against all types of data corruptions. An appealing feature of TRACER is that it can distinguish corrupted data from clean data using an entropy-based uncertainty measure. Experiments are provided to prove the effectiveness of such a methodology.

Strengths

Very clearly stating the problem, and convincing the reader that's a relevant one. Compelling method to address such a problem.

Weaknesses

The authors missed some important references: [1,2] introduce credal Bayesian deep learning, which is a robust way of performing Bayesian inference via Bayesian Neural Networks (where posterior and predictive distributions are approximated via variational inference) taking into account different types of uncertainty (using so-called credal sets, i.e. closed and convex sets of prior and likelihood probabilities), and quantifying them using entropy-based uncertainty measures. In the future, the authors may also look into [3], which seems like a compelling work for the research venue explored in the paper. [1] https://arxiv.org/abs/2302.09656 [2] https://link.springer.com/chapter/10.1007/978-3-031-57963-9_1#:~:text=In%20their%20seminal%201990%20paper,bound%20to%20hold%20with%20equality [3] https://arxiv.org/abs/2308.14815

Questions

I'd like to politely ask the authors to expand the related work with the above references [1,2]. In addition, would it be fair to say that, on page 3, the expectation that finds $\pi^\star$ is an integral taken with respect to the joint probability given by all the sources of randomness $P_0, \pi(\cdot \mid s_t), \rho(\cdot \mid s_t,a_t), P(\cdot \mid s_t,a_t)$? In equation (1), the closed parenthesis ) should go after $a_0=a$.

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

The limitations are explicitly acknowledged by the authors in the conclusion.

Reviewer wmJN6/10 · confidence 3/52024-06-27

Summary

This paper presents a novel approach called TRACER, aimed at addressing the challenge of learning robust policies from offline datasets that are subject to various data corruptions. The key contribution of this work lies in the integration of Bayesian inference to capture uncertainty within the offline data. TRACER models data corruptions as uncertainty in the action-value function and approximates the posterior distribution of this function. Furthermore, it employs an entropy-based uncertainty measure to distinguish between corrupted and clean data, thereby regulating the loss associated with corrupted data to enhance robustness in clean environments. Experimental results indicate that TRACER outperforms SOTA methods in handling both individual and simultaneous data corruptions.

Strengths

- The studied problem is important, and the paper offers a reasonable solution. - Theoretical guarantees are provided. - The empirical results show considerable improvement over prior work.

Weaknesses

- Though I like the insight from Variational Bayesian inference, my major concern is that the method is quite complicated, incorporating numerous components and designs such as the distributional value function, variational inference, and entropy-weighted loss function. This complexity makes this method less preferable in practice. - Moreover, the results in the main tables (Table 2 and Table 3) appear similar to those of RIQL, raising concerns about the necessity of the included components. - Another issue is the readability and clarity of the writing. For example, Equations 10 and 11 lack necessary explanation and insight. Additionally, the implementation details for $\phi_a, \phi_b, \phi_c$ are missing, leaving readers unsure of how the method works in practice. Furthermore, Equation 10 seems to include $\theta$ as an input unnecessarily, as the right side does not depend on $\theta$. - The analysis of the experiments is not thorough. For instance, the accuracy of the measurement for corrupted data decreases during training in Figure 3, which is not discussed and seems unreasonable.

Questions

- The proposed method is complicated. Can this method be simplified? - The results in the main tables (Table 2 and Table 3) appear similar to those of RIQL, raising concerns about the necessity of the included components. - Another issue is the readability and clarity of the writing discussed above. - Why does the accuracy of the measurement for corrupted data decreases during training in Figure 3? - Moreover, I suggest the authors provide the computational cost comparison with prior work.

Rating

6

Confidence

3

Soundness

3

Presentation

2

Contribution

2

Limitations

This paper acknowledges its limitations; however, I identified an additional concern regarding the complexity of the proposed method.

Reviewer QZTE6/10 · confidence 4/52024-07-07

Summary

The paper introduces TRACER, a robust offline reinforcement learning (RL) algorithm designed to address the challenges posed by data corruptions in offline datasets. The corruptions can be realized in form of states, actions, rewards, and dynamics corruption. The proposed methodology uses Bayesian inference to model and mitigate the impact of corrupted data on the action-value function. By leveraging an entropy-based uncertainty measure, TRACER can distinguish between corrupted and clean data, thereby reducing the influence of the former and enhancing the robustness of the learned policies. The paper provides thorough evaluation on MujoCo and Carla offline RL datasets.

Strengths

1. The paper proposes an interesting method of down weighting the contribution corrupted data while learning action value function from the offline dataset using entropy as an uncertainty measure. 2. The paper uses a Bayesian approach to learn the action value function from the data which aids learning from uncorrupted elements in the dataset. 3. The authors provide extensive experimental validation across multiple tasks and corruption types, showing mostly consistent performance improvements over other methods. The paper also reports ablation studies of using entropy showing potential improvement from the proposed methodology.

Weaknesses

1. It is not clear how the final policy is extracted after learning $D_{\theta}$. 2. The paper relies on many hyperparameters. What kind of kernel is used in the gaussian distribution learning in Eq 10? Is there a upper bound on what amount of corruption the method will be able to handle? 3. While the authors consider individual corruption of one element at a time data collected from in a real world system can have simultaneous corruptions. For example corrupted action due to adversarial noise will lead the system to different state causing simultaneous corruptions. 4. Notational clarity: The paper introduces a lot of notations which is difficult to keep track of. Specially in the following contexts : a : The paper introduces LQ (θi), LV(ψ), Lπ(ϕ) in Eq 4, 5 and 6 but does not discuss how they relate to Dθ. Is the θ used for learning Q function and D same? b : The paper introduces a new notation qπ in Eq 18 called value distribution. Is this the same as action value distribution? RIQL already proves the robustness of IQL policy with respect of data corruption. What is the objective of Theorem A.3? It will be helpful to add a discussion on the theoretical analysis. Also Providing a table summarizing the notations in appendix will enhance readability of the text.

Questions

1) How relevant is the reward corruption given offline RL already is robust to noise in the reward function as it mimics the actions for datasets with length bias? Please refer to ref 1 2) The authors aim to leverage all elements in the dataset as observations. Do the authors assume that complete traces of the system are available for which the offline policy is being learned? In situations of dynamic corruption these traces might be different than the system. For example s_t in the corrupted dataset may not be observed in the real system. How are such cases handled? 3) As the method learns action-value function D from the data while reducing the influence of corrupted data. What amount of clean data needs to be present in the dataset for this method to work? 4) What is the reason for not using projected gradient descent for the reward corruption? Also could the author provide some understanding of why the method outperforms RIQL under certain settings and why it does not in some cases? 5) Please also respond to the weaknesses. 1. Li, Anqi, et al. "Survival instinct in offline reinforcement learning." Advances in neural information processing systems 36 (2024).

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

Limitations: The authors discuss the limitation in terms of handling realistic noises in conclusion. However, a limitation is also on the dependence on clean observations present in the dataset when some elements are corrupted. The authors do not discuss any negative sociatal impact of their work.

Authorsrebuttal2024-08-09

Response to Additional Comments

Thank you for your valuable comments. We respond to each of your comments as follows. **Q16**. (1) How does $\tilde{\pi}_{\text{IQL}}$ help to prove the upper bound for TRACER? (2) How do you equate $Z^\pi(s)$ with $D^\pi (s,a)$ in steps 24, 25, and 26 in Theorem A.3? (3) What is $D^\pi (s,a)$ as previously D is defined as D(s, a, r)? - **A16**. - For (1), as TRACER directly applies the weighted imitation learning technique from IQL to learn the policy, we can use $\tilde{\pi}_{\text{IQL}}$ as the policy learned by TRACER under data corruptions, akin to RIQL. In Theorem A.3 of the Appendix, the major difference between TRACER and IQL/RIQL is that TRACER uses the action-value and value distributions rather than the action-value and value functions in IQL/RIQL. Therefore, we further prove an upper bound on the difference in value distributions of TRACER to show its robustness. - For (2), we apologize for the missing reference in steps 24, 25, and 26 of Theorem A.3 (see Lemma 6.1 of [1]). We follow [1] to provide the detailed derivation below. - For any $\pi'$ and $\pi$, we have $$\begin{align} Z^{\tilde{\pi}}(s) &= \sum_{t=0}^\infty \gamma^t \mathbb{E}\_{(S_t,A_t)\sim P,\tilde{\pi}} \left[R(S_t, A_t) | S_0 = s\right]\\\\ &= \sum_{t=0}^\infty \gamma^t \mathbb{E}_{(S_t,A_t)\sim P,\tilde{\pi}} \left[R(S_t, A_t)+Z^{\pi}(S_t)-Z^{\pi}(S_t) | S_0 = s\right]\\\\ &= \sum\_{t=0}^\infty \gamma^t \mathbb{E}\_{(S\_t,A\_t,S\_{t+1}) \sim P,\tilde{\pi}} \left[R(S\_t, A\_t)+\gamma Z^{\pi}(S\_{t+1})-Z^{\pi}(S\_t) | S\_0 = s\right] + Z^{\pi}(s)\\\\ &= Z^{\pi}(s)+\sum\_{t=0}^\infty \gamma^t \mathbb{E}\_{(S_t,A_t)\sim P,\tilde{\pi}} \left[D^{\pi}(S\_t,A\_t,R\_t)-Z^{\pi}(S\_t) | S\_0 = s\right]\\\\ &= Z^{\pi}(s)+ \frac{1}{1-\gamma} \mathbb{E}\_{(s,a)\sim d^{\tilde{\pi}},\tilde{\pi}} \left[D^{\pi}(s,a,r)-Z^{\pi}(s)\right]. \end{align}$$ Thus, we can derive the step 25 from the step 24. - For (3), we apologize for any confusion caused by the notations. We will replace the notation $D^\pi (s,a)$ with $D^\pi (s,a,r)$ in Theorem A.3 of our revision. [1] Approximately optimal approximate reinforcement learning. **Q17**. (1) What is the value of $c$ we use in experiments? (2) What is the effect on performance for varying $c$? - **A17**. - For (1): - For each experiment with a random seed under individual corruptions in Tables 2 and 3 of our main text, we randomly select $c\\% = 30\\%$ of transitions from the offline dataset. Within these selected transitions, we replace one element per transition with corrupted element. - In Table 1 with simultaneous corruptions, we also apply $c\\% = 30\\%$ but extend the corruption process across all four elements of the offline dataset. Specifically, we randomly select $30\\%$ of transitions and corrupt one element in each selected transition. Then, we repeat this step four times until all elements are corrupted. Therefore, approximately $76.0\\%$ of data in the offline dataset is corrupted, calculated as $1 - (1 - c)^4$. In Table G4 of our attached PDF of the global response, we evaluate TRACER using different $c \\%$, including $10\\%, 20\\%, 30\\%, 40\\%,$ and $50\\%$. These rates correspond to approximately $34.4\\%, 59.0\\%, 76.0\\%, 87.0\\%$, and $93.8\\%$ of the data being corrupted. - For (2): - Results in Table G4 show that while TRACER is robust to simultaneous corruptions and significantly outperforms RIQL, its performance depends on the extent of corrupted data it encounters, degrading with increased data corruptions. We hope our responses adequately address your concerns. If you have further concerns, please let us know and we will continue actively responding to your comments, enhancing our submission. We would deeply appreciate it if you could raise your score based on these revisions.

Reviewer QZTE2024-08-10

Response

Thank you for addressing my concerns. I have raised my score. I would urge the authors to include the notation description, additional results and clear explanation of Theorem A3 in the revised version.

Authorsrebuttal2024-08-10

Thanks for your kind support and for helping us improve the paper!

Dear Reviewer QZTE, Thanks for your kind support and for helping us improve the paper! We are committed to including all these additional results, notation descriptions, and detailed explanations and will incorporate your suggestions in the final version, if accepted. Thank you again for your valuable comments and guidance. Best, Authors

Authorsrebuttal2024-08-11

Thanks again for your continued support!

Dear Reviewer QZTE, Thanks again for your continued support and the further improvements you have suggested for our manuscript. We sincerely appreciate your insightful feedback, which has significantly helped us in refining the explanations and enhancing the theoretical derivations. We remain committed to incorporating all your valuable suggestions into the final version, if accepted. We are deeply grateful for your thoughtful comments and for the confidence you have shown in our work by raising your score. Best, Authors

Reviewer V2Jx6/10 · confidence 4/52024-07-11

Summary

This paper seeks to conduct reinforcement learning from corrupted offline data. More specifically, they propose the TRACER algorithm, which uses bayesian inference to calculate the uncertainty in estimating the action-value function. The authors conduct experiments with diverse corruptions on CARLA and Mujoco environments.

Strengths

- This paper considers an important problem. - Their proposed algorithm is interesting and performs well.

Weaknesses

- The contribution compared to other baselines (RIQL) is not really clear. In the introduction the authors claim that their method can handle simultaneous corruptions and RIQL cannot. However no experiments are done in this setting. - The experimental settings are fairly limited: only three Mujoco environments are considered and one carla experiment. - Different perturbations levels are not shown. - The authors do not discuss the computational cost of their method.

Questions

- Can you clarify the contribution of TRACER compared to RIQL? It seems like it is an orthogonal approach, but the two algorithms are not really compared in detail. - Can you show the hyperparameter tuning results? Is TRACER stable under different hyperparameter settings? How does it compare to RIQL in this regard? - How does the computational cost of TRACER compare to the other baselines?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors do discuss their method's limitations.

Authorsrebuttal2024-08-08

Table of Notations used in Appendix for Q8.

For Q8, we provide the detailed table summarizing the notations used in our Appendix. See Table G8 as follows. Table G8. Notations in our Appendix. | Notations used in our Appendix | Descriptions | | ------------------------------ | ------------------------------------------------------------ | | $\mathbb{\zeta}$ | Cumulative corruption level. | | $\mathbb{\zeta}_i$ | Metric quantifying the corruption level of rewards and next states (transition dynamics). | | $\mathbb{\zeta}_i^{'}$ | Metric quantifying the corruption level of states and actions. | | $\pi_{b}(\cdot\|s)$ | The behavior policy that is used to collect clean data. | | $\pi_{\mathcal{B}}(\cdot\|s)$ | The behavior policy that is used to collect corrupted data. | | $\pi_{E}(\cdot\|s)$ | The policy that we want to learn under clean data. | | $\tilde{\pi}_{E}(\cdot\|s)$ | The policy that we are learning under corrupted data. | | $\pi_{\text{IQL}}(\cdot\|s)$ | The learned policy using IQL's weighted imitation learning under clean data. | | $\tilde{\pi}_{\text{IQL}}(\cdot\|s)$ | The learned policy using IQL's weighted imitation learning under corrupted data. | | $d^{\pi}(s,a)$ | The probability density function associated with policy $\pi$ at state $s$ and action $a$. | | $W_1(p,q)$ | The Wasserstein-1 distance that measures the difference between distributions $p$ and $q$. | | $q^{\pi}(\cdot \| s)$ | The value distribution of the policy $\pi$. | | $Z^{\pi}(s)$ | The random variable of the value distribution $q^{\pi}(\cdot\|s)$. | | $\epsilon_1$ | The KL divergence between policies $\pi_{E}$ and $\pi_{\text{IQL}}$, representing standard imitation error under clean data. | | $\epsilon_2$ | The KL divergence between policies $\tilde{\pi}\_{E}$ and $\tilde{\pi}\_{\text{IQL}}$, representing the standard imitation error under corrupted data. | | $\hat{\varsigma}_n$ | The midpoint of the action-value distribution $p_{\theta}$. |

Reviewer QZTE2024-08-09

Response to rebuttal

Thank you for your efforts in answering the questions and conducting additional experiments. I specifically like Fig G1 which helps in understanding the proposed methodology. I still have the following concerns: 1. Theorem A.3 is unclear to me since it is derived in terms of $\tilde \pi_{iql}$ which as per my understand is iql learned on corrupted data. How does this prove the upper bound for TRACER? How do you equate $Z^\pi(s)$ with $D^\pi(s, a)$ is steps 24, 25 and 26. And what is $D^\pi(s, a)$ as previously D is defined as D(s, a, r)? 2. The authors in Appendix B state we randomly select c% of the transitions each time for corruption. I am interested in knowing what is the value of c experimentally and what the effect on performance for varying c.

Reviewer wmJN2024-08-09

Thanks for the response

My concerns have mostly been addressed, so I have decided to raise my score. The added diagram and clarification on the writing greatly improve understanding. The authors are expected to include these in the revised paper.

Authorsrebuttal2024-08-09

Thanks for your kind support and for helping us improve the paper!

Dear Reviewer wmJN, Thanks for your kind support and for helping us improve the paper! We are committed to including all additional diagrams and clarifications and will incorporate your suggestions in the final version, if accepted. Thank you again for your valuable comments and guidance. Best, Authors

Authorsrebuttal2024-08-10

Thanks for your kind support and for helping us improve the paper!

Dear Reviewer V2Jx, Thanks for your kind support and for helping us improve the paper! We are committed to including all these additional results and detailed explanations and will incorporate your suggestions in the final version, if accepted. Thank you again for your valuable comments and guidance. Best, Authors

Reviewer V2Jx2024-08-10

Thank you for the discussion

Thank you for the discussion and the extra experiments. My concerns are addressed and I will raise my score.

Authorsrebuttal2024-08-11

Thanks for your kind support!

Dear Reviewer vWB5, Thanks for your kind support and further improvements you have suggested for our manuscript! We are committed to incorporating all your valuable suggestions in the final version, if accepted. Thank you again for your valuable comments and guidance. Best, Authors

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC