Constrained Policy Optimization with Explicit Behavior Density for Offline Reinforcement Learning

Due to the inability to interact with the environment, offline reinforcement learning (RL) methods face the challenge of estimating the Out-of-Distribution (OOD) points. Existing methods for addressing this issue either control policy to exclude the OOD action or make the $Q$ function pessimistic. However, these methods can be overly conservative or fail to identify OOD areas accurately. To overcome this problem, we propose a Constrained Policy optimization with Explicit Behavior density (CPED) method that utilizes a flow-GAN model to explicitly estimate the density of behavior policy. By estimating the explicit density, CPED can accurately identify the safe region and enable optimization within the region, resulting in less conservative learning policies. We further provide theoretical results for both the flow-GAN estimator and performance guarantee for CPED by showing that CPED can find the optimal $Q$-function value. Empirically, CPED outperforms existing alternatives on various standard offline reinforcement learning tasks, yielding higher expected returns.

Paper

References (85)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer krNs6/10 · confidence 4/52023-07-02

Summary

The paper proposes a novel algorithm for offline model-free reinforcement learning by constraining the actor to lie within a safe area defined by an explicit behavioral density. This behavioral density is obtained by thresholding a FlowGAN model. The resulting algorithm is competitive across various D4RL datasets.

Strengths

- Clearly presented algorithm and well-motivated method - Strong empirical evaluation and competitive results on D4RL MuJoCo - Solid theoretical justification of the convergence of the FlowGAN and RL training

Weaknesses

- Model-based offline methods typically use a state-action-based uncertainty penalty which could also be repurposed for the definition of the safe area. E.g. thresholding the model uncertainty as in MOReL [1]. This would be a useful baseline to show the benefit of the Flow-GAN method - The hand-crafted per-environment piecewise linear scheme for alpha limits the generality of the method - Algorithm 1 seems to be incorrect, the preceding discussion has the Flow-GAN optimization happening before RL training, whereas Algorithm 1 has the Flow-GAN optimization interleaved with RL training. - Line 150: definition of the ‘offline MDP’ is not a real MDP. For example, the state and action spaces should both be sets and the definition is a tuple. It is unclear what this definition adds beyond describing the marginal state distribution and behavioral policy. - In Theorem 4.2, the optimal Q-function on the “restricted product space” needs to be defined and explained. Minor: - Line 145: ‘triples’ is likely a mistake, the transition is a 4-tuple - Tables 1 and 2 missing EDAC [2], the current SOTA model-free offline RL method [1] MOReL: Model-Based Offline Reinforcement Learning. Rahul Kidambi, Aravind Rajeswaran, Praneeth Netrapalli, Thorsten Joachims. [2] Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble. Gaon An, Seungyong Moon, Jang-Hyun Kim, Hyun Oh Song.

Questions

- Could a dynamics model-based uncertainty (e.g. MOReL) provide a simple alternative to the Flow-GAN threshold? - Could the alpha tuning scheme be replaced by something not hand-crafted to increase the applicability of the method?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Not discussed in the paper.

Reviewer 1U1v5/10 · confidence 4/52023-07-04

Summary

This paper proposes a novel approach named Constrained Policy optimization with Explicit Behavior density (CPED) to mitigate the existing policy control methods, e.g., overly conservative or failing to identify OOD areas accurately. CPED use a flow-GAN model to explicitly estimate behavior policy density. The strength of CPED lies in its ability to accurately identify safe regions for explorations, which in turn, leads to less conservative learning policies.

Strengths

1. Well-motivated example in Fig. 1 and overall good writing. 2. Sufficient theoretical evidence to show that CPED has a fast convergence rate and is able to find the optimal Q-function value.

Weaknesses

1. The contribution is limited as it seems to directly borrow the idea of flow-gan into offline RL domains. 2. Besides the motivation example in Fig. 1, there is no faithful evidence to show that "accurately identify the feasible region, which includes both observed and unobserved but safe points" both in theoretical analysis and experiments. 3. Lack of survey and comparison with the most similar work [1], which also learns an expressive generative behavior model but by the diffusion technique. 4. Though there is an obvious edge of CPED against SPOT in experiments, more explanations are lacking to suppose it.

Questions

1. Could the authors provide more comparisons between CPED and SPOT and explain why CPED is more feasible? 2. Regarding policy learning (Eq. 7), why not transfer it to a Lagrange function like SPOT? To the best of my knowledge, explicitly excluding bad transitions has a little edge against other weighting or penalty methods in terms of generalization, which is a key contribution mentioned by the authors.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

Lack of related work: the in-sample learning paradigm [2-8] is now one of the trends in offline reinforcement learning, those new methods are worth being well mentioned in related work. --- I decided to change my score from 4 to 5 after the first round discussion with the authors. --- [1] Chen H, Lu C, Ying C, et al. Offline reinforcement learning via high-fidelity generative behavior modeling[J]. arXiv preprint arXiv:2209.14548, 2022. [2] Kostrikov I, Nair A, Levine S. Offline reinforcement learning with implicit q-learning[J]. arXiv preprint arXiv:2110.06169, 2021. [3] Xu H, Jiang L, Jianxiong L, et al. A policy-guided imitation approach for offline reinforcement learning[J]. Advances in Neural Information Processing Systems, 2022, 35: 4085-4098. [4] Garg D, Hejna J, Geist M, et al. Extreme q-learning: Maxent RL without entropy[J]. arXiv preprint arXiv:2301.02328, 2023. [5] Xiao C, Wang H, Pan Y, et al. The in-sample softmax for offline reinforcement learning[J]. arXiv preprint arXiv:2302.14372, 2023. [6] Zhang H, Mao Y, Wang B, et al. In-sample Actor Critic for Offline Reinforcement Learning[C]//The Eleventh International Conference on Learning Representations. 2022. [7] Xu H, Jiang L, Li J, et al. Offline rl with no ood actions: In-sample learning via implicit value regularization[J]. arXiv preprint arXiv:2303.15810, 2023. [8] Hansen-Estruch P, Kostrikov I, Janner M, et al. Idql: Implicit q-learning as an actor-critic method with diffusion policies[J]. arXiv preprint arXiv:2304.10573, 2023.

Authorsrebuttal2023-08-15

Look forward to your response

Thank you for taking the time to review our paper. We trust that our responses have effectively addressed the concerns you expressed in your review. Nevertheless, should you still have any lingering questions or unresolved matters, please feel free to inform us. We are committed to offering additional clarification and resolving any remaining issues to the best of our ability.

Reviewer 6qDS5/10 · confidence 3/52023-07-06

Summary

This paper proposes a new offline RL algorithm called Constrained Policy optimization with Explicit Behavior density (CPED). The main idea is utilizing a flow-GAN model to estimate the probability density explicitly, which limits the exploration in the safe region. To demonstrate the effectiveness, the author provides some theoretical proofs for the convergence of CPED, and evaluates CPED in various standard offline RL tasks.

Strengths

This paper is overall well-written, and it performs a comprehensive survey in offline RL and related fields such as inverse RL and generative model. The proposed method is well-motivated, which introduces flow-GAN to offline RL appropriately. This paper provides detailed theoretical analyses of the convergence to confirm the effectiveness of CPED.

Weaknesses

1. The novelty of this paper is somewhat limited as it primarily replaces the VAE model used in SPOT [1] with flow-GAN. 2. With the exception of the ho-med dataset, CPED does not demonstrate clear improvements and, performs even worse than some previous methods in the experimental datasets. 3. The implementation of CPED is not clearly stated, and this aspect will require further discussion in the questions. 4. The usage of certain notations is confusing, and there are some grammar errors.

Questions

1. In Equation 7, the Lagrangian multiplier $\alpha$ is mentioned in the algorithm but cannot be found. Could you clarify the actual objective function? Additionally, the hyperparameter $\epsilon$ is not mentioned in your algorithm. How does $\epsilon$ affect the performance of your algorithm? 2. In flow-GAN, specific neural networks are utilized for the generator and discriminator. Does this limitation affect their representational ability? Furthermore, does it result in a significant gap between the estimated policy and the behavioral policy? 3. Previous work often considers online fine-tuning tasks after offline RL. Can CPED be applied to such tasks? If so, how does CPED compare to other offline RL methods in these scenarios? 4. Figure 2 shows the learning curve comparison between BEAR and CPED. However, BEAR is considered somewhat outdated. As SPOT is a more recent offline RL method and seems more relevant to CPED, I'm particularly interested in the comparison between SPOT and CPED. 5. In Section 5.2, you mention an additional trick using time-varying $\alpha$. It appears that this trick could be applicable to SPOT as well. Since the empirical improvement is limited, I'm curious if the improvement primarily results from this trick rather than flow-GAN. Could you provide additional results of SPOT with the time-varying $\alpha"? 6. The choice of $\epsilon$ is not clearly explained. What is the specific quantile of the behavior policy density that you refer to? Does it correspond to the median or other quantiles? It would be helpful if you could provide examples of this "quantile" in several datasets.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

Please refer to the weakness parts.

Authorsrebuttal2023-08-15

Look forward to your response

Thank you so much for reviewing our paper, and we appreciate your helpful suggestions. We sincerely hope that our responses have adequately addressed the concerns you raised in your review. However, if you still have any unresolved concerns or additional questions, please do not hesitate to let us know. We would be more than happy to provide further clarification and address any remaining issues.

Reviewer Y4vE6/10 · confidence 3/52023-07-06

Summary

This work proposes to use flow-GAN model to explicitly model the distribution of the behaviour policy and use that to alleviate the OOD actions problem in offline RL. A theoretical analysis of convergence is presented. Empirical evaluation on d4rl gym locomotion and antmaze show that the proposed CPED algorithm is very competitive.

Strengths

* The method is novel and very intuitive. Instead of using all different tricks to avoid OOD, explicitly modelling the behaviour policy is much more straightforward. * The writing is clear * Empirical evaluation shows the performance of CPED is comparable to SOTAs

Weaknesses

* Compared to prior methods like IQL, DT, or even CQL. The proposed method has an additional generative modelling module, which makes the overall architecture more complex. * Again, the empirical results are good but it's technically not surpassing SOTA. For gym-locomotion, there are algorithms like ATAC [1] and SAC-N [2] that have similar level of performance. As for antmaze, the performance of CPED is inferior than IQL, which is not the SOTA for offline RL in general. For model-based methods, there're methods like trajectory transformer [3] that perform better than IQL. [1] https://proceedings.mlr.press/v162/cheng22b/cheng22b.pdf [2] https://arxiv.org/pdf/2110.01548.pdf [3] https://arxiv.org/abs/2106.02039 Minor issues: > CPED can accurately identify the safe region and enable exploration within the region I would avoid using the term "exploration" here because it can hint to readers that the proposed method can be helpful for online learning but I believe this is not what the authors mean here.

Questions

* What's the computational overhead of training this Flow-GAN? * Why do you pick Flow-GAN? Why not use simpler ones like VAE, or even just discretization?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

Yes

Reviewer krNs2023-08-13

Thanks for the response

Thank you for the responses. W2. I don't think a handcrafted schedule should be presented as an advantage. Any other algorithm could have manually done this but it would have reduced the applicability of their method. W3. Could the authors clarify why it is interleaved? Why would it not be better to use a GAN trained to convergence? Q1: The dynamics model in MOReL provides an uncertainty bonus over both state and action, so may already incorporate action uncertainty.

Authorsrebuttal2023-08-14

Thanks for your comments

**W2. I don't think a handcrafted schedule should be presented as an advantage. Any other algorithm could have manually done this but it would have reduced the applicability of their method.** **Response:** Thank you for your comment. We agree with you that handcrafted alpha may increase the difficulties in hyperparameter tuning, thus limiting the method's applicability. However, our piece-wise alpha scheme is straightforward to implement. Specifcally, we would like to further emphasize the following: 1. While the idea of automatic learning alpha (optimizing Lagrangian multiplier) appears in some early works (BEAR/UWAC), more recent papers(BRAC) tend to prefer hand-craft alpha based on experimental considerations. 2. We agree that the handcraft scheme of alpha is not the specific advantage of our method. (The advantage of our manuscript comes from the Flow-GAN model). As a matter of fact, the hand-craft scheme is a userful trick in model training for better convergence and results. 3. In our manuscript, the piece-wise linear (constant) scheme is suggested, and this scheme is easy to implement. We only need to set continuous intervals (4 intervals in Mujoco and Antmze tasks, actually not very sensitive from experiments, see Figure 2(a,b) in the manuscript), and the alpha for each interval decreases exponentially during the training process. **W3. Could the authors clarify why it is interleaved? Why would it not be better to use a GAN trained to convergence?** **Response:** Thank you for your comment. We are sorry for our unclear statement in our last response. First, we would like to clarify that before conducting the offline RL task, the GAN has undergone training to achieve convergence (when Flow-GAM becomes stable). Subsequently, the GAN is trained in an interleaved manner with the RL model. The reason for interleaving the training of the Flow-GAN and RL model is to enhance the Flow-GAN's adaptation to downstream offline RL tasks. Actually, we have attempted only training the RL model with a converged and fixed Flow-GAN, we observed that the convergence of the RL model was not sufficiently stable. Therefore, we recommend trying the interleaved training approach. **Q1: The dynamics model in MOReL provides an uncertainty bonus over both state and action, so may already incorporate action uncertainty.** **Response :** Thank you for pointing out the merits of MORel. We acknowledge that MORel captures the uncertainties in both action and state, which we also mentioned in our previous response.(*Though MoReL also considers the uncertainty of action space....*). As for whether MORel can be a substitute for the flow-GAN threshold, its feasibility relies on its capacity to learn complex distributions. We find this to be an interesting topic (along with the utilization of model-based mechanisms to assist model-free methods), and it would be meaningful to delve deeper into this direction in the future. We again appreciate your helpful suggestions.

Reviewer Y4vE2023-08-14

Response to the Rebuttal

I would thank the authors for their clarification. I like the fact that the proposed method has only 10% computational overhead. For the reason of defending Flow-GAN, the arguments are not strong enough to convince me, especially without further empirical results. I'll keep the score for now.

Authorsrebuttal2023-08-15

Response to the Reviewer & Thank you for you comments

**I would thank the authors for their clarification. I like the fact that the proposed method has only 10% computational overhead.** **Response:** Thank you so much for your nice comment. **For the reason of defending Flow-GAN, the arguments are not strong enough to convince me, especially without further empirical results.** **Response:** Thank you for your comment. To provide further empirical results and demonstrate the advantages of the flow-GAN, we conduct comparisons between CPED, SPOT[1], and SfBC[2]. In SPOT, the VAE model is employed to approximate the lower bound of the behavior likelihood (also compared in our manuscript). On the other hand, SfBC employs diffusion techniques, another type of generative model, to estimate the behavior model. The results of each method for D4RL tasks are presented in the table below. | Task | CPED | SPOT | SfBC | | ---- | ---- | ---- | ---- | | HalfCheetah-Medium | **61.8** | 58.4 | 45.9 | | Hopper-Medium | **100.1** | 86.0 | 57.1 | | Walker-Medium | **90.2** | 86.4 | 77.9 | | HalfCheetah-Medium-replay | **55.8** | 52.2 | 37.1 | | Hopper-Medium-replay | 98.1 | **100.2** | 86.2 | | Walker-Medium-replay | **91.9** | 91.6 | 65.1 | | HalfCheetah-Medium-expert | 85.4 | 86.9 | **92.6** | | Hopper-Medium-expert | 95.3 | 99.3 | **108.6** | | Walker-Medium-expert | **113.04** | 112.0 | 109.8 | From the table, it becomes evident that CPED outperforms its competitors across various tasks. Coupled with the reasoning analysis provided in our previous response, we are confident that Flow-GAN stands as a superior selection compared to alternative tools for behavior model estimation. [1] Supported policy optimization for offline reinforcement learning. NeurlIPS 2022. [2] Offline reinforcement learning via high-fidelity generative behavior modeling. ICLR 2023.

Reviewer 1U1v2023-08-18

Official Response by Reviewer 1U1v

Thank you for the clarifications provided, as most of they have addressed most of my concerns. I now understand the rationale behind using the Flow-GAN to estimate the density of the behavior policy. Specifically, the \textbf{further motivation for Flow-GAN} and Chapter 3.2 in the manuscript have become more convincing to me instead of finding a higher-fidelity density estimator. I also appreciate the authors' efforts to present more persuasive experiments demonstrating the efficiency of the Flow-GAN model by including additional comparisons with 'SfBC' in response to reviewer 6qDS. As a result, I have decided to raise my score. Furthermore, if the authors are willing to provide more compelling evidence as to why the Flow-GAN model outperforms the VAE model for estimating the behavior policy density, by conducting didactic toy examples, I would be inclined to further increase my rating. Other Questions: Is the code available?

Authorsrebuttal2023-08-20

(1/2) Response to Reviewer 1U1v

Thank you for your reply, and we appreciate your kind response. **Furthermore, if the authors are willing to provide more compelling evidence as to why the Flow-GAN model outperforms the VAE model for estimating the behavior policy density, by conducting didactic toy examples, I would be inclined to further increase my rating.** **Response:** Thank you for your comment. Of course we are glad to conduct additional studies comparing the VAE and flow-GAN as well as the advantages of CPED. Here, we firstly want to cite the Figure 1 in [1], which is a typical motivating example showing GAN-based model is much better than VAE-based model for approximating behavior policy. We are also conducting additional toy example comparing VAE and flow-GAN in generating behavior samples. Since we cannot attach further figures on openreview website at this stage, we will compare the likelihood of generated samples (by VAE and flow-GAN). We are still running the program, and we will soon provide the results in 1 day. In addition, in our response to 6qDs, we further compare BEAR, SPOT and CPED under the Halfcheetah-medium task, and the result figure is in our github code page <https://github.com/rl-study-group/rl_study_cped/tree/main> (github: Readme -> Note. Not allowed to attach further figures now). We can see that SPOT could reach a score of 6500 very fast, while CPED could achieve higher scores. both SPOT and CEPD significantly outperform BEAR. **Other Questions: Is the code available?** **Response:** Yes, we have uploaded the code at <https://github.com/rl-study-group/rl_study_cped/tree/main>. Note: * Please kindly search rl-study-group or rl\_study\_cped on <https://github.com/> if the above link is invalid due to the markdown grammar issue. * Due to the tight timeline for organizing the CPED code and uploading to github, some test code hasn't been completely removed. We will make an effort to tidy up all the code as soon as possible. [1] A Behavior Regularized Implicit Policy for Offline Reinforcement Learning: https://arxiv.org/pdf/2202.09673.pdf

Authorsrebuttal2023-08-21

(2/2) Follow up to Reviewer 1U1v

We have completed the toy-example experiments comparing VAE and Flow-GAN. Considering no further figures are allowed to be attached on openreview website at this stage, we compare the **mean of log-likelihood** of the generated samples in the following table. (We will attach the figure of generated samples in the revised manuscript) |Setting| Ground Truth | VAE Model | Flow-GAN | |----| ---- | ---- | ---- | |Setting 1| -2.89 | -140.83 | -24.18 |Setting 2| -2.78 | -50.05 | -6.12 Referring to Figure 1 in [1], it seems VAE-based model is not able to learn multi-modal data well. Thus, we select two simpler settings in our toy example. * **Setting 1**: We gather approximately $12,800,000$ data points from a multivariate normal distribution with a mean of $[1,9]$ and covariance matrix of $\Sigma = [[1,0],[0,1]]$. * **Setting 2**: We gather similar amount (with setting 1) data points from a mixture gaussian distribution, in which data are randomly drawn from two independent gaussian distributions ($\mu_1 =[1,1]$ and $\mu_2 = [9,9]$, $\Sigma_1 = \Sigma_2 = [[1,0],[0,1]]$) with equal probability. We then employ both VAE and flow-GAN to generate $1000$ samples and calculate the mean of log-likelihood. The table demonstrates that Flow-GAN outperforms VAE significantly in approximating the original data distribution. [1] A Behavior Regularized Implicit Policy for Offline Reinforcement Learning: https://arxiv.org/pdf/2202.09673.pdf

Reviewer 6qDS2023-08-18

Thank you for the clarification and additional experimental results. The rebuttal has addressed most of my concerns. However, the reviewer still feels it necessary to perform more detailed experiments to compare CPED with SPOT to confirm the superiority of CPED since SPOT is **fairly relevant** to CPED. Besides, the performance of SPOT is much better than BEAR, as shown in Table 2. The authors should not claim “SPOT would likely demonstrate similar outcomes” without further experiments. Besides, the reviewer is curious about the meaning of d4rl score in the pdf file and suggests that the same metrics (average return) should be used here to make it convenient to compare the results.

Authorsrebuttal2023-08-20

(1/2) Response to Reviewer 6qDS

Thank you for your response. We are sorry for some unclear statements and definitions in our last response. We have added extra experiements, and corrected unclear statements and definitions in the following. **However, the reviewer still feels it necessary to perform more detailed experiments to compare CPED with SPOT to confirm the superiority of CPED since SPOT is fairly relevant to CPED.** **Response:** Thank you for your comment. In last 2 days, we add extra experiments further comparing SPOT and CPED. Per the request of reviewer 1U1v, we have released our code on github <https://github.com/rl-study-group/rl_study_cped/tree/main>, where we also add the addtional comparison figure (github: Readme -> Note. We are currently not allowed to attach further figures on openreview website at this stage. Limited by time, we only finish the half-cheetah task for demonstration, and we will add more task results in the revised manuscript). Please note as we haven't disclosed any personal information in this github link, this **does not** violate the double-blind reviewing policy. From the figure in the link, we can see that the average return of SPOT could quickly reach a score of 6500 and then fluctuates around this level. Neverthesless, although CEPD is not as fast as SPOT, it could finally reach a higher score around 7000. **Besides, the performance of SPOT is much better than BEAR,as shown in Table 2. The authors should not claim ''SPOT would likely demonstrate similar outcomes'' without further experiments.** **Response:** We apologize for the unclear statements. In our last response, we want to claim **''SPOT would likely demonstrate similar outcomes''** ***with CPED, not with BEAR***, since both SPOT and CPED are less conservative. As shown in the link above, we have compared BEAR, SPOT and CPED, and we can see that both SPOT and CEPD significantly outperform BEAR, and CPED works better than SPOT. **Besides,the reviewer is curious about the meaning of d4rl score in the pdf file and suggests that the same metrics (average return) should be used here to make it convenient to compare the results.** **Response:** Thank you for pointing out our unclear definition. Actually, the definition of *d4RL Score* in our manscript (defined from SPOT source code) is **d4rl\_score = normalized_score/100**, where *normalized_score* is **the standard definition for d4rl tasks** from the original d4rl paper[1]. In the SPOT paper [2], the authors also call *normalized_score* as *normalized return*, and the name **d4rl\_score** comes from the results of SPOT after running the source code. Since we cannot attach further figure here, we will revise the metric as **''normalized score''** for consistency (same metric with the original d4rl paper, SPOT and other references) in the revised manuscript. Actually the shape of the figure remains unchanged, only the vertical axis will be scaled up by a factor of 100. [1] d4rl paper: https://arxiv.org/pdf/2004.07219.pdf. [2] SPOT: https://openreview.net/pdf?id=KCXQ5HoM-fy

Authorsrebuttal2023-08-21

(2/2)Follow up to Reviewer 6qDS

In this response, we provide additional evidence by presenting the outcomes of new experiments comparing the VAE and Flow-GAN models, as per the comment from Reviewer 1U1v. Firstly, we intend to cite Figure 1 in [1], a nice illustrative example, to show that GAN-based models outperform VAE-based models in approximating behavior policies. Furthermore, we conduct additional toy example experiments by comparing the performance of VAE and flow-GAN in generating behavior samples. Due to constraints on attaching additional figures to the OpenReview website at this stage, we present a comparison of the **mean of log-likelihood** of generated samples in the table below. (We will attach the figure of generated samples in the revised manuscript) |Setting| Ground Truth | VAE Model | Flow-GAN | |----| ---- | ---- | ---- | |Setting 1| -2.89 | -140.83 | -24.18 |Setting 2| -2.78 | -50.05 | -6.12 Referring to Figure 1 in [1], it seems VAE-based model is not able to learn multi-modal data well. Thus, we select two simpler settings in our toy example. * **Setting 1**: We gather approximately $12,800,000$ data points from a multivariate normal distribution with a mean of $[1,9]$ and covariance matrix of $\Sigma = [[1,0],[0,1]]$. * **Setting 2**: We gather similar amount (with setting 1) data points from a mixture gaussian distribution, in which data are randomly drawn from two independent gaussian distributions ($\mu_1 =[1,1]$ and $\mu_2 = [9,9]$, $\Sigma_1 = \Sigma_2 = [[1,0],[0,1]]$) with equal probability. We then employ both VAE and flow-GAN to generate $1000$ samples and calculate the mean of log-likelihood. The table demonstrates that Flow-GAN outperforms VAE significantly in approximating the original data distribution. [1] A Behavior Regularized Implicit Policy for Offline Reinforcement Learning: https://arxiv.org/pdf/2202.09673.pdf

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC