Diffusion-Reward Adversarial Imitation Learning

Imitation learning aims to learn a policy from observing expert demonstrations without access to reward signals from environments. Generative adversarial imitation learning (GAIL) formulates imitation learning as adversarial learning, employing a generator policy learning to imitate expert behaviors and discriminator learning to distinguish the expert demonstrations from agent trajectories. Despite its encouraging results, GAIL training is often brittle and unstable. Inspired by the recent dominance of diffusion models in generative modeling, we propose Diffusion-Reward Adversarial Imitation Learning (DRAIL), which integrates a diffusion model into GAIL, aiming to yield more robust and smoother rewards for policy learning. Specifically, we propose a diffusion discriminative classifier to construct an enhanced discriminator, and design diffusion rewards based on the classifier's output for policy learning. Extensive experiments are conducted in navigation, manipulation, and locomotion, verifying DRAIL's effectiveness compared to prior imitation learning methods. Moreover, additional experimental results demonstrate the generalizability and data efficiency of DRAIL. Visualized learned reward functions of GAIL and DRAIL suggest that DRAIL can produce more robust and smoother rewards. Project page: https://nturobotlearninglab.github.io/DRAIL/

Paper

References (60)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer 5b3J6/10 · confidence 4/52024-07-02

Summary

The paper proposes Diffusion-Reward Adversarial Imitation Learning (DRAIL) an Adversarial Imitation Learning method where the discriminator is parameterized by the loss of a conditional diffusion model. The diffusion model takes in a state-action pair $(s, a)$ and a binary label $c$ indicating whether the state-action pair comes from the discriminator (label $c^-$) or from the training set (label $c^+$). Its loss on these inputs is denoted $\mathcal{L}\_{\textrm{diff}}(s, a, c)$. The discriminator is then defined as $D(s, a) = \sigma(\mathcal{L}\_{\textrm{diff}}(s, a, c^+) - \mathcal{L}\_{\textrm{diff}}(s, a, c^-))$, where $\sigma$ is the sigmoid function. The method resembles DiffAIL (Wang et al. 2023), which defines the discriminator as $\exp(-\mathcal{L}\_{\textrm{diff}}(s, a))$ (unconditional diffusion model), as well as Diffusion Classifier (Li et al. 2023). They benchmark DRAIL on six settings, including long-horizon planning (e.g. Maze2d), high-dimensional control (e.g. AntReach) and dexterous manipulation (e.g. HandRotate). They compare against methods including Behavior Cloning, the original GAIL, and variants such as Wasserstein AIL and DiffAIL. They also compare against Diffusion Policy, which uses a diffusion model to parameterize a policy rather than a reward function. In terms of raw performance, DRAIL obtains: - visibly better results than the baselines in 4/6 environments; - a result very close to that of DiffAIL in 1/6 environment; - a result worse than DiffAIL in 1/6 environment. Further experiments show: - DRAIL is more robust to noise on initial states and goal positions than the baselines on a block-pushing task (FetchPush); - DRAIL is more data-efficient than the baselines on the Walker environment (continuous high-dimensional control) and FetchPush; - DRAIL learns more generalizable rewards than GAIL on a synthetic 2d point dataset.

Strengths

1. Clarity: the paper is well-written and the proposed method is explained clearly. 1. Extensive comparison with DiffAIL, albeit in the appendix. 1. Methodological novelty: the authors propose using a conditional diffusion model to parameterize the discriminator for AIL, which removes limitations from DiffAIL (such as an arbitrary decision boundary at $\ln 2$; see Appendix A). 1. Significance and quality of experimental results: the paper demonstrates that defining the discriminator using a conditional diffusion model improves performance, robustness and data-efficiency, compared to DiffAIL (which uses an unconditional diffusion model in the discriminator) and to other baselines. Hence, one can argue it constitutes a step forward in Adversarial Imitation Learning. 1. Referencing strongly related prior work: the authors point to DiffAIL and Diffusion Classifier, which strongly resemble different parts of DRAIL (namely using the loss of a diffusion model to parameterize a discriminator, and how to construct such a discriminator). I personally was not aware of these two papers before, and commend the authors for bringing these to the reader’s attention.

Weaknesses

## Better delimiting the author’s contributions Despite DiffAIL and Diffusion Classifier being mentioned and their relationship to DRAIL being discussed, my impression is that more credit should be materially given to these two prior works. For example, in line 143, the authors state “we propose to leverage the training procedure of DDPM and develop a mechanism to provide learning signals to a policy using a single denoising step”. However, it seems to me that this much was already present in DiffAIL. Similarly, line 145 states “Our key insight is to leverage the derivations developed by Kingma et al. [22], Song et al. [47], which suggest that the diffusion loss, i.e., the difference between the predicted noise and the injected noise, indicates how well the data fits the target distribution since the diffusion loss is the upper bound of the negative log-likelihood of data in the target distribution”. However, it seems that a similar insight had already been used by Li et al. (2023), as mentioned in line 177. Hence, my understanding is that, in the context of these two prior works, DRAIL can be seen as a combination of DiffAIL and Diffusion Classifier (DC). Please feel free to correct me if this understanding is incorrect. This still constitutes novelty, although less than if DiffAIL and DC hadn’t preceded this work. In fact, in Appendix A the authors make what I consider to be a compelling case for why the discriminator should be “symmetric”, rather than “one-sided” like in DiffAIL. As such, I believe this paper’s contributions could be presented in a way that better separates its original contribution from that present in prior work. One way of doing this could be: - Add a section on DiffAIL and Diffusion Classifier to Section 3 (preliminaries). - In Section 4 (Approach), discuss the limitations of DiffAIL, e.g. as done in Appendix A. Argue that using a conditional diffusion model with a symmetric discriminator (via Diffusion Classifier) would address these limitations. - Hence present the final methodology as the combination of DiffAIL and DC. Again, I am open to changing my mind about the above assessment, and will raise my score accordingly in that case, or in case a refactoring of the presentation of the method is implemented by the authors. ## Baselines against planning methods that use diffusion models The authors baseline DRAIL against 3 AIL methods and Diffusion Policy. The motivation for including the latter is to “compare learning a diffusion model as a policy (diffusion policy) or reward function (ours)”. As per a recent survey of Zhu et al. (2023), diffusion models have been used in sequential decision-making not only for parameterizing policies, but also as planners, as is the case in the seminal work of Janner et al. (2022). As such, for a more holistic comparison against methods outside of the AIL framework, the authors could include diffusion-based planning baselines such as Diffuser (Janner et al. 2022). Recent work by Nuti et al. (2023) has also shown reward functions can be recovered from pairs of diffusion models in the setting of Janner et al. (2022), so that it would also be possible to compare the rewards learned via DRAIL against those obtained from Diffuser. ### References Zhu, Zhengbang, et al. "Diffusion models for reinforcement learning: A survey." arXiv preprint arXiv:2311.01223 (2023). Nuti, Felipe, Tim Franzmeyer, and João F. Henriques. "Extracting reward functions from diffusion models." Advances in Neural Information Processing Systems 36 (2024).

Questions

See Weaknesses.

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

The authors consider the limitations of their work in Appendix G and societal impacts in Appendix I. My main concern with this work is the extent to which the authors claim originality over aspects of their method that are arguably already present in prior work. If the authors can clarify that this concern is misplaced, or if they modify the exposition of their contributions as outlined in the Weaknesses sections, I am open to raising my score.

Reviewer PZWW5/10 · confidence 4/52024-07-11

Summary

This paper aims to address the training instability problem in generative adversarial imitation learning. The authors propose a diffusion discriminative classifier that helps achieve a more stable policy learning process by enhancing the smoothness and robustness of the reward model. Additionally, the author experimentally validates the effectiveness of DRIL across various control tasks, including manipulation, locomotion, and navigation benchmarks.

Strengths

- The instability of AIL training is a key problem to its application, making research on this issue valuable. - DRAIL performs well across multiple benchmarks, improving sample efficiency and training stability.

Weaknesses

- The advantages of the diffusion discriminative classifier compared to diffusion reward [1] are not well explained. The authors claim that their proposed diffusion discriminative classifier only requires two reverse steps to obtain the corresponding reward, thereby reducing the computational resource consumption needed for sampling in diffusion models. However, I do not fully agree with this. According to equations (4) and (6), DRAIL needs to compute $L_{\text{diff}}(s,a,c^+)$ and $L_{\text{diff}}(s,a,c^-)$ when obtaining rewards. From equation (3), we know both terms require taking an expectation over the diffusion steps $T$. Therefore, to obtain an accurate reward, a number of inference steps equivalent to the diffusion steps are still needed. - This paper lacks ablation experiments, including a sensitivity analysis of the hyperparameters. - Some important AIL baselines are missing, such as DAC [2] and IQLearn [3]. [1] DiffAIL: Diffusion Adversarial Imitation Learning. AAAI, 2024. [2] Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning. ICLR, 2019. [3] IQ-Learn: Inverse soft-Q Learning for Imitation. NeurIPS, 2021.

Questions

- Why does DRAIL only require two reverse steps to obtain an accurate reward? Can you provide a more detailed explanation or analysis? - I believe that the requirement for stable algorithm training includes robustness to hyperparameters. A major issue with AIL methods is the need for extensive hyperparameter tuning, and well-tuned AIL can achieve good performance. What are the key hyperparameters for DRAIL? Can you conduct additional ablation experiments on these hyperparameters? - Can you compare DAC and IQLearn in your experiments? Both DAC and IQLearn use the gradient penalty for reward smoothing, and I am very interested in seeing a comparison between the diffusion discriminative classifier and the gradient penalty. I would like to raise my score if my concerns are well addressed.

Rating

5

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

The authors have discussed the limitations of this paper. DRAIL is unable to learn from state-only trajectories and suboptimal trajectories.

Reviewer 3pMZ6/10 · confidence 3/52024-07-11

Summary

The paper proposes a novel imitation learning framework that integrates a diffusion model into Generative Adversarial Imitation Learning (GAIL). The primary aim is to address the instability and brittleness associated with GAIL by introducing more robust and smoother reward functions for policy learning. The authors develop a diffusion discriminative classifier to enhance the discriminator's performance and generate diffusion rewards. Extensive experiments in various domains such as navigation, manipulation, and locomotion demonstrate that DRAIL outperforms or is competitive with existing imitation learning methods. The paper highlights DRAIL’s effectiveness, generalizability, and data efficiency, offering a significant contribution to the field of imitation learning.

Strengths

1. **Innovative Integration**: The integration of diffusion models into GAIL is innovative and addresses the common issue of instability in adversarial imitation learning. 2. **Comprehensive Experiments**: The authors conduct extensive experiments across diverse domains, including navigation, manipulation, and locomotion, which provide robust evidence of DRAIL's effectiveness. 3. **Generalizability and Data Efficiency**: The paper demonstrates superior generalizability to unseen states and goals, as well as high data efficiency, making the approach practical for real-world applications. 4. **Robust Reward Mechanism**: The diffusion discriminative classifier enhances the robustness of the reward functions, contributing to more stable and reliable policy learning. 5. **Visualizations**: The visualized comparisons of learned reward functions between GAIL and DRAIL effectively illustrate the advantages of the proposed approach. ###

Weaknesses

1. **Complexity**: The introduction of diffusion models adds complexity to the framework, which may pose challenges in implementation and require significant computational resources. 2. **Limited Scope in Real-World Applications**: While the experiments are diverse, the applicability in highly dynamic and unpredictable real-world environments is not thoroughly explored. 3. **Comparative Analysis Depth**: Although the paper compares DRAIL with several baselines, a deeper analysis of why certain methods perform better in specific tasks could enhance understanding. 4. **Scalability**: The scalability of DRAIL in larger and more complex environments is not extensively evaluated, which could be a limitation for broader adoption. 5. **Hyperparameter Sensitivity**: The sensitivity of DRAIL to various hyperparameters is not discussed, which could impact its robustness across different settings. ###

Questions

1. How does DRAIL handle environments with rapidly changing dynamics, and what are the limitations in such scenarios? 2. What are the computational overheads associated with incorporating diffusion models, and how does it compare with other state-of-the-art methods in terms of efficiency? 3. How sensitive is DRAIL to the choice of hyperparameters, and what guidelines can be provided for tuning these parameters in different environments? 4. Can the diffusion discriminative classifier be extended or modified to further improve the reward robustness and policy performance in more complex tasks? 5. What specific strategies were employed to ensure the stability of the diffusion model training, and how do these strategies affect the overall performance of DRAIL?

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Shown in weakness

Authorsrebuttal2024-08-10

Reminder: The reviewer-author discussion period ends in three days

We would like to express our sincere gratitude to the reviewer for the thorough and constructive feedback. We are confident that our responses adequately address the concerns raised by the reviewer, including the following points. - A description of computational resources - A discussion of the broad applicability of our proposed method - An additional hyperparameter sensitivity experiment - A discussion of further improving the robustness and stability of our proposed method Please kindly let us know if the reviewer has any additional concerns or if further experimental results are required. We are fully committed to resolving any potential issues, should time permit. Again, we thank the reviewer for all the detailed review and the time the reviewer put into helping us to improve our submission.

Authorsrebuttal2024-08-13

Reminder: The reviewer-author discussion period ends in 20 hours

We would like to express our sincere gratitude to the reviewer for the thorough and constructive feedback. We are confident that our responses and additional experimental results adequately address the concerns raised by the reviewer. Please consider our rebuttal and kindly let us know if the reviewer has any additional concerns.

Reviewer uNxe5/10 · confidence 3/52024-07-22

Summary

Imitation learning (IL) is a research area that focuses on learning a policy from expert demonstrations. One of the most popular imitation learning techniques, General Adversarial Imitation Learning (GAIL), has been proposed to mitigate some of the issues that naive IL algorithms suffer from. Although GAIL has seen a lot of success, it is known to be very unstable and difficult to tune. The authors propose Diffusion Reward Adversarial Imitation Learning (DRAIL) to mitigate some of the difficulties of training GAIL. In particular, the authors use a diffusion model to train the reward function. With this change in reward function design, the authors showed in practice that policies trained with DRAIL performed better than past approaches that attempted to mitigate GAIL issues. Additionally, the authors demonstrated that they could learn a reward function with a very low number of expert trajectories compared to other approaches.

Strengths

- The paper was well written, and the author's contribution was easy to follow. - The authors performed a thorough empirical investigation to show how their proposed approach compared against baselines. - The authors propose an algorithm that takes full advantage of recent advancements in generative modeling.

Weaknesses

- The paper needs more novelty. DiffAIL proposed using diffusion as a reward, and DiffAIL also proposed using the training error as a reward signal. - The paper needs to include comparisons to the two missing cites [1] and [2]. - The authors do not provide any insight into why the proposed reward function is better than DiffAIl, especially since DIffAIL sometimes matches or performs better than the proposed algorithms.

Questions

- What denoise step values did you try? And how did they perform? - Could you explain why the proposed reward model performs better than DiffAIL? - If BC performs better than all the baselines on Walker, then that means the task should be easy for all the baselines. - Did you try increasing or decreasing the number of expert samples used for learning? Are you using D4RL? If so, what dataset split are you using? - Did you try training various levels of optimal demonstration, such as walker2d-expert-v0, walker2d-medium-v0, and walker2d-random-v0, to see how well the algorithms perform when noise is introduced? - Are you taking the argmax or sampling from the learned imitation learning policies? - Did you try other divergences besides Jensen-Shannon? Missing citations [1], [2], [1] A Coupled Flow Approach to Imitation Learning by Freund 2023 [2] Diffusion Model-Augmented Behavioral Cloning by Chen et al. 2024

Rating

5

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

Yes

Authorsrebuttal2024-08-10

Reminder: The reviewer-author discussion period ends in three days

We would like to express our sincere gratitude to the reviewer for the thorough and constructive feedback. We are confident that our responses adequately address the concerns raised by the reviewer, including the following points. - A detailed description of our contributions compared to DiffAIL - An intuitive explanation of why our method works - A discussion of Freund et al., 2023 (CFIL) and Chen et al., 2024 (DBC) - A description of data efficiency experiments, i.e., learning from different amounts of expert data, provided in the main paper - An explanation of training and sampling from learned policies using PPO - A discussion of using different divergence measures Please kindly let us know if the reviewer has any additional concerns or if further experimental results are required. We are fully committed to resolving any potential issues, should time permit. Again, we thank the reviewer for all the detailed review and the time the reviewer put into helping us to improve our submission.

Authorsrebuttal2024-08-13

Reminder: The reviewer-author discussion period ends in 20 hours

We would like to express our sincere gratitude to the reviewer for the thorough and constructive feedback. We are confident that our responses and additional experimental results adequately address the concerns raised by the reviewer. Please consider our rebuttal and kindly let us know if the reviewer has any additional concerns.

Reviewer PZWW2024-08-09

Thank you for your detailed answer to my questions. The response addressed my main concerns, therefore, I increased my score to 5.

Authorsrebuttal2024-08-10

Re: Official Comment by Reviewer PZWW

We sincerely thank the reviewer for acknowledging our rebuttal and for helping us to improve our submission.

Authorsrebuttal2024-08-10

Reminder: The reviewer-author discussion period ends in three days

We would like to express our sincere gratitude to the reviewer for the thorough and constructive feedback. We are confident that our responses adequately address the concerns raised by the reviewer, including the following points. - A clarification of our contributions and a detailed plan to reorganize our paper based on the reviewer's suggestion - Discussions of Janner et al., 2022 (Diffuser) and Nuti et al., 2023 (Extracting Reward Functions from Diffusion Models) Please kindly let us know if the reviewer has any additional concerns or if further experimental results are required. We are fully committed to resolving any potential issues, should time permit. Again, we thank the reviewer for all the detailed review and the time the reviewer put into helping us to improve our submission.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC