HIQL: Offline Goal-Conditioned RL with Latent States as Actions

Unsupervised pre-training has recently become the bedrock for computer vision and natural language processing. In reinforcement learning (RL), goal-conditioned RL can potentially provide an analogous self-supervised approach for making use of large quantities of unlabeled (reward-free) data. However, building effective algorithms for goal-conditioned RL that can learn directly from diverse offline data is challenging, because it is hard to accurately estimate the exact value function for faraway goals. Nonetheless, goal-reaching problems exhibit structure, such that reaching distant goals entails first passing through closer subgoals. This structure can be very useful, as assessing the quality of actions for nearby goals is typically easier than for more distant goals. Based on this idea, we propose a hierarchical algorithm for goal-conditioned RL from offline data. Using one action-free value function, we learn two policies that allow us to exploit this structure: a high-level policy that treats states as actions and predicts (a latent representation of) a subgoal and a low-level policy that predicts the action for reaching this subgoal. Through analysis and didactic examples, we show how this hierarchical decomposition makes our method robust to noise in the estimated value function. We then apply our method to offline goal-reaching benchmarks, showing that our method can solve long-horizon tasks that stymie prior methods, can scale to high-dimensional image observations, and can readily make use of action-free data. Our code is available at https://seohong.me/projects/hiql/

Paper

References (100)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer kqhw7/10 · confidence 4/52023-06-27

Summary

The paper introduces HIQL, a hierarchical algorithm for offline goal-conditioned RL. HIQL utilizes an action-free version of IQL to learn the value function and subsequently derives both high-level and low-level policies from this shared value function using AWR. The paper asserts that the hierarchical structure offers increased robustness to value function noise, leading to enhanced performance in achieving long-horizon goals. The effectiveness of the proposed method is validated through experiments conducted in both state-based and pixel-based tasks.

Strengths

* Overall, the paper is well written and well organized. * The proposed method HIQL incorporates IQL into a hierarchical framework, similar to prior offline RL work POR. However, HIQL incorporates additional designs from hierarchical RL, including predicting latent waypoints and k-step waypoints. Consequently, it is a new approach from both directions, offering valuable insights for tackling offline GCRL with hierarchical RL. * The examples in Section 4.1 and Section 4.3 are helpful for readers to understand. * The experiments are thorough, showing improved performance on both state-based and image-based tasks.

Weaknesses

* The given examples show that the hierarchical structure is more robust to value noises in discrete tasks. Generally, in continuous tasks, the value function tends to be smoother. This raises the question of whether the performance improvement observed in HIQL is solely attributed to the hierarchical structure. If the claim in the paper is right, a method using smoothed value function (e.g., [1]) without hierarchical structure may achieve comparable performance with HIQL. Additional explanation and comparison are encouraged. * More information to reproduce the Figure 2 need to be provided. * In the appendix, the authors concat [s,g] before feeding into $\phi$. Therefore, it is recommended that Figure 1 be revised to address this matter. * In the experiments, all the baselines considered are related to (weighted) imitation learning, making the inclusion of additional baselines utilizing temporal difference learning [2][3] necessary for a more comprehensive evaluation. In addition, it is notable that HIQL leverages the advantage of goal relabeling, while baselines such as IQL do not utilize this technique. Despite this, IQL has already demonstrated strong performance on the pixel-based Procgen Maze benchmark. Therefore, it remains unknown if HIQL can achieve comparable performance when compared to IQL+HER. * For the experiments, I encourage the authors to report the performance of the low-level policy without the high-level policy. This can effectively highlight the true usefulness of the hierarchical structure. * There is an error in Eq (7) in which the variable g is mistakenly used to train the low-level policy, while $s_{t+1}$ in the expectation is not used. [1] Hong Z W, Yang G, Agrawal P. Bilinear value networks[J]. arXiv preprint arXiv:2204.13695, 2022. [2] Chebotar Y, Hausman K, Lu Y, et al. Actionable models: Unsupervised offline reinforcement learning of robotic skills[J]. arXiv preprint arXiv:2104.07749, 2021. [3] Li J, Tang C, Tomizuka M, et al. Hierarchical planning through goal-conditioned offline reinforcement learning[J]. IEEE Robotics and Automation Letters, 2022, 7(4): 10216-10223.

Questions

* Additional explanation is needed to clarify how the hierarchical structure can be applicable in continuous situations, as well as a comparison with smoothed Q methods * More information should be provided to enable the reproduction of Figure 2. * Additional baselines for temporal difference learning and goal relabeling should be included in order to facilitate a more comprehensive and fair comparison. * Authors are encouraged to report the performance of the low-level policy in order to emphasize the necessity of a hierarchical structure. * Is it possible to expand the HIQL framework to include more than two levels? Considering that the parameter k is manually determined, the top level could be assigned a large k value, while a medium level would utilize a moderate k value. * A fix in Figure 1 and Eq (7) may be needed. ## post rebuttal I am delighted to see that the authors have addressed all of my concerns, and I am pleased to raise my score for this paper.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

One limitation mentioned in the paper is the necessity for deterministic dynamics. It would be beneficial for the authors to further elaborate on why disentangling the controllable aspects from the uncontrollable elements in the environment is advantageous.

Reviewer ZY687/10 · confidence 4/52023-06-28

Summary

This paper introduces a hierarchical approach to address the issue of offline goal-conditioned problems, specifically when certain trajectories in the dataset have missing actions. The proposed method tackles this challenge by simultaneously learning a value function with a modified version of IQL and extracting the high-level and low-level policies of the learned value function. The high-level policy predicts the representation of waypoints, while the low-level policy, conditioned on the waypoint, determines the primitive actions. Extensive experimental results on offline goal-conditioned tasks demonstrate the effectiveness of the proposed approach.

Strengths

1. The writing is very easy to follow. 2. The paper proposes an elegant approach to goal-conditioned offline RL problem by decomposing the high-level and low-level policy while sharing the same value functions V. 3. The approach taken in this paper allows the adoption of the existing large amount of action-free data. 4. Good empirical performance on a comprehensive set of evaluation environments with both state and pixel observations.

Weaknesses

For some of the design choices adopted in this paper, I think there are some alternatives in the literature that are good to compare with. See my comment in the question section.

Questions

1. (**Clarification Question**) I am confused by the notation $V(s, \phi(g))$, $\pi(a|s, \phi(g))$. I understand that for the high-level policy, the waypoint it predicted is the latent representation, not the pixel values of the image. But is $s$ also represented by the same state representation/encoder $\phi$? So is it $V(\phi(s), \phi(g))$ (or with some stop gradient on either $\phi(s)$ or $\phi(g)) $? 2. (**How to learn the value function $V$**) In this paper, the author adapts the approach of the IQL algorithm to learn the value function V (with some modification to account for missing actions). But this is not the only approach to learn the value function. [1] proposes to learn the value-function V by dualizing the objective, which also does not require action labels in the dataset. Maybe it would be nice to compare this approach and see which approach gives better quality of the value function, since it is crucial for the extraction of high-quality policies. [1] Ma et al. VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

N/A

Reviewer cj3e7/10 · confidence 3/52023-07-03

Summary

The paper identifies an issue with offline goal-conditioned reinforcement learning: namely that goal-conditioned value estimation can be noisy, so long-horizon tasks can be difficult to accomplish due to accumulating errors in value estimation. In addition, when states are so close together, there is very little signal for learning as any mistakes can be corrected in future states. Thus, HIQL is proposed, which learns a high-level goal-conditioned policy to predict subgoals (such that there is sufficient learning signal), and a low-level goal conditioned policy to predict actions, which benefits from only considering nearby goals. Experiments show the advantages of this approach with respect to baseline methods in hierarchical and flat offline RL, as well as the ability to use unlabeled (without action) trajectory data to improve learning, and data efficiency.

Strengths

- The argument is extremely clear and well exposed - I appreciate the demonstration of issues with current methods (Fig. 2, 3, 4) before introducing the solution, which helps my understanding of the field in general. Such a simple demonstration, though perhaps not exactly well validated, may even be worth more as expository writing than the final proposed method. - The method is simple, in particular reuse of internal representations instead of learning some VAE or something more complex is nice. - The ability to have different data requirements for different parts of the policy is quite nice, given that action labels are harder to come by (Table 3).

Weaknesses

- Though the exposition is very clear, sometimes it is a bit repetitive. In particular, the same justification for hierarchy is pointed out in line 36 and 160, as well as the repetition between sections 4.2 and section 5, though I can see how this is a matter of personal taste. - The difference in domain between Figure 2 and Figure 3 is a little bit odd, though this isn't a huge issue. The domain in Figure 3 is maybe a bit _too_ toy, which leads me to discount Proposition 4.1, the major point was already in Figure 2. - I'm a bit suspect of the claim that Procgen Maze is a good demonstration for a visual environment, given that the visual observation is already so structured. It would be nice to work on a more complex task within Procgen, like Coinrun, though I'm unaware of the data availability for these tasks. - Proposition 5.1 is unnecessary in my opinion, that or just a simple proof sketch could be given inline. - I think a more in-depth discussion of particularly related work is merited. Novelty is not very clear to me from the given presentation. In particular the proposed method is so simple that it's surprising to me that it does not already exist.

Questions

Besides the points in the weakness section, it's a bit unclear to me the novelty of this submission, so I would appreciate some further contextualization to other hierarchical methods that take advantage of data.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

Limitations are addressed.

Reviewer xrL87/10 · confidence 5/52023-07-07

Summary

This paper introduces hierarchical IQL (HIQL) for offline goal-conditioned reinforcement learning. HIQL uses the IQL algorithm to learn both a high-level waypoint policy as well as a low-level action policy; in both case, the goal-conditioned value is provided by the same value function learned using goal-conditioned IQL procedure. HIQL is evaluated on both state-based as well as image-based environments that require hierarchical planning. In all settings, HIQL outperforms prior methods

Strengths

Originality: This paper builds on IQL, which has already been extended to goal-conditioned settings, though not in the hierarchical manner this paper proposes. Furthermore, the use of single value function is a nice idea. The didactic examples and theoretical results are also interesting and contribute to the overall story. Therefore, I believe this paper contains enough original components for NeurIPS. Quality and Clarity: This paper is very well written and presented. The experiments are rigorously executed and explained. Significance: Offline GCRL is an emerging and important topic to study, and this paper proposes a conceptually simple and effective algorithm for the setting.

Weaknesses

This is a strong paper without major weaknesses. There are several places to further improve the paper: 1. An ablation that demonstrates that AWR is needed for both policies. I suspect that it could be possible that it is not super important for the low-level controller if the horizon is short enough. 2. Connection with existing approaches that learns a high-level planner and uses an inverse dynamics model to back out low-level actions can be better drawn in related work. 3. I believe more substantive error bound in the form of policy performance can be derived in addition to Proposition 4.1; [1] makes one such attempt. [1] Ajay, Anurag, et al. "Opal: Offline primitive discovery for accelerating offline reinforcement learning." arXiv preprint arXiv:2010.13611 (2020).

Questions

I have listed several suggestions in the section above. Overall, I think this is a solid paper, and I recommend acceptance.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

4 excellent

Presentation

4 excellent

Contribution

2 fair

Limitations

N/A.

Reviewer tvKa7/10 · confidence 4/52023-07-08

Summary

The paper proposes HIQL a hierarchical algorithm for offline goal-conditioned RL. The approach consists in utilizing a single action-free value function to acquire knowledge about the structure and employ two policies: a high-level policy that predicts or represents a waypoint, and a low-level policy that predicts the action required to reach that waypoint. The approach is well explained and motivated. The experiments comparing most state of the art approaches of offline-RL on the main benchmarks seem convincing.

Strengths

Reasonably novel approach, well explained and illustrated with convincing results on main offline RL benchmark of locomotion and manipulation against reasonable baselines.

Weaknesses

Maybe performing TT on kitchen as baseline would have been useful to improve the comparison part. Otherwhile, no major weakness in my opinion.

Questions

Why not using the TT in the kitchen scenario for comparison ?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Nothing noticeable IMO.

Reviewer ZY682023-08-10

Thanks!

Thanks for the response. I appreciate the additional experiments on two pixel-based environments as well as including more baselines into the comparison. I will be sticking to the deserved high score that I have given the paper.

Reviewer cj3e2023-08-10

I thank the authors for their clarification on novelty, especially as I am somewhat unfamiliar with the prior work. They should take my original comment on surprise at the method being novel as a compliment. Simplicity is always appreciated. I also appreciate the additional visual domain experiments, which are much more convincing to me than the original maze domain. I see that the conclusions in the original paper remain undisturbed. I don't have any additional concerns at this time, and I think the original score of 7 that I gave is merited.

Reviewer kqhw2023-08-11

Thank you for the response

Thank you for providing such a detailed response. After thoroughly reading your explanation, I am pleased to say that all of my concerns have been addressed. The clarification of "IQL+HER" and the additional comparisons have significantly strengthened the paper's persuasiveness. I found the discussion on the multi-level hierarchical structure particularly intriguing, and I hope to see it included in the revised version of the paper. Overall, I am now inclined to raise my score to 7, and I am delighted to see this work accepted by the conference.

Reviewer xrL82023-08-15

Thank You for Your Response

Dear Authors, Thank you for your response. I have carefully read it and am satisfied with the new experimental results that justify the design decision of using AWR at both levels. Given the initial high rating, I will keep my original score as I believe that this paper merits acceptance at NeurIPS.

Program Chairsdecision2023-09-21

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC