Grounded Decoding: Guiding Text Generation with Grounded Models for Embodied Agents

Summary

The paper introduces a novel method to tackle a key challenge in utilizing Internet-scale knowledge from LLMs (Language Model Models) in robotics tasks. Specifically, the focus is on aligning the instructions generated by LLMs with the actual actions performed to manipulate objects. The paper endeavors to frame this issue in a manner analogous to probabilistic filtering. This framing aims to decode a sequence that exhibits both high probability according to the language model and high probability according to a predefined set of objectives or attainable skills in the grounded model.

Strengths

The paper introduces a novel formulation focused on decoding language models in a robot-centric manner to facilitate long-horizon robotics tasks using token-conditioned grounded models. Additionally, the paper provides empirical evidence showcasing the efficacy of this approach in comparison to CLIPort, which utilizes CLIP as an open vocabulary knowledge-based grounding technique combined with the Transporter Network. Furthermore, the paper demonstrates the transfer-ability of this idea to various domains, including 2D mazes and mobile manipulation.

Weaknesses

1.The paper could benefit from improvements in certain sections of the writing. For instance, there is an awkward expression in the phrasing of the sentence on lines 116-117. 2.It would be valuable if the paper included a comparison with other techniques, such as Grounded SAM, which involves detecting objects in the scene and using them as input for LLMs to enable grounded reasoning. This alternative baseline would provide a more robust comparison for the proposed approach, as it still allows for the use of a planner to generate the task plan. 3.The author's efforts to demonstrate the effectiveness of their approach across different domains are appreciated. However, it would be advantageous if the author conducted real-world experiments that are reproducible with a broader range of robots and baselines.

Questions

1. Would like to see comparison with other methods using detection to learn the labels of the object, and feeding those into the LLMs for grounded reasoning. 2. Implementation on more general robot framework for reproducible.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

2 fair

Limitations

The authors have address the limitations adequately.

Summary

The paper introduces Guiding Text Generation with Grounded Models for Robot Control. Previous works have shown that large pre-trained language models can generate long action plans for robots based on abstract instructions. However, these models struggle to handle unstructured environments with large action spaces. The project aims to enable language models to generate control commands in a grounded and efficient manner. The paper explains how auto-regressive language models generate text by scoring all possible tokens based on their likelihood of completing the current action step. The approach incorporates affordance, preference, and safety functions to create grounded models. The decision-making process of selecting tokens is illustrated using the example of stacking blocks. The work also demonstrates how the method can generalize to different tasks without additional training. The flexibility of grounded decoding allows for including objective functions and constraints on the fly. The project concludes by discussing the potential applications of grounded decoding in other embodied tasks and the possibility of using advanced search methods for better performance.

Strengths

The work demonstrates its strength through plenty of experiments. By conducting numerous experiments (Embodied environment, maze, robot control in real space), the researchers provide a comprehensive evaluation of the proposed method and its performance in various scenarios. This enhances the robustness of the findings and allows for a better understanding of the method's capabilities and limitations. The work takes into consideration the environmental conditions during planning. By incorporating affordance functions, preference functions, and safety functions, the grounded models consider the feasibility, preferences, and safety aspects of actions within the given environment. This ensures that the generated action plans are not only coherent with the high-level goals but also take into account the specific constraints and requirements of the environment, leading to more practical and context-aware robot control. The key idea of the grounded model is well-illustrated. Through clear and concise explanations supported by visual representations and examples. By effectively communicating the core concept of the grounded model, it is easier to understand how the model works and how it addresses the challenges of generating control commands in a grounded and efficient manner. The illustrations help to visualize the decision-making process and the interaction between the language model and the affordance model, contributing to a more intuitive grasp of the proposed approach.

Weaknesses

Even though the paper is titled with `Robot Control`, it lacks discussion of the robot motion planning considering the environment; i.e. how the robot motion planning changes when involving the grounded model. Especially, comparing the robot motion planning between Saycan and this work can be more persuasive. The motivation for involving grounded information is not sufficient. It seems that the grounded model is a direct fusion of affordance map derived by CLIPort, while the whole system is equipped with template language instructions. A more straightforward approach would be using CV model to detect the object in the scene and then include the scene description in the prompt. For example: pick up the fruit in the scene, given `orange, apple, potato, ...`. The key idea for considering the grounded map should be that the robot motion needs to think about its configuration, safety, environmental limitations, etc. However, those facts seemed to be disregarded.

Questions

(1) What is the task size and the robot search space? (2) Why the 2D Maze experiment matters since it is unrelated to robot control

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

2 fair

Limitations

The limitations are mentioned; No societal impacts found

Summary

This paper studies grounding Large Language Models (LLMs) for robot control. The planning tokens are generated by the joint probability of an LLM and grounded models sequentially. The grounded models can be RL value functions, multi-modal perception models or human rules. As an extension of SayCan, this method allows open-vocabulary control and combining multiple grounded models. Experimental results in two robotic environments and a maze environment demonstrate the effectiveness of the proposed method for solving long-horizon complex tasks.

Strengths

1. The paper is well-written, with relevant references and easy-to-follow narration. 2. The paper conducts experiments in various domains and tasks, showing that the proposed grounded decoding method can be widely applied in robotic tasks.

Weaknesses

1. The contribution of this paper is mainly incremental to SayCan. Though it allows open-vocabulary planning, the key difference is that SayCan requires a fixed set of low-level skills while this paper requires a language-conditioned low-level policy. 2. For the proposed 3 types of grounded models: token-conditioned value functions are similar to the skill affordance in SayCan. Multi-modal foundation models and rule-based methods are more like technical tricks in applications rather than a systematic method.

Questions

1. For the results in Table 3 and 4, does SayCan use the same LLM to your method? 2. According to Table 4, SayCan costs a large number of tokens since it computes scores for all the low-level skills. In its setting where a fixed set of low-level skills are provided, can we use the LLM to output open-vocabulary tokens and match the skill names using text embedding's similarity, to decrease the token cost of SayCan?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors discussed the limitations in the paper. Some of these are addressed and others are left to future work.

Summary

This paper proposes Grounded Decoding, a methodology to ground the high level semantic plans output by an LLM into a realizable action sequence. Specifically, the authors construct a model which decodes a sequence that is likely to be realizable given the current state of the robot and environment, but also likely according to the language model. Experimental results in both simulation and the real world demonstrate the utility of the proposed method over baselines from prior work.

Strengths

- Well-written. The paper is well-written, and the figures aid in the understanding of the methodology. - Strong experimental results. Show the utility of using both the ungrounded LLM and their Grounded Decoding relative to prior work on a range of tasks both in simulation and the real world. Outperform SayCan in terms of efficiency by a non-trivial margin. The inclusion of the qualitative results is also useful. - Ablations. The experiments in simulation include ablations which isolate the contributions of the proposed grounded decoding module. Furthermore, the authors include ablations demonstrating the impact of the choice between greedy search and beam search, which is insightful. - Inclusion of failure mode analysis. The analysis on the failure modes of the system is insightful.

Weaknesses

- Limited experimental evaluation. The authors include results on just two environments in simulation and one in the real world. Showing results on a larger number of environments would make the claims more convincing. - Lack of comparisons between grounding functions. Section 3.4 proposes three different techniques for obtaining grounded models that can be leveraged in GD’s formulation. However, the three techniques do not seem to be directly compared in the experiments. An experiment of this form may provide insight about which formulation works best for the proposed algorithm. Furthermore, how do the different grounding functions (AF, S, P) impact performance in the experiments? - Ground truth map for navigation. The navigation module used in the mobile manipulation in a physical kitchen experiment is scripted and uses the ground truth map of the environment for navigation. This is a major assumption and limits deployability to only seen environments. - Prompt engineering. It is claimed that a certain amount of prompt engineering was done to steer the LLMs. This is an undesirable property of the proposed system. How sensitive is the system to such prompt engineering? Was the same effort also applied in prompt engineering for the baseline methods?

Questions

- In Figure 3 (leftmost), why is the grounded model's score for _orange higher than that of _red, when the red block is visible but there is no orange block present? Similarly, why is the language model's score for _yellow so high in the rightmost plot?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

The authors include a detailed section on the limitations of the work.

Summary

This paper proposes Grounded Decoding (GD), a method that filters the decoded task descriptions by the probability of a grounded model, LLM-based task planning. This paper shows the framework can integrate different types of grounded models. Experiments across physical and simulated environments show that this decoding method outperforms prior models in instruction following tasks.

Strengths

- The introduction of grounding functions covers a family of methods that evaluates the likelihood of the generated text plans conditioned on the current states. - This paper presents the method clearly and it is easy to follow. - The inclusion of grounding in the decoding process reduces the planning failure. ----- Post-rebuttal update The authors' responses answered my questions. I agree that GD is a more flexible framework and can be more easily extended to include various grounding functions as opposed to SayCan. I would encourage the authors to include the rebuttal responses to the paper to improve clarity.

Weaknesses

- While it is possible to include safety and preference in the grounding function, the current form to include safety and preference is quite preliminary. It only considers the presence of objects but usually, these should be represented as statements. - It is a good modification of SayCan to constrain the grounding in the brackets. It is unclear how reliable the ungrounded decoding and activate or revert grounding at the right time and how this modification fits the formulation of GD (Eq. 7). - The grounding function can be used as the affordance function in SayCan. If we consider the grounding function this way, the main advantage of GD is its efficiency not the introduction of the grounding function.

Questions

- The results show the overall success rate. How does it perform given the safety and preference grounding? - There is a huge gap between planning and execution in the real robot experiment. What causes this big gap? - In Fig 5, why doesn’t include SayCan’s failure breakdown? - Fig 6 is quite confusing, the description says each dot is an instruction and they are colored by the affordance value computed against four different scenes. How are those scenes selected and why some instructions are infeasible if they are meaningful instructions for that domain?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

2 fair

Limitations

The appear discussed the limitation of the proposed method well. The joint decoding may be limited as the grounding models are limited and most of them are object-centric grounding.

Thank you

Thank the authors for response, and clarifying my doubts. Would take all these into consideration.