HumanVLA: Towards Vision-Language Directed Object Rearrangement by Physical Humanoid

Summary

This work aims at building a vision-language-action (VLA) model that can control a humanoid to interact with dynamic objects without ground-truth state information. The proposed method first trains a teacher policy with ground-truth information. It then distills the policy to a VLA model via imitation learning. Additionally, the paper presents a new dataset and evaluates the paper against a recent baseline.

Strengths

+ The work is well-motivated and addresses two fundamental challenges: interacting with dynamic objects and the lack of ground-truth state information in real-world scenarios. + The method works well based on the demo videos and the quantitative evaluation. Each component of the method is well-motivated. + There are sufficient details about the method and the dataset. + The dataset can be a valuable resource for future research.

Weaknesses

- The paper combines many existing methods together and builds a very complex system. It introduces some interesting components such as active rendering, but it is unclear how general the system is for a broader class of synthesis. For instance, can this system be applied to any humanoid-object control? What about one humanoid interacting with multiple objects (e.g., one agent carrying two objects in both hands) or multiple humanoids interacting with the same object (e.g., two agents carrying a sofa)? Can you extend the object category to those that are unseen during training? - Related to the first point, there is a lack of discussion on failure and limitations. - It would be good to show videos of the baseline results for a clearer comparison. - What is the sim2real gap? I assume that the end goal is to control a real-world humanoid to complete the tasks?

Questions

Please address the points in the weaknesses. Additionally, I have the following questions: 1. Could you please comment on whether your method can be suitable for human 3D body motion synthesis (e.g., SMPL-X like the synthesis in OMOMO)? 2. Would it be possible to directly train VLA to simplify the training procedure? 3. Could you apply your method to longer activities that involve a sequence of goals? If not, what could be missing?

Rating

6

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

There is not much discussion on limitations.

Summary

In this paper, the authors address the task of room arrangement with a humanoid agent and propose a teacher-student framework to achieve vision-language guided object rearrangement. The teacher policy utilizes privileged state information, including object states, goal states, and waypoints derived from A* planning. Goal-conditioned reinforcement learning together with AMP is employed to train a human-like policy that guides the humanoid agent in completing the task. This teacher policy is then distilled into a student policy, which relies on high-level language instructions and ego-centric view images instead of ground truth state information. A DAgger-like training paradigm is used, and an active rendering technique is developed to focus the camera on objects, ensuring informative ego-centric view images. The authors construct an object rearrangement dataset as a test benchmark for the proposed framework. Experimental results on this benchmark demonstrate the effectiveness of the framework, although generalization to novel tasks and environments remains challenging.

Strengths

1. The writing is clear and easy to follow. 2. This work is addressing a very promising problem setup - the visual-language guided policy learning, and it can have the potential to be applied to the real humanoid. 3. The build dataset and benchmark can facilitate the community research in human-scene interaction.

Weaknesses

1. The major methodologies employed in this work, including the AMP+goal-conditioned RL and teacher-student policy distillation were widely-used in many previous works, but still it is good to see this new application in the humanoid room arrangement task. 2. More investigations into the generalization ability of the current pipeline is desirable, see more in the question section.

Questions

* Regarding the teacher policy: could you justify which modules or techniques help in achieving (slightly) higher success rate, better precision, and less execution time than the InterPhys baseline? Since the proposed teacher policy training paradigm share the similar training paradigm in the InterPhys, but it is not justified clearly about the major differences between the InterPhys and the teacher policy implemented in this work. * Regarding the student policy: * Without the waypoint information, how is the agent capable of searching and navigation? Does this mean that the agent learns to recognize objects via the vision-language distilling? * The active rendering technique is suggested to encourage the camera to focus on the object state, but does this still access to the privileged object state information? * It is good to see the discussions on the generalizing performance to unseen tasks, and the performance kind of actually makes sense, and I believe it would provide more insightful views if we can investigate how can the policy generalize to unseen scene layouts, unseen visual appearances, unseen language instructions, unseen object geometries individually. * The sphere-like hand can limit the capacity of the agent, and the authors also mentioned that one future work can be include dexterous manipulation. Could the authors give some comments on what additional challenges could be by involving the dexterous manipulation skills?

Rating

6

Confidence

4

Soundness

3

Presentation

4

Contribution

3

Limitations

Yes, the authors have addressed the limitations,

Summary

This paper proposes HumanVLA, a framework for training humanoid controllers powered by vision and language. First, a teacher control policy is trained to control a simulated humanoid to carry objects to specific positions. Then, this policy is distilled into a vision-language-action model that uses vision to guide the movement of the humanoid and objects. A few techniques such as active rendering, reward balancing, etc., are proposed to improve performance; a dataset containing sequences of object-carry locomotion is also proposed.

Strengths

- This paper paints a very promising picture for simulated humanoid control and vision-language models. The formulation (vision + language + proprioception) provides an enticing research direction for embodied AI. - The proposed solution (teacher-student) is a well-tested formula for humanoid control and is performing well in the proposed task. The vision element has not been explored much for humanoid control due to its high dimensionality but is incorporated in the current pipeline. - The proposed curriculum learning, reward clipping, etc., while small innovations, contribute to the overall performance of the pipeline. - Experiments show that the model achieves a high success rate compared to previous SOTA (InterPhys) in box arrangement when given privileged information. The success rate for the vision-language-action model is also promising.

Weaknesses

- The major weakness of this work is qualitative evaluation. Only four video demos are shown without any information about the text prompt or egocentric rendering. Thus, there are limited ways to know how well the proposed VLA method performs in terms of generalization. - Along this line, since the evaluation dataset is provided by the authors, the diversity of the text prompts & tasks is unknown and not demonstrated. - The VLA part of the proposed method is relatively weak and understudied. Given only BERT text encodings, it is very hard to imagine that the MLP-based agent could complete complex tasks without any guidance. How does the agent know where the target location is? Does the agent start with a scanning phase where it locates where the target position is? For such a low error in location placement (~20cm) and such a coarse language instruction, the only way the MLP-based agent could succeed is by memorizing the training data. - The proposed VLA agent has no memory and no planning capability, and the vision part essentially acts as an object classifier. The language instruction performs a similar role. Also, the active rendering encourages the agent to always look at the object, so there is little way it can interpret the scene layout or plan paths. In order to really prove “generalization,” truly unseen instructions and scenes need to be shown, and its success rate separately reported. At the moment, there are no real indications that the proposed method can generalize to unseen scenarios. - L20: “. specific object dynamics, such as carrying a box [15] and throwing a 22 ball [47].” - the proposed method also only handles one type of interaction, carrying objects with two end effectors. Overall, I feel like this work shows great potential in providing a task formulation for embodied AI; however, it is a little overclaimed at this moment in terms of its language and planning capabilities. I recommend scaling down on the formulation (e.g. focus on known scenes and objects) instead of claiming that it can tackle unseen rooms and instructions.

Questions

How does the agent plan its path around a new room if it can is encouraged to always look at the object? The environment-awareness and target-location awareness of the agent is not properly addressed.

Rating

6

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

Limitations on the language instructions side is not adequately discussed.

Thanks for the responses

I appreciate the responses. They ahve addressed my questions. I have raised my rating to weak accept. I urge the authors to incorporate the discussions in the responses in the final version and also provide video demos of different methods.