Omnigrasp: Grasping Diverse Objects with Simulated Humanoids

Summary

The paper introduces Omnigrasp, a method for controlling a simulated humanoid with dexterous hands to grasp and manipulate a wide variety of objects along complex trajectories. The authors leverage a universal humanoid motion representation to improve training efficiency and scalability. This method achieves state-of-the-art success rates in object manipulation tasks and generalizes well to unseen objects and trajectories. Key contributions include a dexterous motion representation, the allowance for simple state and reward designs for training, and high success rates in diverse object manipulation scenarios.

Strengths

The challenging yet important topic regarding 1. a full-body dexterous grasping, which considers the physical constraints and unstability of the body; 2. omnidirectional movement after grasping, i.e., moving the object along any directions within a reachable range; is really worth the exploration. The paper not only presents an effective algorithm that does not heavily rely on existing object trajectories but also conducts experiments on extensive objects to evaluate grasping success rates.

Weaknesses

While the paper marks a significant advancement in humanoid dexterous grasping, some inherent flaws in the setting hinder its seamless transfer to real-world scenarios. I.e., some state variables, such as the object mesh, pose, and velocity, cannot be accurately accessed in the real world. This discrepancy between simulation and reality poses a challenge for the practical deployment of the proposed method.

Questions

1. How do you plan to bridge the gap between simulation and real-world applications? Given that the input assumes access to the object pose, velocity, and mesh, which are impossible to obtain in the real world. 2. How do you envision the way your method could be adapted for, or at least benefit, downstream tasks? E.g., more fine-grained dexterous manipulation.

Rating

7

Confidence

4

Soundness

3

Presentation

4

Contribution

4

Limitations

1. The approach is validated only in a simulated environment, which may not capture all real-world complexities and variabilities. 2. The method assumes accurate state estimation for object meshes, poses, and velocities, which is challenging to achieve in real-world scenarios. 3. Fine-grained dexterous manipulation is yet to be achieved by this method.

Summary

The paper introduces a method for controlling a simulated humanoid robot to grasp and move objects along a trajectory with the use of a dex hand, This approach enables the robot to handle diverse objects with diverse trajectories. The key contribution is a humanoid motion representation that enhances training efficiency and human-like manipulation skills and applies it to grasping tasks. The method achieves a reasonable success rate in completing object trajectories and generalizes well to unseen objects.

Strengths

* This work extends previous research on PHC with dexterous hand capabilities, enabling whole-body manipulation in simulation with a high success rate. The results, including supplemental videos, demonstrate human-like motion for object grasping and trajectory following. * Experiments show that the proposed controller exhibits reasonable robustness and generalization ability across objects of different scales. * Extensive ablation studies validate the effectiveness of various components, providing plausible analysis and explanations for comparisons, such as why experiments without object shape still achieve reasonable grasping performance. * The paper is well-written, easy to follow, and provides sufficient technical details, including supplemental code, for the community to reproduce the results.

Weaknesses

* Despite extensive ablation studies, the work lacks some important baselines, such as fully end-to-end RL with similar compute (R1 in ablation), considering that training PHC-X and PULSE-X takes half a month. Baselines utilizing human motion prior data in different ways, such as AMP or ASE, are also missing. * More details about the dexterous hand should be included, given its significance over previous work on PHC. Comparisons with experiments using PHC but not PHC-X, such as using PHC for body control and raw action space for the hand, are needed. The ablation study compares with motion prior to training without Dex-AMASS, but it’s unclear how the hand joints are controlled, and more details are necessary. * [Minor - Nothing significant]Though the authors claim the work can be extended to real robots in supplementary material, it seems unlikely due to the low simulation frequency (leading to low simulation fidelity), unrealistic robot methodology design, and the use of privileged information.

Questions

* Considering most of the human body motions and hand motions are randomly paired from different datasets, why not train the model separately (PHC for body motion and another model for hand motion)? What additional benefits does modeling them together provide?

Rating

5

Confidence

5

Soundness

2

Presentation

3

Contribution

2

Limitations

The authors include a reasonable discussion of the limitation and no obvious potential negative societal impact needs to be discussed.

Summary

This paper proposes an approach to learning a humanoid controller that can manipulate objects to follow trajectories. It first assembles a dataset of human bodies and hands motions, and learns a control policy from the state transitions in the dataset. Then, they distill this policy using a VAE to obtain an action decoder that models realistic human action distributions. After that, they use this distribution as the action space to learn an RL policy whose reward is based on object trajectory following. With this action space, the neural network policy can output realistic actions and avoid exploration difficulties. The authors demonstrate several impressive, yet natural and physically plausible motions for a simulated humanoid to grasp and move objects according to trajectories.

Strengths

The results are generally good, and the motion is relatively natural and physically plausible. The demonstration of extending this approach to high-dimensional control is important. The authors also provide several insightful observations, such as how to assemble the dataset to extend PULSE to human hand motion and how to achieve diverse grasping behaviors. These contributions are all important to the community.

Weaknesses

While I definitely think this paper is good, I believe there are several critical aspects that should be more carefully evaluated, either qualitatively or quantitatively. I will provide the summary comments in this section and write the specific questions that I would like answers to during the discussion phase in the next section. First, I’m generally curious about the robustness of the proposed system. The object trajectory following policy takes a few inputs without noise from the simulator (joint positions, object latent code, proprioception). However, in reality, these quantities are far from perfect. Second, this approach assumes the humanoid is similar enough to the mocap data. I agree this is a reasonable assumption. However, in reality, humanoid robots and their hands are usually not perfect replicas of humans. How similar do their morphologies need to be? Regarding the learning approach, it first learns a policy to output actions given the human sequence data. Then it distills this policy network using VAE to learn the distributions as the action space. There should be an alternative approach that can serve as a simple baseline (details below). This is not discussed (or I missed the reference?). The current reward contains several terms with different reward weights and hyper-parameters. I would not say this is a simple reward, especially without a concrete comparison to previous work.

Questions

Regarding the robustness of the system: How would the policy perform in noisy situations? For example, when the joint positions are not accurate (real robots will have imperfect joint encoders). And how accurate does the object mesh need to be? How important is the morphology similarity between the humanoid and the hand? For example, do they need to have the same number of joints or links? I also observed even in the current system, it seems the simulated humanoid does not perfectly match human kinematics. Is this correct? If so, how do you bridge this gap? Regarding the simple baseline alternatives: * Instead of learning actions from human trajectory sequences between two timesteps, why not directly using a VAE to reconstruct human poses? This VAE will directly learning a distribution of human joint positions. Then in the RL policy training, you can map the policy output to natural human joint positions using this VAE. * Even when one wants to learn the “actions” instead of “joint positions”, one could use a encoder-decoder structure (either VAE or vanilla auto-encoder) as the policy backbone and directly learn in the PULSE-X phase, instead of separating it into two stages. I’m curious if the authors tried these alternatives before? Or does this comparison exist in the literature? Are they performing not well? It would be much helpful to analyze this problem or provide references in the text. A more comprehensive discussion on the failure cases would be helpful. You show several videos of the failure cases, which I think is very helpful. But why do they fail? Is it because the objects themselves are quite difficult? Regarding this point, it would also be much helpful to have a per-object accuracy analysis.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors describe the limitations mainly as not performing dexterous in-hand manipulation. The authors could consider using recent learning-based dexterous in-hand manipulation work as the “mocap dataset” in your formulation and learn action distribution using it. I think there is another limitation, which is that the current formulation does not demonstrate capability when the object needs to interact with the environment. While I acknowledge that the above two points are out of scope and will not affect my assessment at all, they might be interesting future directions.

Thank you for your comments and response!

The authors appreciate your suggestions and Accept recommendation. Please let us know if you have any additional questions!