CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics

Summary

This paper works on learning multi-human cooperative object manipulation, specifically the collaborative carrying of objects. This paper proposes a two-stage method to learn collaborative object carrying. In the first stage, the agent learns single-person object carrying from motion capture data and heuristic-based task rewards following the Adversarial Motion Priors (AMP) framework. In the second stage, the agent policy is fine-tuned using multi-agent reinforcement learning to learn multi-agent collaborative object carrying. Experiments show that the trained policy can successfully control the agents to complete collaborative objects carrying tasks given objects of varying shapes and weights.

Strengths

1. The proposed cooperative interaction learning framework can learn multi-human collaborative behaviors requiring only single-human motion capture data. In addition, the learned policy has better generalization compared to tracking-based methods. 2. The implicit communication via object dynamics unifies the observation in the single-human and multi-human stages, reducing the gap between the two stages and facilitating the learning of collaborative object carrying. 3. This paper presented extensive ablation study and boundary analysis to show how and to what extent the proposed method works.

Weaknesses

1. This paper simulates an oversimplified humanoid without hand modeling considering the fact that it works on object manipulation problems, for which the hands play essential roles. The authors also recognize this issue leads to capacity limitations. 2. The agent observation features may not be sufficient for more complex manipulations. In the presented paper, the agent observation mainly consists of the self-motion and cropped object bounding box features. However, more complex collaborative tasks like carrying articulated objects and assembling a sofa require a global overview of the object instead of only a local observation. In addition, the agent also needs to observe the collaborators to avoid interference such as grasping at the same point. 3. The proposed method needs to train a separate policy for a category of objects with similar shapes (L246-247), which indicates a generalization problem.

Questions

1. How the object weights can affect the humanoid in the simulation? Is there an upper bound for the joint force of the simulated humanoid? How does the humanoid apply forces to the object in the simulation? 2. Are the standing point and held point also part of the goal inputs? If not, during the training, the same motion sequence can have different rewards given differently sampled standing points? How are the standing points sampled given an arbitrary object mesh?

Rating

7

Confidence

5

Soundness

4

Presentation

4

Contribution

4

Limitations

The author discussed the limitation of lacking dexterous hands, which limits the agent interaction capacity and is left as future work. Please also refer to the weakness discussion above.

Summary

This work address the problem of multi-character collaboration for object transporting tasks. Different from previous works approaching the multi-character HOI task with tracking-based method, this work learns a physics-based multi-agent policy with reinforcement learning. Instead of directly training a multi-agent policy from scratch, the authors suggest using a two-phase learning approach: CooHOI uses a two-phase learning approach: individual skill acquisition through Adversarial Motion Priors (AMP) and then collaborative learning with Multi-Agent Proximal Policy Optimization (MAPPO). Agents learn to coordinate implicitly by responding to changes in object dynamics caused by others.

Strengths

* The manuscript is well-written, the methodology is clearly presented and easy to follow. * This paper is addressing an under-studied and important application - the physics-based multi-agent cooperation HOI, where most previous works explores in the single-agent setup. * The proposed two-state learning pipeline is reasonable and few important designs including the stand point, held point shows to be effective in addressing this multi-agent cooperation tasks. * The ablation study and analysis provide comprehensive analysis and capacity boundary test for the proposed pipeline and these provide useful insights for the community.

Weaknesses

* This paper build a framework for multi-agent cooperation policy learning with AMP for single-agent policy pertaining and MAPPO for cooperation skill learning. The technical contribution to the community is minor. * The method include few heuristic designs to simplify the problem, for example, the manually defined held point, and object bounding box observation, making it not easy to generalize and scale to more diverse objects. * The current framework can be hardly generalise to objects with different size and different shape, and the human motion styles are quite limited. * Regarding generalization to objects weights and scales, are the results presented in Figure 5 the transferring results to different weights and scales or the per-object training results. * Regarding the motion styles, in the training dataset, limited motion sequences are involved with only walking, lifting up and putting down, would the proposed method build upon AMP be able to cover more diverse skills, and this could be helpful to generate more natural and diverse HOI motions. * It is presented that when increasing the number of characters, the policy has problem in putting the object down due to the absence of dexterous hands and limited friction. Could the author elaborate more on the major bottlenecks encountered when dealing with a larger number of characters? Assuming that the friction parameter can be adjusted in the simulator and should not be a significant issue for motion generation tasks purely within the simulation, what specific contributions could dexterous hands make to this scenario? Additionally, while the introduction of dexterous hands would undoubtedly enhance the agent's capabilities, what other factors should be considered?

Questions

See more in the weakness section :-)

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes, the authors addressed the limitation.

Summary

In this paper, the authors introduce a novel framework, Cooperative Human-Object Interaction (CooHOI), aimed at tackling the problem of multi-agent object transportation. The framework consists of two phases: initially, a single agent learns to perform tasks, followed by multiple agents learning to collaborate through shared dynamics of manipulated objects. The utilization of shared object dynamics has proven effective in learning cooperative tasks. The authors conduct comprehensive experiments to validate the efficacy of their approach and explore its capability boundaries.

Strengths

* The idea of sharing manipulated object dynamics among multiple agents as implicit communication to facilitate multi-agent collaboration, is intuitively appealing and has proven effective. This approach aligns with object-centric concept, designed to leverage object dynamics to guide agent actions and promote collaboration. It provides valuable insights for future research. * The authors conduct extensive experiments to validate the effectiveness of their approach, including many ablation studies to assess the impact of different design choices, as well as exploration into the framework's capability boundaries. * The paper is well-organized and easy to understand. * The inclusion of diverse visualizations enhances qualitative comprehension of the method's performance.

Weaknesses

* It appears that the shapes and categories of the manipulated objects are not very diverse, potentially constraining the method's ability to generalize to novel objects. I recommend that the authors summarize the counts of shapes and categories used in the experiments. Additionally, if the shapes of the manipulated objects are limited and the grasp actions are abstracted in the simulation, the authors are encouraged to compare CooHOI with some heuristic methods. An intuitive heuristic baseline could involve having two agents grasp each end of a long object, and then move at the same speed to the target position. * While the authors provide detailed explanations in the appendix, it would enhance comprehension if they provide simple explaination when introducing a new concept in the main paper. Here are some examples: * In Section 3.1, the authors introduce the task-specific goal feature $g_t$ without explanation. * While it is understandable that the proposed style reward can help the method produce behaviors similar to those in the dataset, it would be better to provide a simple explanation of how to evaluate such similarity. * It would be beneficial to provide a brief introduction to the baseline InterPhys in one or a few sentences.

Questions

* While acknowledging the idea of utilizing the object’s dynamics as implicit communication between agents, such communication is not synchronous. Specifically, agents adjust their actions only after one agent changes the object dynamics, resulting in a delay between the change in object dynamics and the response actions taken. I wonder if this delay could lead to jitter of the manipulated object. If not, could the reviewers discuss why such jitter would not occur? * The held point is described as the geometric center of each end of the object. However, in Figure 1, when cooperatively carrying a large box, the characters grasp the edges of the box instead of the center of its face. I am unsure if I misunderstood the definition of the held point, or if the current definition is inadequate, leading agents to choose other positions they find more suitable for completing the task. * In the second row of Figure 4, I am curious about why the four agents fail to put down the box. The authors attribute this failure to the absence of dexterous hands and limited friction, however, similar limitations exist in two-agent manipulation tasks. Besides, the multi-agent put-down operation closely resembles the multi-agent pick-up operation (but in reverse), while the agents can successfully pick up the box, they struggle to put it down. Could the authors provide more discussion or visualizations for better understanding?

Rating

7

Confidence

3

Soundness

4

Presentation

3

Contribution

3

Limitations

The authors have provided a detailed discussion of limitation and failure cases.

Summary

This paper proposes a framework for multi-agent cooperative manipulation in the context of humanoids carrying and transporting large furniture. This task is decomposed into several steps. First, they train a single humanoid to learn how to hold and carry relatively small objects. The agent is trained with ground-truth object information and AMP for natural humanoid behavior. Next, they train a multi-agent policy initialized from the single-agent policy for transporting larger objects. The authors present several ablations and report high performance compared to baseline methods in simulation.

Strengths

This paper studies a practical task and solves it with a well-designed engineering system. It is also well-written, making it easy to read. The paper presents comprehensive ablation experiments on both the single-agent and multi-agent policy parts, revealing important information for the framework. The framework can learn natural humanoid behavior.

Weaknesses

I’m curious about the robustness of the proposed system. If I understand correctly, the bounding box information is given as ground-truth parameters. However, in reality, these quantities are far from perfect. The task is restricted to moving and transporting large furniture-type objects.

Questions

In Table 1, what does the “w/o CooHOI” mean? Does that mean training the entire policy from scratch? Is that policy trained with AMP and the same rewards? And what is the definition of CooHOI? What is included in CooHOI? Regarding the robustness of the system: How would the policy perform in noisy situations? For example, when the object bounding boxes are not accurate (in real scenarios, object perception will have errors). In Figure 6, can you elaborate more on why policies, except for CooHOI, fail to carry objects even when they can still hold objects? Specifically, what are the failure cases? Do these policies fail to maintain a stable holding motion, or do they fail to accurately move the objects to the target location? In addition, r_carry is only defined in the supplementary material. I suggest including it in Section 3.2.2. It would be better to plot the standard deviations in Figure 6, given that you evaluate four seeds. What is the action space of the policy? Is it the target joint position for each joint?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors provide a comprehensive limitation section of their method.