DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics

Summary

The paper works on trajectory generation for soft body manipulation with differentiable simulation. To address the key challenge of representing task goals for optimization, the authors propose to use natural language descriptions to lower the barrier for annotation, where a framework that utilizes LLM for translating natural language into optimization programs is developed to facilitate data collection.

Strengths

The idea of task specification with natural language is an interesting idea; The overall framework seems to be reasonable and technical sound, it is smart to leverage LLM for converting natural language descriptions into optimization programs; The paper is well-presenting and easy to read.

Weaknesses

More evaluations can be performed to validate the robustness of the trajectory generation; There is no discussion of the failure cases; Some missing citations in line 21; Typo in figure 4: smapling -> sampling; Some works that leverage natural language for task goal specifications, which may need to be discussed in the related work: [1] StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objects, ICRA 2022; [2] Differentiable Parsing and Visual Grounding of Natural Language Instructions for Object Placement, ICRA 2023

Questions

In section 4.3, the authors say that a sampling-based RRT planner is used to determine the path for the actuator, and then the trajectory is refined with a gradient-based optimizer. Could the authors provide more details on how the trajectory is initially calculated and how it is refined for execution? Besides, it would be better to provide more snapshots for the rollouts to show the difference between the initial trajectory and the final ones; Would the trajectory optimization be sensitive to the initialization? It would be interesting to conduct some experiments for demonstrating the robustness of the initialization strategy; Are there any failure cases that are caused by the LLM translation? It would be good to understand the failure mode in which situations the LLM failed to generate appropriate optimization programs.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Limitations are discussed in the paper, but no failure cases are presented and analyzed.

Summary

This study focuses on soft-body manipulation problems such as flattening dough using a rolling pin, cutting deformable objects, and more. It introduces a method that engages non-expert users to provide detailed annotations and identify sub-goal states within key frames of the task video. Despite requiring additional human intervention, this approach generates a rich learning signal for manipulation tasks involving deformable objects. This significantly eases the learning difficulty associated with such tasks. The proposed method has been evaluated based on six fundamental deformable object manipulation skills, demonstrating its effectiveness.

Strengths

This work proposes a unique representation of task specifications. It incorporates human-labeled annotations for deformable object tasks to shape an objective/reward function. Such a reward function provides far richer reward signals than just a goal state, potentially greatly reducing policy learning difficulties. Furthermore, this study offers a new dataset that includes 100 deformable object manipulation tasks, complemented by human annotations on key frames. This contribution could be beneficial to the broader community and greatly aid studies related to deformable object manipulation. The presentation of this research is both clear and organized.

Weaknesses

Human annotation offers a valuable reward signal, which could potentially simplify the learning process. However, it also imposes certain constraints on the policy strategy. When states deviate from the demonstrated trajectory, adhering strictly to specific action commands could exacerbate the situation. Hence, at its current stage, it lacks the flexibility needed to handle unseen states not accounted for in the instructions/annotations. Moreover, each task necessitates considerable human involvement in creating subgoal states and corresponding annotations. Although software solutions have been developed to assist in this process, it still represents a significant involvement of human resources and incurs considerable costs.

Questions

Q1: How is a key frame chosen? Also, can you explain how the decomposition of long-horizon tasks occurs?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

See weaknesses and questions.

Summary

This paper presents a novel approach to soft body manipulation employing the strengths of the Large Language Model (LLM). The key innovation is viewing tasks as data, with each data point consisting of an initial scene and an optimization objective. To tackle the challenge of task representation, the authors introduce DiffVL, a framework enabling non-expert users to define soft-body manipulation tasks to a differentiable solver using a combination of vision and language. The users specify tasks via an interactive simulator with a sequence of 3D scenes (keyframes) connected by natural language instructions. The authors also developed a corresponding GUI and curated SoftVL100, a vision-language dataset with 100 diverse tasks. They further developed a method that combines the power of a large-language model and differentiable physics to solve a wide variety of challenging long-horizon tasks in SoftVL100. This study's contributions lie in its new task representation, the developed GUI, the curated dataset, and the DiffVL method.

Strengths

The paper under discussion is a robust, well-rounded work, bringing to the table significant contributions across multiple facets. It excels in terms of novel methodology, tool development and proposes a new dataset: • The authors introduce an innovative multi-stage vision-language representation. This new approach simplifies the definition of soft-body manipulation tasks, making it accessible for non-expert user annotations - a noteworthy contribution to research in this area. • In terms of tool development, the authors have crafted a corresponding Graphical User Interface (GUI), enhancing the user experience and overall accessibility of their proposed approach. They have also curated SoftVL100, a compilation of 100 realistic soft-body manipulation tasks derived from online videos. This dataset stands as a valuable resource for further research and application in this domain. • Moreover, the authors have devised DiffVL, a method that bridges the gap between a large-language model and differentiable physics. This combination of strengths is adeptly applied to tackle a wide variety of challenging long-horizon tasks presented in SoftVL100, which marks a significant advancement in the field. The experimental section of the paper is another key strength, featuring a thorough ablation study.

Weaknesses

The relative weakness of the approach can be considered the amount of human labor required for dataset collection. Also, DSL could sometimes be too constrained to define arbitrary new tasks. Despite these points, they in no way diminish the overall quality of the work presented in the paper. The minor limitations identified simply highlight areas for potential future refinement. The project stands as an excellent example of research and tool development, and I am eager to see how it evolves.

Questions

• It would be good to have more info about time constraints - simulation and training time for different tasks, not only training progress per epoch. • IWere there any simulation failure cases? How challenging was simulator tuning for different tasks?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

Limitations were addressed fairly well.

Summary

This paper demonstrates curating a set of 100 soft-body manipulation tasks and provides expert policies for them by using a mix of: annotators that provide supervision in the form of keyframes and/or natural langauge annotation, translating the annotations into programs via an LLM, and solving an optimization.

Strengths

One strength of this paper is that obtaining manipulation policies for soft bodies is undoubtedly hard, and so anything that can do is has some potential value to the community. Additionally, the dataset if released by the authors could be valuable. The tasks and expert policies could be used as data to study methods that require some amount of expert data (imitation / offline RL / etc), and it's hard to both get these types of policies, as well as build simulation environments, in order to study these well. Good datatsets and tasks can be considered a bottleneck of manipulation research, especially with soft body manipulation, and the dataset could be valuable here.

Weaknesses

The first sentence of this paper has unfinished citations. It reads, verbatim: "This paper focuses on soft body manipulation, a research topic with a wide set of applications such as folding cloth[cite], untangling cables [cite], and cooking foods[cite]." This is a pity because it makes it hard to take the work too seriously given an incomplete first sentence. Meanwhile, the rest of the paper seems relatively polished, so I'm doing my best to evaluate it seriously. One limitation that should be kept in mind is that in multiple ways I only see the methods of this paper as being relevant to simulation, and not directly to the real world. This applies both to the 3D GUI annotation process, as well as the process for converting into something that the differentiable physics solver can solve. The work could help enable further studies in simulation that could eventually impact the real-world, indirectly either through helping provide evaluation tools or perhaps eventually sim-to-real, but not directly. Overall, I think the main potential interesting thing is that the authors have actually provided expert policies for 100 or so soft-body manipulation tasks. This is hard to achieve. I personally find that the existence of the policies and tasks is the main valuable and interesting thing, and less so the methods used to obtain them. Additionally, 100 tasks is not that large for a crowdsourced effort. This is an important point in my opinion because it's likely the process might have just gone faster if the first author designed all the tasks themselves rather than teaching others to do so. One might need to get to 1,000 or 10,000 tasks in order to really start to see the benefits of a crowdsourced effort. Accordingly this in my opinion undermines the care taken in the paper to make the annotation process usable by non-experts. Another lens to look at this paper is: what actual results are shown? There are two results tables/figures: Table 1, and Figure 7. Table 1 shows that it is hard for SAC or PPO to solve the tasks, demonstrating that they are indeed not easy. It also shows that their method, which has access to considerably more info than SAC/PPO due to its use of the annotations, can solve the tasks. This is not a fully fair comparison, and that's fine. The second results table shows that the multi-stage annotations and optimizations are useful. This also makes sense. It is hard to pinpoint what specific contribution claim this results figure supports though, since I don't think there is clear novelty in the multi-step framework presented by the authors.

Questions

Do each of the points discussed in the weaknesses make sense?

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors do discuss limitations in the conclusion. They should add that the methods are limited to simulation and impact on the real-world is left as a future question to address. They don't discuss societal impacts but I think that's okay here.

Summary

This paper proposes DiffVL, a novel framework that tackles soft-body manipulation, which consists of a GUI for users to specify tasks easily and a large language model (LLM) for translating text instructions to programs for policy learning and execution.

Strengths

- An intuitive user interface for task specification - A novel combination of LLM and a differentiable simulator for program generation and optimization - Good empirical success rate - Clear writing

Weaknesses

- Heavy user input needed on the magnitude of hours to specify task - Comparison in Table 1 is a bit unfair since other pure RL baselines do not leverage language guidance - Writing is not polished (e.g., missing citations in line 21)

Questions

How does this approach differ from other recent robot learning papers utilizing LLM for instruction generation/high-level planning? The novelty in this aspect is obscure to me.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

The authors discussed limitations adequately in the paper.

Thank you and we are looking forward to your post-rebuttal feedback!

Dear AC and all reviewers: Thanks again for all the insightful comments and advice, which helped us improve the paper's quality and clarity. The discussion phase has been on for several days and we have not heard any post-rebuttal responses yet. We would love to convince you of the merits of the paper. Please do not hesitate to let us know if there are any additional experiments or clarification that we can offer to make the paper better. We appreciate your comments and advice. Best, Authors

Thank you for your response

I appreciate the detailed response. My concerns are addressed well.

Reply to Authors

Thanks for the rebuttal. My concerns have been addressed.

POST-REBUTTAL

Thanks to the authors for their updates. The supplementary technical details effectively address the inquiries I had, leading me to revise my rating to a "weak accept."