Closed-Loop Visuomotor Control with Generative Expectation for Robotic Manipulation

Despite significant progress in robotics and embodied AI in recent years, deploying robots for long-horizon tasks remains a great challenge. Majority of prior arts adhere to an open-loop philosophy and lack real-time feedback, leading to error accumulation and undesirable robustness. A handful of approaches have endeavored to establish feedback mechanisms leveraging pixel-level differences or pre-trained visual representations, yet their efficacy and adaptability have been found to be constrained. Inspired by classic closed-loop control systems, we propose CLOVER, a closed-loop visuomotor control framework that incorporates feedback mechanisms to improve adaptive robotic control. CLOVER consists of a text-conditioned video diffusion model for generating visual plans as reference inputs, a measurable embedding space for accurate error quantification, and a feedback-driven controller that refines actions from feedback and initiates replans as needed. Our framework exhibits notable advancement in real-world robotic tasks and achieves state-of-the-art on CALVIN benchmark, improving by 8% over previous open-loop counterparts. Code and checkpoints are maintained at https://github.com/OpenDriveLab/CLOVER.

Paper

Similar papers

Peer review

Reviewer eUsr7/10 · confidence 4/52024-07-08

Summary

This paper introduces a novel framework that incorporates closed-loop feedback into vision-based control systems for robot manipulation tasks. The author proposes a text-conditioned video diffusion model for high-level reference path planning. The error measurement and feedback policy are obtained through an encoder-decoder architecture. This framework improves task performance over open-loop policies with a higher success rate and good instruction-following without the need for giant LLM models. The method is tested on proper benchmarks for language-based manipulation and has a thorough discussion of the method and results.

Strengths

1. Novelty: This paper presents a novel feedback policy for visual input. The proposed measurable embedded space can help identify the error between the reference plan and the executed plan, and the encoder-decoder bridges the input-feedback-output modules. This design helps in long-horizon planning where open-loop control easily fails. 2. Presentation: This paper is well-written and provides a thorough discussion of the methods. 3. Potential impact: A feedback system for visual-based control has a potential impact in other robotics systems other than manipulators.

Weaknesses

1. Generalization: how this framework can be generalized across different task/robot platforms/datasets? Especially for the sub-goal replan/transition, which seems to be very specific for each task. The replanning threshold is also designed by hand. Can it be determined during training? 2. The paper seems not to report the computational time for the model during testing, which is crucial for a close-loop policy in real-time. Especially with a diffusion-based policy, the sampling time could be long. Could provide more discussion on addressing this issue. 3. Do you expect your proposed architecture to scale well when pre-trained on a large dataset; what additional design might be needed?

Questions

1. How the dataset in CLOVER is obtained? What data helps train the CLOVER? More details will be welcomed. 2. How does CLOVER perform in environments with significant variations from the training scenarios? Have you tested the framework on completely new tasks or objects not seen during training?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The author discussed the potential societal impact in the Appendix.

Reviewer hKyy6/10 · confidence 4/52024-07-10

Summary

This paper proposes CLOVER, a closed loop visuomotor control framework that incorporates feedback mechanisms to improve adaptive robotic control. The framework consists of a text-conditioned video diffusion model for generating visual plans as reference inputs, an error measurement module to model the discrepancy between the current and goal states in the embedding space, and a feedback-driven controller that refines actions from feedback and initiates replans as needed.

Strengths

1. The paper is well written and organized, clear to read. 2. Integrating closed loop feedback into robotic visuomotor control is interesting and useful, which can help deal with deviation and improve control accuracy. 3. Combines visual planning, error measurement, and feedback-driven control in a cohesive system.

Weaknesses

1. Limited evaluation to real robot environment. The authors only show one experiment comprising three consecutive sub-tasks, which is not enough to test the generalizability and robustness of the approach across different environments and tasks. 2. The measurable embedding space for error quantification seems effective not be sufficiently validated. This component’s effectiveness could vary significantly in different scenarios, and the paper does not provide extensive evidence of its robustness. 3. The framework seems need a lot of computation, each step in the loop the diffusion model needs to generate a video, then get embeddings through two ViT-based encoders for RGB and depth respectively. I guess the high computation required could limit its real-time execution and practical applicability. 4. Further validation on a wider range of benchmarks and task environments is necessary to confirm the framework's generalizability. 5. Minor, in Figure 5, the authors show generated videos of four tasks conditioned on the same initial frame, but the first figure in each task looks different, better to make them the same.

Questions

1. In Figure 3, the authors mention is obtained during a roll-out, what task and what environment are you using? How many tasks and environments did you test? 2. How long does it take to execute one task in simulation and real robot? 3. Can you show video demos of your experiments not just screenshots in the paper?

Rating

6

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

1. High computation required for both training and real-time execution could limit practical applicability. The proposed framework's complexity might pose challenges in implementation. 2. The scalability of the framework to diverse and complex tasks remains uncertain. 3. The real-time execution of the feedback-driven controller might face latency issues, especially in dynamic and unpredictable environments.

Reviewer DjFL5/10 · confidence 4/52024-07-11

Summary

The paper presents a novel framework named CLOVER. The proposed system aims to enhance the adaptability and robustness of robotic manipulation in long-horizon tasks by incorporating closed-loop control principles. CLOVER consists of three main components: a text-conditioned video diffusion model for generating visual plans, a measurable embedding space for accurate error quantification, and a feedback-driven controller that refines actions based on real-time feedback. The framework shows significant improvement in real-world robotic tasks and sets a new state-of-the-art performance on the CALVIN benchmark.

Strengths

1. Originality: CLOVER introduces a unique combination of closed-loop control principles with generative models for robotic manipulation. The use of a text conditioned video diffusion model to generate visual plans is a creative approach that leverages recent advances in generative AI. 2. Quality: The methodology is well-articulated, detailing the design of each component in the framework. The inclusion of depth map generation and optical flow regularization to enhance the reliability of visual plans demonstrates the challenges in robotic manipulation. 3. Clarity: The paper is clearly written, with a logical structure that guides the reader through the motivation, methodology, and experimental validation. Figures and diagrams are effectively used to illustrate complex concepts and the overall system architecture. 4. Significance: The proposed framework addresses a critical challenge in robotics—improving the robustness and adaptability of robotic systems for longhorizon tasks. The notable performance improvements on the CALVIN benchmark and real-world tasks underscore the potential impact of CLOVER on the field of robotic manipulation.

Weaknesses

1. Novelty of the error measurement approach: While the paper introduces a measurable embedding space for error quantification, it does not provide a detailed comparison with existing methods in terms of novelty and performance. Additional analysis or experiments comparing this approach with other state-of-the-art error measurement techniques would strengthen the contribution. 2. Clarification of the effect of closed-loop system: The article seems to tackle the long horizon manipulation tasks, but in the article there are no obvious evidences to prove it. 3. Scalability of the feedback-driven controller: The paper presents the feedback-driven controller as an effective solution for adaptive control. However, it lacks a discussion on the scalability of this approach for more complex and diverse task environments. Evaluating the controller's performance across a wider range of scenarios, particularly dynamic scenarios would provide a better understanding of its generalizability. 4. Lacked methodology clarification: Some technical aspects, particularly the algorithms and specific implementation details, are not as thoroughly explained as they could be. This makes it harder to fully grasp the intricacies of the proposed system. 5. Relatively limited experiments: The experiments conducted on the CALVIN dataset and real-world scenarios may not be sufficient to demonstrate the effectiveness of the proposed method. It is recommended to perform additional experiments in more long-horizon simulation environments such as RLBench and robosuite. Additionally, the real-world experiments are relatively limited, so further experiments should be conducted to provide a more comprehensive evaluation. 6. Computational complexity: The incorporation of a video diffusion model and feedback mechanisms might introduce significant computational overhead. The paper does not provide an in-depth analysis of the computational requirements and how they impact real-time performance, which is crucial for practical deployment.

Questions

1. Can the authors provide more detailed training process used for the text-conditioned video diffusion model and the error measurement strategy, including training datas, network architecture? 2. How does CLOVER handle highly dynamic environments where changes occur rapidly and unpredictably? 3. How is the training data organized during the diffusion model generation phase of the visual plans? How many intermediate points are selected for each trajectory? I believe this needs to be explained, as it affects the performance of the final trajectory. 4. What are the computational requirements for running CLOVER in real-time, and how does it scale with more complex tasks?

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

The authors have addressed some limitations, such as they validate CLOVER for simulation and real-world experiments by training the models heavily on the corresponding data. However, further discussion on the scalability of the framework and its adaptability to a wider range of robotic morphologies and environments would be beneficial.

Reviewer DjFL2024-08-13

Thank you for your response. I've read the other reviews and the rebuttal. I’m keeping my initial score.

Authorsrebuttal2024-08-14

Official Comment by Authors

Thank you for the kind feedback. We appreciate the time and effort you have put into reviewing our work! We will update the manuscript based on your helpful review.

Reviewer hfKW7/10 · confidence 3/52024-07-19

Summary

The authorsThe authors introduce CLOVER, a generalizable closed-loop visuomotor control framework that incorporates a feedback mechanism to improve adaptive robotic control. The method uses a text-conditioned video diffusion model to generate reference visual subgoals, an error measurement to quantify the difference between the current state and the planned sub-goal, and a feedback-driven controller via inverse dynamics model to ourput actions. Experiments on CAlVIN benchmark and ALOHA real robot shows the method outperforms all previous methods by a notable margin.

Strengths

1. The problem of using closed-loop visuomotor control to solve long-horizon tasks is very meaningful. 2. The motivation and storyline is stated clearly. The paper is well-written and easy to follow. 3. The experiments on both the simulation and the real robot are convincing. 4. The ablation study of visual embedding, optical flow regularization, error measuring, multi modal fusion and sampling rate are very sufficient.

Weaknesses

See questions

Questions

1. How is your work different from AVDC? 2. Could the authors elaborate more on the superiority of their design which use diffusion model generate reference frames and then use inverse dynamics model to get the actions, over the design like diffusion policy[60] which directly generate actions from diffusion model with a closed-loop style? 3. What is the orange dash line meaning in Figure 6(a)? 4. It seems author defines sub-goals as unreachable when the cosine distance between the state embeddings of consecutive are too far. 5. Would there be circumstance when the embedding is close but the sub-goal is actually not reachable due to singularity or other robot limit? Especially when testing the framework on real robot?

Rating

7

Confidence

3

Soundness

3

Presentation

4

Contribution

3

Limitations

Did not test on generalization ability.

Reviewer eUsr2024-08-11

Thank you for the rebuttal and the additional experiment results presented. My concern is addressed and I will raise my score to 7.

Reviewer hKyy2024-08-12

Comment

Thanks authors for the response to address the concerns, I increased my score to 6.

Authorsrebuttal2024-08-12

Thank you for reviewing our responses and raising the score. We will update our paper based on your insightful comments.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC