Robot Deformable Object Manipulation via NMPC-generated Demonstrations in Deep Reinforcement Learning

In this work, we conducted research on deformable object manipulation by robots based on demonstration-enhanced reinforcement learning (RL). We present FADERL (Fuzzy-Augmented Demonstration-Embedded Reinforcement Learning), a novel framework for robotic manipulation of deformable objects that significantly improves reinforcement learning efficiency through synergistic unification of High-Dimensional Takagi-Sugeno-Kang (HTSK) fuzzy systems, Generative Adversarial Behavior Cloning (GABC), and Conditional Policy Learning (CPL). Compared to the Rainbow-DDPG baseline, FADERL achieves $2.01\times $ higher global average reward and reduces standard deviation to 45% while requiring fewer computational resources. To address the high cost of human demonstration collection, we introduce a Nonlinear Model Predictive Control (NMPC)-based data augmentation method that generates high-quality demonstrations at minimal cost. Simulation results demonstrate that NMPC-generated demonstrations enable FADERL to achieve performance comparable to human demonstrations. Physical experiments on fabric manipulation tasks—diagonal folding, central-axis folding, and flattening—achieve success rates of 83.3%, 80.0%, and 96.7% respectively, validating our approach’s effectiveness in real-world scenarios. Unlike computationally intensive large-model approaches, FADERL provides a lightweight, task-specific solution with efficient adaptability, making it suitable for practical robotic applications in manufacturing, medical surgery, and service robotics. Note to Practitioners—This study addresses the challenges of efficiency and cost in robotic manipulation of deformable objects by proposing a lightweight algorithmic framework (FADERL) and a low-cost demonstration data augmentation approach based on NMPC. Traditional methods often rely on extensive manual demonstration data or high computational resources. Our solution integrates fuzzy system optimization, reinforcement learning enhancements, and automated demonstration generation to improve success rates in tasks like folding and flattening while reducing data acquisition costs. Experimental results demonstrate robust performance in both simulation and real-world scenarios. The methodology is applicable to industrial applications (e.g., flexible cable assembly), medical surgery (e.g., soft tissue suturing), and household service robots (e.g., clothing organization). Compared to current large-scale models, the proposed algorithm requires fewer computational resources and supports task-specific customization.