RLDG: Robotic Generalist Policy Distillation via Reinforcement Learning

Recent advances in robotic foundation models have enabled the development of\ngeneralist policies that can adapt to diverse tasks. While these models show\nimpressive flexibility, their performance heavily depends on the quality of\ntheir training data. In this work, we propose Reinforcement Learning Distilled\nGeneralists (RLDG), a method that leverages reinforcement learning to generate\nhigh-quality training data for finetuning generalist policies. Through\nextensive real-world experiments on precise manipulation tasks like connector\ninsertion and assembly, we demonstrate that generalist policies trained with\nRL-generated data consistently outperform those trained with human\ndemonstrations, achieving up to 40% higher success rates while generalizing\nbetter to new tasks. We also provide a detailed analysis that reveals this\nperformance gain stems from both optimized action distributions and improved\nstate coverage. Our results suggest that combining task-specific RL with\ngeneralist policy distillation offers a promising approach for developing more\ncapable and efficient robotic manipulation systems that maintain the\nflexibility of foundation models while achieving the performance of specialized\ncontrollers. Videos and code can be found on our project website\nhttps://generalist-distillation.github.io\n