A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objects

We present a framework for solving long-horizon planning problems involving\nmanipulation of rigid objects that operates directly from a point-cloud\nobservation, i.e. without prior object models. Our method plans in the space of\nobject subgoals and frees the planner from reasoning about robot-object\ninteraction dynamics by relying on a set of generalizable manipulation\nprimitives. We show that for rigid bodies, this abstraction can be realized\nusing low-level manipulation skills that maintain sticking contact with the\nobject and represent subgoals as 3D transformations. To enable generalization\nto unseen objects and improve planning performance, we propose a novel way of\nrepresenting subgoals for rigid-body manipulation and a graph-attention based\nneural network architecture for processing point-cloud inputs. We\nexperimentally validate these choices using simulated and real-world\nexperiments on the YuMi robot. Results demonstrate that our method can\nsuccessfully manipulate new objects into target configurations requiring\nlong-term planning. Overall, our framework realizes the best of the worlds of\ntask-and-motion planning (TAMP) and learning-based approaches. Project website:\nhttps://anthonysimeonov.github.io/rpo-planning-framework/.\n