End-to-end Trainable Deep Neural Network for Robotic Grasp Detection and Semantic Segmentation from RGB

In this work, we introduce a novel, end-to-end trainable CNN-based\narchitecture to deliver high quality results for grasp detection suitable for a\nparallel-plate gripper, and semantic segmentation. Utilizing this, we propose a\nnovel refinement module that takes advantage of previously calculated grasp\ndetection and semantic segmentation and further increases grasp detection\naccuracy. Our proposed network delivers state-of-the-art accuracy on two\npopular grasp dataset, namely Cornell and Jacquard. As additional contribution,\nwe provide a novel dataset extension for the OCID dataset, making it possible\nto evaluate grasp detection in highly challenging scenes. Using this dataset,\nwe show that semantic segmentation can additionally be used to assign grasp\ncandidates to object classes, which can be used to pick specific objects in the\nscene.\n