Kinematically-Informed Interactive Perception: Robot-Generated 3D Models for Classification

To be useful in everyday environments, robots must be able to observe and\nlearn about objects. Recent datasets enable progress for classifying data into\nknown object categories; however, it is unclear how to collect reliable object\ndata when operating in cluttered, partially-observable environments. In this\npaper, we address the problem of building complete 3D models for real-world\nobjects using a robot platform, which can remove objects from clutter for\nbetter classification. Furthermore, we are able to learn entirely new object\ncategories as they are encountered, enabling the robot to classify previously\nunidentifiable objects during future interactions. We build models of grasped\nobjects using simultaneous manipulation and observation, and we guide the\nprocessing of visual data using a kinematic description of the robot to combine\nobservations from different view-points and remove background noise. To test\nour framework, we use a mobile manipulation robot equipped with an RGBD camera\nto build voxelized representations of unknown objects and then classify them\ninto new categories. We then have the robot remove objects from clutter to\nmanipulate, observe, and classify them in real-time.\n

Paper

Similar papers

© 2026 NYSGPT2525 LLC