From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network

3D object detection from LiDAR point cloud is a challenging problem in 3D scene understanding and has many practical applications. In this paper, we extend our preliminary work PointRCNN to a novel and strong point-cloud-based 3D object detection framework, the part-aware and aggregation neural network (Part-<inline-formula><tex-math notation="LaTeX">$A^2$</tex-math><alternatives><mml:math><mml:msup><mml:mi>A</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:math><inline-graphic xlink:href="shi-ieq1-2977026.gif"/></alternatives></inline-formula> net). The whole framework consists of the part-aware stage and the part-aggregation stage. First, the part-aware stage for the first time fully utilizes free-of-charge part supervisions derived from 3D ground-truth boxes to simultaneously predict high quality 3D proposals and accurate intra-object part locations. The predicted intra-object part locations within the same proposal are grouped by our new-designed RoI-aware point cloud pooling module, which results in an effective representation to encode the geometry-specific features of each 3D proposal. Then the part-aggregation stage learns to re-score the box and refine the box location by exploring the spatial relationship of the pooled intra-object part locations. Extensive experiments are conducted to demonstrate the performance improvements from each component of our proposed framework. Our Part-<inline-formula><tex-math notation="LaTeX">$A^2$</tex-math><alternatives><mml:math><mml:msup><mml:mi>A</mml:mi><mml:mn>2</mml:mn></mml:msup></mml:math><inline-graphic xlink:href="shi-ieq2-2977026.gif"/></alternatives></inline-formula> net outperforms all existing 3D detection methods and achieves new state-of-the-art on KITTI 3D object detection dataset by utilizing only the LiDAR point cloud data.