Summary
Recent advancements in 3D point cloud understanding have explored the use of Transformers, resulting in notable progress. However, the computational demands of applying global self-attention to large point cloud datasets, which contain over 0.1 million points, present a significant challenge. To mitigate this issue, researchers have proposed using Transformers within local regions, such as spherical or cubic windows. Nevertheless, these approaches still involve a considerable number of Query-Key pairs, leading to high computational costs. Moreover, previous methods often neglect the local 3D geometric structure by employing linear projections to learn the query, key, and value.
In this paper, a new transformer block named ConDaFormer is introduced to address these challenges while also considering the local geometry prior. ConDaFormer decomposes the cubic window into three orthogonal 2D planes, reducing the number of points involved in attention modeling within a similar range. Although this disassembling operation sacrifices some contextual information, a local structure enhancement strategy is implemented using depth-wise convolutions before and after the attention step. This strategy effectively captures local geometric information.
By leveraging these innovative designs, ConDaFormer is capable of capturing both long-range contextual information and local priors. Experimental results on various benchmarks for 3D point cloud understanding demonstrate the effectiveness of ConDaFormer.
Strengths
(1) The authors propose a novel disassembled window attention module for 3D point cloud semantic segmentation by disassembling the 3D window into three orthogonal planes for self-attention. This strategy effectively reduces computational overhead with negligible performance decrease.
(2) To enhance the modeling of local features, the authors introduce depth-wise sparse convolution within the disassembled window attention module. This combination of self-attention and convolution provides a comprehensive solution for capturing both long-range contextual information and local priors in 3D point cloud data.
(3) Experiments show that our method achieves state-of-the-art performance on widely adopted large scale 3D semantic segmentation benchmarks and comparable performance in 3D object detection task. Extensive ablation studies also verify the effectiveness of the proposed components.
Weaknesses
(1) In Table 1, why is there no results on test set for ConDaFormer?
(2) In Table 5, you said "in comparison with FCAF3D, our method achieves comparable performance but performs more steadily.". What do you mean by more steadily? Do you have any experimental results to support that?
(3) In Table 7, why don't try the window size smaller than 0.16m?
Questions
I'm positive about this paper. I really like the idea to disassemble 3D cubic window into three orthogonal planes for self-attention. It can reduce the computational cost. However, I still have some questions about the experimental results. Please see the Weaknesses and respond to those questions. Thank you.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Limitations
This paper still have some limitations when using larger attention window size. If the authors enlarge the window size from 0.32m to 0.48m, the training loss drops from around 0.52 to around 0.47 while the mIoU does not increase on the S3DIS dataset.