Official Comment by Authors (Part 1)
We extend our heartfelt thanks to the reviewer for the valuable feedback. In addressing the concerns raised, we are pleased to provide the following clarifications and additional information.
**W-1: Analysis of 2D/3D masked autoencoders.**
We are grateful for the insightful suggestion and deeply appreciate the reviewer's accurate understanding of our insight. We agree that, unlike grid-organized 2D image data, point cloud data is irregular and possibly noisy, making direct recovery of point positions less effective in capturing the intrinsic features of 3D shapes. We will clarify our statement more explicitly in our revision.
**W-2: Surface normal is noisy in real world scenes.**
We thank the reviewer for this valuable feedback. During our experiments, we also observed that surface normals in scene data are noisy and often lack consistent orientation due to incomplete point clouds. However, we found that surface variation remains robust in these conditions. Therefore, as mentioned in Section 4.4, we took the rgb color signal and surface variation as the target features, rather than the surface normal.
**W-3-1: Naive extension of Point-MAE**.
We appreciate the reviewer for raising this important concern. We would like to argue that our method is not merely a simple extension of Point-MAE.
Firstly, as elaborated in Section 1 and illustrated in Figure 1, in addition to the insight mentioned by the reviewer in W-1 (which we believe is also an important contribution), we introduce a general pre-training framework with a novel attention-based decoder, thereby enhancing the pre-training model. Our model is not restricted to any specific encoder design and is easily adaptable to various task settings, including those in challenging indoor environments. Also, by disregarding point positions and focusing on intrinsic 3D shape features, our model learns robust features suitable for diverse downstream tasks.
Secondly, as mentioned in Table 4 of the main paper (referenced below), a naive extension of Point-MAE with additional features does not yield the same level of significant improvement as our method. This finding further proves the validity of our insights and the effectiveness of our decoder design.
----
| Method | Target Feature | | ScanObjectNN PB-T50-RS |
| ---------- | ------------------------------ | ---- | ---------------------- |
| PointMAE | position only | | 85.2 |
| | position + normal∗ | | 85.7 |
| | position + surface variation∗ | | 85.9 |
| | position + normal + variation∗ | | 86.0 |
| MaskFeat3D | normal | | 86.5 |
| | surface variation | | 87.0 |
| | normal + surface variation | | 87.7 |
**Table 4: Ablation study on different features.** ∗ uses position-index matching Zhang et al.[1] for feature loss computation.
----
**W-3-2: More analysis.**
In the supplementary material, we provide further experimental analysis. In Section 2.3, we visualized the point features of our model and other MAE-based approaches. Our approach shows more discriminative features than other methods. This observation proves that learning to reconstruct high-order geometric features robustly enables the encoder to extract more distinctive and representative features.
In Section 2.4 of the supplementary material, we investigated how the learned features of our model assist in zero-shot correspondence learning, showing that points nearest in the feature space exhibit similar semantic information, even across different objects within the same class. We hope these analyses further highlight the robustness and effectiveness of our pre-training strategy.
[1] Zhang, Yabin, et al. "Masked surfel prediction for self-supervised point cloud learning." *arXiv preprint arXiv:2207.03111* (2022).