Mining and Transferring Feature-Geometry Coherence for Unsupervised Point Cloud Registration

Summary

The paper introduces a new unsupervised outdoor point cloud registration method called INTEGER, which dynamically integrates high-level contextual information and low-level geometric information to generate reliable pseudo-labels, addressing the issue of poor performance in complex outdoor environments seen in previous methods reliant on geometric data. By incorporating the Feature-Geometry Coherence Mining module, Anchor-Based Contrastive Learning, and the Mixed-Density Student model, this method not only performs well in scenarios with significant variations in data density but also demonstrates its efficiency and generalizability on standard datasets like KITTI and nuScenes. Overall, the INTEGER method significantly improves in accuracy and generalizability, especially in handling complex and distant outdoor scenes, showing advantages that traditional methods struggle to match.

Strengths

Originality:The INTEGER method demonstrates originality by innovatively combining high-level contextual and low-level geometric information for unsupervised point cloud registration. This method creatively addresses challenges in outdoor environments, where previous methods relying solely on geometric data often fail. Quality:The paper showcases high-quality research through detailed methodological execution and robust evaluations on standard benchmarks like KITTI and nuScenes. The proposed mixed-density student model, which learns density-invariant features, ensures robust performance across diverse scenarios. Clarity:The manuscript is clearly written and well-organized, effectively communicating its core ideas, methodologies, and results. It uses figures and tables effectively to aid understanding, making it accessible to readers familiar with point cloud processing. Significance:The work is highly significant, offering potential transformative impacts in autonomous driving and robotics by enhancing unsupervised point cloud registration in outdoor scenes. Its competitive performance against supervised methods highlights its practical relevance and potential to reduce reliance on labeled data.

Weaknesses

Dependency on Initial Conditions The INTEGER method's performance heavily relies on the quality of the initial teacher model. A poorly initialized teacher model could propagate errors and inefficiencies through the learning process. The paper could explore and discuss alternative strategies for more robust initializations or provide a more detailed analysis of how the initial conditions affect the overall performance. This would give readers a clearer understanding of the method's robustness and potential limitations in real-world scenarios. Comparison with State-of-the-Art Methods While the paper presents comparisons with existing methods, the selection seems limited to those that are closely aligned with the proposed method's framework. Including a broader range of state-of-the-art methods, especially recent unsupervised learning approaches that use different paradigms, could help validate the strengths and uniqueness of INTEGER more convincingly. This comparison could also be extended to include a discussion on how INTEGER performs relative to these methods in terms of computational efficiency and scalability. Discussion on Failure Cases and Limitations The paper could benefit from a more thorough discussion of scenarios where INTEGER might underperform, such as extremely sparse point clouds or highly noisy environments. Understanding the method's limitations and potential failure cases would help in setting realistic expectations and could guide future research to address these specific challenges. Generalizability Across Different Datasets While the paper tests INTEGER on KITTI and nuScenes, additional experiments on datasets from different domains or with different characteristics could help demonstrate the method's adaptability and robustness. Exploring performance on datasets with varying densities and noise levels would provide a more comprehensive evaluation of the method's effectiveness across diverse real-world conditions.

Questions

Robustness of Initial Teacher Model: Could you elaborate on the strategies used for the initialization of the teacher model? Given the dependency of the INTEGER method on the quality of the initial teacher, understanding its robustness in adverse conditions or with suboptimal initialization would be informative. Are there specific conditions under which the initial model tends to fail, and how does this impact the overall registration accuracy? Broader Comparisons: The paper currently compares INTEGER with a select group of methods. Could you include comparisons with additional state-of-the-art unsupervised methods that employ fundamentally different approaches? This could help in understanding the unique advantages or potential trade-offs of INTEGER. Additionally, it would be beneficial if you could provide insights into the computational efficiency and scalability of INTEGER compared to both supervised and unsupervised counterparts.

Rating

5

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

The manuscript does discuss some technical limitations related to the initialization of the teacher model and the conditions under which the method may underperform. However, these discussions could be expanded to include: Scalability: How does INTEGER scale with increasingly large datasets or more complex environments? Including a discussion on computational resources and runtime could provide a clearer scope of applicability. Robustness: More detailed exploration of robustness against different types of noise or outliers in point cloud data would be beneficial. Given the application domains of point cloud registration, such as autonomous driving and robotics, understanding these aspects is crucial.

Summary

This paper introduces an unsupervised framework for point cloud registration that generates reliable pseudo-correspondences using both low-level geometric and high-level contextual information. It employs a widely used teacher-student architecture and proposes Anchor-Based Contrastive Learning to facilitate robust feature space development through contrastive learning with anchors. Additionally, the paper introduces a Mixed-Density Student approach to learn density-invariant features, effectively addressing challenges associated with density variations and low overlap in outdoor scenarios.

Strengths

This paper focuses on registering point clouds without relying on pose prior datasets to supervise model training. The observation that in the feature space, points of latent new inlier correspondences tend to cluster around respective positive anchors summarizing features of existing inliers is particularly interesting. The FCEM adaptation of the teacher model to a data-specific teacher for the current mini-batch is intriguing. However, the procedure described in the manuscript is complex and challenging to follow. It seems that the teacher model is first updated using the current mini-batch dataset, then correspondences are selected to train the student, followed by an update of the teacher weights using Exponential Moving Average (EMA). This sequence raises questions about the efficiency and effectiveness of such a dynamic updating scheme. The authors need to provide a clearer explanation of this process.

Weaknesses

First limitation is that some parts of the framework appears remarkably similar to the established EYOC method, thereby calling into question the novelty of this contribution. Besides, the integration of different levels (low and high) of information for mining pseudo-labels is highlighted as a distinguishing contribution, yet the manuscript lacks a clear explanation of the specific mechanisms. If Spatial Compatibility Filtering is employed, it should be noted that this technique is already a component of EYOC.

Questions

This paper introduces an unsupervised framework for point cloud registration that integrates both low-level geometric and high-level contextual information to generate reliable pseudo-labels. While the concept is promising, the manuscript could benefit from additional detail and clarification in several areas: 1. Similarity to Existing Methods: Some parts of the framework appears remarkably similar to the established EYOC method, thereby calling into question the novelty of this contribution. 2. Using low and high level information for mining pseudo-labels is highlighted, yet the manuscript lacks a clear explanation of the specific mechanisms. If Spatial Compatibility Filtering is employed, it should be noted that this technique is already a component of EYOC. 3. The manuscript mentions pairs with distances [d1, d2], but it does not explain the impact of these distances for registration. A detailed explanation would enhance understanding of how these challenge the registration. 4. Dataset Differences and Overlap Ratios: The experimental section discusses the use of KITTI and nuScenes datasets. It is important to delineate the differences between these datasets, especially concerning their overlap ratios. Providing specific overlap ratios for these and other datasets would facilitate a more nuanced discussion of the framework's applicability and robustness across different scenarios. 5. Performance on Various Datasets: The framework's performance is evaluated on outdoor datasets like KITTI and nuScenes, but there is no discussion on its efficacy in indoor settings, such as with the 3DLoMatch dataset. Insights into performance in diverse environments would be particularly valuable, given the unique challenges posed by indoor scenarios. 6. Training Time: The manuscript omits details regarding the training duration. Including this information is essential for assessing the practicality of deploying this framework in real-world applications. 7. Comparative Analysis with EYOC: If EYOC also utilizes synthetic pretraining, a direct comparison with the proposed method under similar conditions is necessary. Such a comparative analysis would provide a fairer assessment of the proposed method's strengths and help highlight any genuine advancements. 8. Architecture Details for Predator: Since Predator architecture employs cross-attention and a dual-branch structure but lacks clarity on how the network is divided into teacher and student components. This is non-trivial, and thus, more detailed explanations are required to understand its implementation and functionality fully. 9. Generalization to Indoor Environments: There is a need to assess how networks trained on outdoor datasets perform in indoor point cloud registration tasks. Evaluating the generalization capabilities of the framework would give insights into its versatility and effectiveness across different application settings.

Rating

4

Confidence

5

Soundness

2

Presentation

2

Contribution

3

Limitations

The authors adequately addressed the limitations

Summary

This submission proposes an unsupervised framework for point cloud registration. The key contribution is a two-stage training scheme, which first trains a teacher network on synthetic data that extract features in a density-invariant manner, and then trains a student network with pseudo label produced by the teacher net.

Strengths

1. The submission is in general well structured. Similar to this type of works, it contains quite a few notations and architecture design details, which are presented in a relatively clear manner. 2. The key insight, namely, "points of latent new inlier correspondences tend to cluster around respective positive anchors..." seems interesting and fresh to me (though I am not an expert in this area). 3. The experimental performance looks competitive.

Weaknesses

1. The key insight is more presented as an empirical observation, is there any chance that such can be justified in a more concrete way, even on some toy examples? 2. I didn't find report on the training cost and inference efficiency in the submission.

Questions

In KITTI and nuScenes, as the data are scanned with limited degree of viewpoint freedom, can such method be applied to cases with significant rigid transformations? For example, in robotics, one may have to handle objects in varying poses randomly sampled in SE(3). If it is doable, could you please provide some insight/estimation on cost of the pre-training stage. Minor points: . L161, missing space.

Rating

6

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

The limitation discussion looks good to me.

Summary

This paper focuses on unsupervised point cloud registration in 3D computer vision. To tackle the problem, it leverages the observation that in the feature space, points of latent new inlier correspondences tend to cluster around respective positive anchors. Based on that, this paper proposes a novel unsupervised registration method termed INTEGER to incorporate high-level contextual information for reliable pseudo-label mining. Extensive experiments were conducted on outdoor benchmarks including KITTI and nuScenes, which demonstrates the effectiveness of the proposed method.

Strengths

1. This paper is well-written and well-organized; 2. The observation w.r.t. the new inliers and positive anchors seems interesting and correct; 3. The use of teacher-student networks is rational, and the design of Feature-Geometry Coherence Mining is novel. 4. The experimental results are strong on outdoor scenes.

Weaknesses

1. There are some problems in terms of figures. For example, Fig. 1 is not very easy to understand. As a teaser, it does not demonstrate the core idea of this paper very clearly. Fig. 2 and Fig.3 include too much content and make it hard to focus.

Questions

1. In Tab.1, why there are fewer baseline methods on nuScenes compared to those on KITTI? As well as the generalization test from KITTI to nuScenes? 2. Although the title demonstrates that this paper focuses on outdoor scenes, the proposed method could also work on Indoor scenes. Why Indoor scenes are excluded? Could some experiments be added to further demonstrate the superiority of the proposed method?

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Limitations have been dicussed.

Post-rebuttal

I appreciate the authors' clarifications. After reading the rebuttal, I still have the following questions: It seems we may have different understandings of the dual-branch structure. Let me restate the concept as described in the original paper: PREDATOR is a two-stream encoder-decoder network, producing point-level features for both the source and target point clouds. How the final loss is calculated remains unclear to me. Did you calculate the loss from the two stream? Again, I thank the authors for their work and time.