3D Focusing-and-Matching Network for Multi-Instance Point Cloud Registration

Summary

This paper proposes FMNet, an end-to-end deep learning approach for multi-instance point cloud registration. The key novelty is an attention-weighted feature matching module that can adaptively focus on reliable point correspondences during matching. Unlike traditional two-step methods, FMNet jointly learns feature extraction and matching within a unified network using an attention mechanism. The attention module computes matching probabilities between point features through scaled dot-product, allowing the network to dynamically highlight meaningful matches. Integrated with geometric and cycle consistency losses, FMNet achieves state-of-the-art registration performance on multiple datasets, demonstrating robust handling of noise, occlusions and point density variations. This work pioneers applying attention for solving the point cloud registration problem in a unified deep learning framework.

Strengths

1. The proposed an attention-weighted feature matching module that can adaptively focus on important point pairs, enhancing robust matching. This innovative design surpasses many previous hand-crafted or geometry-constrained feature matching strategies. 2. The proposed FMNet end-to-end framework is well-designed, with clear mathematical explanations for the attention module. Extensive evaluations on public datasets and robustness analysis against noise/missing data provide strong empirical evidence.

Weaknesses

1. The author's contribution is not summarized in points, which seems rather unclear. 2.Even though your network is clearly expressed in the article, I think it is overly structured, which is why I find it less than innovative. 3. In contrast to previous work PointCLM utilized density information for clustering relationships, it seems to me that this article does not effectively extract and utilize density information or other bases used for clustering. I think this information is more important for multi-instance alignment. 4. Compared to previous work PointCLM and ECC, the article seems to be less theoretical, with a large number of formulas used to describe the structure of the network. 5. I think this method of finding the center point first and then doing multi-instance point cloud registration is time consuming.

Questions

What is the approximate percentage of time spent finding the center point and the registration, respectively?

Rating

5

Confidence

5

Soundness

3

Presentation

3

Contribution

2

Limitations

Yes, the author fully addresses the limitations of his proposed methodology.

Summary

This paper introduces a novel focusing-and-matching technique for addressing the multi-instance point cloud registration challenge. Instead of fitting multiple models from a set of incorrect correspondences, this method initially detects potential instance regions and subsequently performs standard pairwise point cloud registration. The approach was tested on two benchmarks, where it outperformed existing state-of-the-art methods in terms of recall and precision metrics.

Strengths

1. The primary contribution of this paper is the establishment of a new approach to solving the multi-instance registration task. Unlike previous methods that rely on multi-model fitting (using RANSAC-like methods) from a set of spurious correspondences, which are computationally expensive and unreliable due to the large matching space, this method uses learned scene priors to narrow down the matching space within individual regions of interest before performing standard pairwise point cloud registration. 2. The writing is clear and easy to follow. In particular, the methodology section is well-structured. The concept of each variable is connected and explained fluently.

Weaknesses

1. Certain details in Section 3.2 need clarification. Specifically, when the first module predicts K object centers but there are only (K-2) ground truth instances, how does the pair-wise registration model manage these two falsely detected objects? From my understanding, it appears that the model only predicts the transformation parameters without providing additional confidence scores. 2. The metrics require explanation or references. For MR, MP, and MF, how is a “registered instance” defined? What criteria are used to consider a predicted pose successful, such as RMSE, chamfer distance, relative translation error, or relative rotation error? Additionally, what are the thresholds? 3. More experimental results related to the 3D focusing module are needed, as this module determines the number of pairwise registrations performed, which sets the upper bound for the number of “successfully registered instances.” Results such as the number of detected objects, correctly detected objects, and wrongly detected objects are necessary, as these significantly impact the metrics, including MR, MP, and MF.

Questions

I don't have any questions.

Rating

6

Confidence

4

Soundness

3

Presentation

4

Contribution

3

Limitations

I can see the limitation is the performance of the 3D focusing module, espeecially its generalizability to unseen scene. Once it wrongly detects object proposals, the following pairwise registration module will fail to register the instance as well.

Summary

For multi-instance point cloud registration, the authors proposed a 3D focusing-and-matching network by learning multiple pair-wise point cloud registration. Specifically, a 3D multi-object focusing module is proposed to locate the center of each object and generate object proposals. In addition, a 3D dual-masking instance matching is introduce to estimate the pose between the model point cloud and each object proposal. Extensive experiments on two popular datasets, Scan2CAD and ROBI, show that the proposed method achieves new state-of-the-art performance on the multi-instance point cloud registration task.

Strengths

- From the experimental results, it can be seen that decomposing multi-instance registration into multiple pair-wise registrations is very simple and effective. - The method achieves new state-of-the-art on Scan2CAD and ROBI. Especially in the challenging ROBI dataset, the proposed method is significantly better than the previous SOTA (+9%). - It is very convincing that the authors analyzed the upper bound of the method.

Weaknesses

- Since the overall architecture is a two-stage structure for multi-instance point cloud registration, its inference time is lower than one-stage MIRETR (0.54s vs. 0.40s per scene in Table 2). Why the inference time is slightly lower than MIRETR? Considering the two-stage strategy, the author should conduct a detailed analysis. - The authors did not provide detailed training and testing strategies for its two-stage approach. - In Table 1, it can be observed that the proposed method did not achieve SOTA in terms of MR on the Scan2CAD dataset. What are the possible reasons behind this phenomenon? - In Figure 4, it can be observed that both MIRETR and the proposed method 3DFMNet cannot successfully match all parts on the ROBI dataset, especially on dense scenes with lots of parts. - As mentioned in the limitation, the proposed method is a two-stage method, and its inference time is slightly lower than MIRETR. It is better to discuss a potential plan to solve this issue.

Questions

Please refer to weaknesses.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors did not discuss the computational complexity of method

Change to "Weak Accept"

The rebuttal from the authors addressed most of my questions. Additional experiment results show good support for the proposed method. The potential improvement for this work is predicting additional pair-wise confidence scores or using pair-wise registration accuracy as the indicator to handle the falsely detected objects. Overall, I would like to update the score to "Weak Accept".