CoDA: Collaborative Novel Box Discovery and Cross-modal Alignment for Open-vocabulary 3D Object Detection

Open-vocabulary 3D Object Detection (OV-3DDet) aims to detect objects from an arbitrary list of categories within a 3D scene, which remains seldom explored in the literature. There are primarily two fundamental problems in OV-3DDet, i.e., localizing and classifying novel objects. This paper aims at addressing the two problems simultaneously via a unified framework, under the condition of limited base categories. To localize novel 3D objects, we propose an effective 3D Novel Object Discovery strategy, which utilizes both the 3D box geometry priors and 2D semantic open-vocabulary priors to generate pseudo box labels of the novel objects. To classify novel object boxes, we further develop a cross-modal alignment module based on discovered novel boxes, to align feature spaces between 3D point cloud and image/text modalities. Specifically, the alignment process contains a class-agnostic and a class-discriminative alignment, incorporating not only the base objects with annotations but also the increasingly discovered novel objects, resulting in an iteratively enhanced alignment. The novel box discovery and crossmodal alignment are jointly learned to collaboratively benefit each other. The novel object discovery can directly impact the cross-modal alignment, while a better feature alignment can, in turn, boost the localization capability, leading to a unified OV-3DDet framework, named CoDA, for simultaneous novel object localization and classification. Extensive experiments on two challenging datasets (i.e., SUN-RGBD and ScanNet) demonstrate the effectiveness of our method and also show a significant mAP improvement upon the best-performing alternative method by 80%. Codes and pre-trained models are released on the project page.

Paper

Similar papers

Peer review

Reviewer goao5/10 · confidence 5/52023-06-29

Summary

This paper addresses the challenge of open-vocabulary 3D object detection (OV-3DDet). The proposed framework consists of two modules: the 3D Novel Object Discovery (3D-NOD) module and the Discovery-Driven Cross-Modal Alignment (DCMA) module. The 3D-NOD module utilizes a detector trained on annotations of base categories (without class-specific information) to localize potential objects. It then employs the pre-trained CLIP to discover the potential super categories of these objects. In the DCMA module, two types of alignments are performed. First, the 3D object features are aligned with their corresponding 2D object features using the CLIP image encoder. Second, the 3D object features are aligned with their corresponding text features using the CLIP text encoder. The proposed framework is mainly evaluated on the SUN RGB-D dataset to assess its performance and effectiveness.

Strengths

1. The problem of open-vocabulary 3D object detection addressed in this paper is highly relevant in real-world scenarios where the number of object classes is not predefined or limited. The ability to detect and classify objects in an open-world setting is crucial for applications such as autonomous driving, robotics, and augmented reality. 2. The proposed method is technically sound. The approach of jointly training the novel object discovery module for localization and the cross-modal alignment module for classification is well-motivated.

Weaknesses

1. The experimental results presented in the paper lack convincing evidence and could be improved in several aspects. + First, in Table 4, it would be beneficial to compare the proposed method with the existing method [14], which is the first work on OV-3DDet, as well as [R1], which explores both open-vocabulary recognition and open-vocabulary localization. This would provide a clear assessment of the effectiveness of the proposed approach. Additionally, it is mentioned in line 150 and Figure 2 that the proposed method utilizes both images and point clouds, making an incorrect comparison with [22] in terms of inputs. + Second, in Table 1, the detailed settings of the compared methods, such as distillation and 3D-NOD + Distillation, are unclear. It is essential to clarify how super category annotations for novel objects are obtained without 3D-NOD. Furthermore, it is unclear whether the compared methods are trained with the same number of epochs, which can significantly impact the performance comparison. + Third, all experiments in the main paper are conducted solely on the SUN RGB-D dataset, it would be valuable to include evaluation results on the ScanNetV2 dataset, which contains much more categories and is a more practical setting for open-vocabulary learning. Moreover, it is mentioned in line 227-228 that the remaining categories are treated as novel on the both datasets, indicating the presence of 190 novel categories on ScanNetV2. However, Table B in the appendix only considers 50 categories on ScanNetV2 as novel categories, creating a discrepancy. + Lastly, according to Table 3, the performance seem to be sensitive to the two hyperparameters. It would be beneficial to provide more insights and analysis regarding the sensitivity of these hyperparameters. 2. The paper's writing style tends to be wordy, leading to repeated chunks of information in the introduction, figure captions, and methodology sections. Streamlining the text and avoiding redundancy would enhance the overall readability and clarity of the paper. [R1] Zeng, Yihan, et al. "CLIP2: Contrastive Language-Image-Point Pretraining from Real-World Point Cloud Data." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

Questions

1. The correctness of Eq. 4 is questionable. It is unclear why a novel object must have an IoU over 0.25 with objects from base categories. Further clarification and justification from the authors would be helpful to understand the rationale behind this threshold. 2. It is not explicitly mentioned whether the base objects are included in O^novel. If the base objects are not part of O^novel, it is important to clarify how the base objects contribute to the cross-modal alignment when only O^novel is used in computing the alignment loss (Eq. 8). 3. In line 235, the meaning of "additional 32 object queries" is not explicitly explained. It is unclear whether it refers to the total number of queries in this alignment being 128+32.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

The authors did not discuss its limitations and potential negative societal impact of their work.

Reviewer zhog6/10 · confidence 3/52023-07-06

Summary

This paper aims to address the problem of detecting objects from an arbitrary list of categories within a 3D scene, which is less explored area in the literature. It proposes a unified framework that simultaneously addresses the fundamental problems of localizing and classifying novel objects, under the condition of limited base categories. The framework incorporates a 3D Novel Object Discovery (3D-NOD) strategy to localize novel 3D objects, which utilizes both 3D box geometry priors and 2D semantic open-vocabulary priors to generate pseudo box labels of the novel objects. Besides, it introduces a cross-modal alignment module based on discovered novel boxes, to align feature spaces between 3D point cloud and image/text modalities for novel object box classification. Experiments on the SUN-RGBD and ScanNet demonstrate the effectiveness.

Strengths

1. The idea makes sense to me. The framework includes a system that discovers new 3D objects and aligns them with other types of data. This helps the system learn better and eliminates the need for a 2D detector. The system also uses CLIP semantics to find objects in 3D scenes during training. 2. The proposed framework performs much better than other methods on the ScanNet-200 and SUN RGBD datasets and achieves a significant improvement in mAP.

Weaknesses

1. The paper doesn't discuss how the proposed framework's computational complexity compares to other state-of-the-art methods, or how efficient it is. 2. The paper doesn't include an analysis of cases where the proposed framework fails to detect objects. 3. It would be helpful to analyze how the convergence rate is affected when the novel object list is similar to the ScanNet-200 dataset. 4. The experiments don't compare the proposed framework to a fully supervised ScanNet-200 method, which would help readers understand the gap in detecting novel objects. Additionally, comparing the proposed framework to an oracle-supervised method with the same model would be useful in understanding the usefulness and importance of the framework in real-world scenarios.

Questions

1. It would be interesting to see if the proposed model can detect objects beyond the 200 categories in ScanNet-200. Some visualizations or illustrations that demonstrate the upper bound of the proposed method would help readers understand its potential capabilities.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The paper doesn't explicitly discuss the limitations of the proposed method.

Reviewer Qxx55/10 · confidence 5/52023-07-10

Summary

This paper proposes an end-to-end open-vocabulary 3D object detection (OV-3DDet) framework which can learn to localize and classify novel objects simultaneously. To achieve OV-3Det, the author designed a 3D Novel Object Discover (3D-NOD) strategy which utilizes both the 3D box geometry priors and 2D semantic open-vocabulary priors to generate pseudo box labels of the novel objects and develop a Discovery-Driven Cross-Modal Alignment (DCMA) module based on discovered novel boxes, to align feature spaces between 3D point cloud and text modalities for better feature alignment.

Strengths

1. The key difficulty of Open-vocabulary 3D Object detection lies in the localization of novel categories. The paper proposes a novel pseudo novel object annotation generation approach, 3D Novel Object Discovery (3D-NOD) by utilizing the cross-priors from both 3D and 2D domains. Different from previous work, the approach discards the reliance on any additional 2D detection model. The model can learn to localize more objects by updating the novel box label pool. 2. The author provides a detailed analysis of the proposed open-vocabulary detection framework and verifies the benefits of each proposed module. The paper conducts relatively detailed ablation experiments for each proposed module and provides a clear introduction to the implementation details within each module. 3. This paper is relatively clear and easy to follow.

Weaknesses

1. There are minor grammar and spelling issues in the paper, such as: 1. Line 161: We then combine both the objections (-> objectness) p_n from the 3D geometry priors a xxxxx. 2. Line 235: For feature alignment, we randomly selected (-> select) an additional 32 object queries to be involved in the alignment with the CLIP model. 2. Cross-modal feature alignment is common in previous open-vocabulary 3D object detection methods, such as OV-3DET. From my perspective, the DCMA method in the paper demonstrates that not only at the feature level for seen class objects but also on textual alignment for novel class objects should be considered in point-cloud-text feature alignment. From the overall description, it appears that this is not a novel solution but rather an integrated utilization of previous work. 3. The novel box labels generated by the 3D-NOD approach originate from inaccurate 3D bounding boxes produced by the model during training, which are filtered based on certain conditions. These generated novel box labels still have issues in terms of accuracy in both position and size, which limits the final novel object localization accuracy. 4. In Section 3.2, there is inconsistency in the use of symbols during the description of details, and it is rather complex. For example, $l_{n}^{3D}$ and $l_{n}^{2D}$ are unnecessary because people can understand what they are; the image encoder $E_{I,n}^{CLIP}$ can be simplified as $E_{img}$.

Questions

1. In line 186, the author claims that "even if the box covers a background region, the class-agnostic distillation still narrows the distance between the two modalities, leading to a more general alignment on different modalities and scenes." However, I didn't see any experiments to support such claim. 2. In line 235, the paper states "For feature alignment, we randomly selecte an additional 32 object queries to be involved in the alignment with the CLIP model." Why do we need to select additional 32 object queries to conduct the feature alignment with CLIP model, not using the 128 object queries? 3. Figure 3 shows that the AP_novel keeps stable and AR_novel even decreases after 1080 training epochs. Why are the long epoch training settings used for the final model ? 4. The paper lacks some experiments to demonstrate the upper bound of the detection performance. For example, the setting of training a class-agnostic 3DETR model on all the categories (base + novel) annotations and using CLIP to classify the object during inference could be added to demonstrate the how well the cross-modal feature is aligned. 5. Is it possible to show the cross-dataset evaluation performance, for example, training on ScanNet and test on SUN RGBD?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The generalization ability of this method is unknown. For example, the open-vocabulary ability remains unknown or limited when more categories (e.g., 200) are introduced.

Reviewer DwKh6/10 · confidence 4/52023-07-10

Summary

This paper is about addressing open-vocabulary 3D object detection, which describes a method aiming for two key challenges, localization and classification for novel objects in point-cloud. The key components are 3D novel-object discover module and the discovery-driven cross-modal alignment module. The first module tries to find as many as possible novel objects based on both geometry and semantic while the second module attempts to align the features of different modalities. The proposed method shows performance improvements compared with the designed baselines.

Strengths

The paper is well-written and shows superior performance improvement compared with the designed baseline.

Weaknesses

I have concerns about both the method novelty and the experiment details. Regarding the method novelty, the key module DCMA is very similar to [15], the difference seems that instead of using contrastive learning, the authors proposed to use distillation loss. However, there is neither insightful discussion nor experiments to analyze why the proposed method is better. Regarding experiments, [15] is discussed to be similar to this paper in the introduction while no more comparison in experiments, the authors should compare either the whole model or the key components. On the other hand, I also have concerned about the fairness of the experiments based on 3DETR, as far as I know, 3DETR has much better performance on both SUNRGBD and ScanNet datasets. For the "base" category setting of Table 1, the performance is extremely low, does it mean using CLIP will degenerate the performance of 3DETR? How about directly using the same setting? Furthermore, it is necessary to evaluate the performance on top of different point-cloud detectors instead of only 3DETR.

Questions

My main concern is about the discussion to [15] and more necessary experimental analysis. Please refer the weakness part.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

The work is only designed based on 3DETR, an indoor point-cloud detector. It should be evaluated based on more detectors.

Authorsrebuttal2023-08-15

Dear Reviewer DwKh, thank you very much for your valuable suggestions and time. Regarding the experiment mentioned in Q3, the model training is finished, and our method achieves clearly better performance than 3DETR using the same setting, confirming the benefits of using CLIP. May we ask if you would like to see the detailed results? According to the discussion rule, we can provide the results when you are interested.

Reviewer DwKh2023-08-16

Dear Authors Yeah, please provide the results. A clarification also should be addressed as that why 3DETR performs not as well as the original paper reported, for example, 3DETR achieves 62&58 AP25 on ScanNet&SunRGBD, while your "base" setting only got ~30. Why the model performs much worse. So I have concerns that CLIP degenerates performance. What I expect is that your method will not degenerate the "BASE" setting while is able to improve the detection for novel classes. Besides, different class-splitting settings should also be conducted, e.g., Do the different "base" classes and "novel" classes have different improvements? Could you also provide it? Thanks!

Authorsrebuttal2023-08-19

Dear Reviewer DwKh, thank you for your reply and valuable suggestions. We have conducted experiments these days for your questions and are now providing you with further answers: **Q1 Regarding the results of our model trained in the close-vocabulary 3DETR settings:** As suggested, we trained our method using the same 10 categories as 3DETR, without considering the novel categories (*i.e.*, keeping the cross-modal alignments and disabling 3D-NOD for novel box discovery), and report the results in Tab. A shown below. It can be observed that our method achieves an $\text{AP}_{0.25}$ of 57.98, which outperforms 3DETR, suggesting the potential benefits of incorporating CLIP. | | | | | | | | | | | | | |:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:| | Methods | **Mean** | chair | table | desk | bed | sofa | toilet | dresser | nightstand | bookshelf | bathtub | | 3DETR | 56.80 | 68.00 | 50.00 | 28.70 | 81.80 | 58.30 | 90.30 | 28.60 | 56.60 | 27.50 | 77.60 | | Ours | **57.98** | 69.33 | 49.01 | 28.79 | 85.54 | 62.88 | 91.61 | 29.65 | 61.89 | 25.88 | 75.27 |  Tab. A: Performance comparison with the 3DETR [19] in the same close-vocabulary 3DETR settings. Regarding the mentioned $AP_{0.25}$ performance of 3DETR, *i.e.*, 58 on SUNRGBD, we have identified an inconsistency between the camera-ready version [A] and the arXiv version [B] of the 3DETR paper. Specifically, Tab. 1 in the camera-ready version [A] reports an $AP_{0.25}$ of 56.8 for 3DETR, while Tab. 1 in the arXiv version reports an $AP_{0.25}$ of 58.0. To further investigate the inconsistency, (i) we carefully check the detailed comparisons in Tab. 11 in the supplemental material of both **the camera-ready and the arXiv versions**, which reveal that the $AP_{0.25}$ of 3DETR is consistently reported as 56.8 in both versions. (ii) we also carefully check the per-class AP comparisons in Tab. 9 in the supplemental material of both the camera-ready and the arXiv versions. Then we calculate the mean AP of the 10 categories, resulting in a mean AP of 56.74. Based on all of the above details, we consider $AP_{0.25}$ of 56.8 shown in the camera-ready version as the performance of 3DETR, and use this number for our comparison, as shown in Tab. A above. [A] Ishan Misra, Rohit Girdhar, and Armand Joulin. An end-to-end transformer model for 3d object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2906–2917, 2021. [B] Ishan Misra, Rohit Girdhar, and Armand Joulin. An end-to-end transformer model for 3d object detection. arXiv preprint arXiv:2109.08141, 2021 **Q2 The setting difference between the $AP_{Base}$ of '3D-CLIP' and the AP in the 3DETR paper:** The main reason why there is a gap between the $AP_{Base}$ of '3D-CLIP' and the AP in 3DETR paper is that **the category setting is significantly different**. As we clarified for this point in the answer for Q3 in the rebuttal, following the typical category-splitting strategy widely considered in 2D open-vocabulary object detection, we use top-10 categories with the highest number of training samples as the base categories. We show the category names in Tab. C in the rebuttal PDF. There are **5 categories in our base category setting that do not overlap with the 3DETR category setting**. These 5 categories may be harder for point cloud object detection. For instance, the $AP_{0.25}$ of the category 'box' and 'computer' is only 0.57 and 2.41, respectively. This is probably because their geometric structures mostly resemble simple cubes and lack distinctive information. The worse performance in these hard categories leads to a lower mean AP in our setting. Following your suggestions, we also conducted the experiments with the same 10 categories of 3DETR as our base categories shown in the answer to the next question. Due to the space limitation, the answers to the next questions are provided in **the following reply**.

Authorsrebuttal2023-08-19

**Q3 Will CLIP degrade the performances?** We answer this question from the following three aspects: (i) When we **do not consider novel categories** and train our model in the close-vocabulary setting (*i.e.*, keeping the cross-modal alignments and discarding the 3D-NOD), CLIP does not degrade the performance as shown in Tab. A in last reply. This is because, during the training, the CLIP knowledge can be adapted for 3D detection on the base categories by learning the cross-modal alignment. (ii) When we **consider novel categories and directly apply pre-trained CLIP in the testing stage** to achieve open-vocabulary detection (*i.e.*, '3D-CLIP', the open-vocabulary baseline model), the mAP of the base categories of '3D-CLIP' is lower than the AP of 3DETR, as shown in Tab. B below. The base categories of our method in this experiment are the same as the categories in the 3DETR paper. It is reasonable because the pre-trained CLIP is directly used for open-vocabulary classification in the testing. There is no cross-modal alignment between CLIP and the target data distribution during training. However, '3D-CLIP' can detect objects of novel categories and obtain an $AP_{Novel}$ of 2.39, while the 3DETR cannot perform novel object detection, further confirming the benefits of using CLIP for the problem. (iii) When we **consider novel categories and involve CLIP during training in our designed method**, the $AP_{Novel}$ and $AP_{Base}$ of our method are both significantly higher than the open-vocabulary baseline model '3D-CLIP', as shown in Tab. B below. While $AP_{Base}$ of our method is slightly lower than 3DETR, we think it is a reasonable phenomenon since 3D-NOD may also possibly discover noisy 3D boxes and the method needs to jointly learn both the base and the novel categories which is more challenging compared to 3DETR that only learns the base categories. |Methods | $\text{AP}_{Novel}$ | $\text{AP}_{Base}$ | |:-------:|:-------------------:|:------------------:| | 3DETR | - | 56.80 | | 3D-CLIP | 2.39 | 41.64 | | Ours | **5.13** | **55.26** | ***Tab. B:** Performance comparison using the same 10 categories in the 3DETR paper as the base categories. '3DETR' results show the performance of the original camera-ready version of 3DETR. '-' indicates that the close-vocabulary 3DETR cannot perform detection on novel categories. '3D-CLIP' is our open-vocabulary baseline model. Our method achieves significant improvements on both $AP_{Novel}$ and $AP_{Base}$ upon the baseline.*   **Q4 Other class-splitting settings:** We follow the suggestion of the reviewer and set the same 10 categories of 3DETR as our base categories. As shown in Tab. B above, our method also achieves significant improvements on both $AP_{Novel}$ and $AP_{Base}$ compared to the baseline model, further demonstrating the effectiveness of our proposed method.

Authorsrebuttal2023-08-21

Dear Reviewer DwKh, thank you very much for providing your valuable comments. As the deadline is approaching, we would like to kindly inquire if our discussions have addressed your concerns. If you have any further questions, we would be happy to continue our conversation. If our response has cleared your concerns, we would greatly appreciate it if you could consider updating your score and leaving your feedback. Thank you again.

Authorsrebuttal2023-08-15

Dear reviewers, we sincerely appreciate your time and valuable comments. If you have any questions regarding the rebuttal, we would be delighted to discuss them with you.

Reviewer goao2023-08-17

Post-rebuttal Comment

Thank the authors for their thorough response, the clarifications provided, and the additional results. Many of my concerns have been addressed. However, I have two more questions: 1. Why wasn't there a comparison with [15] on the 20 common classes on SUN RGB-D? 2. The response to Q7 still lacks persuasiveness in my view. It seems evident that varying thresholds in 3D-NOD lead to distinct performance differences. How can the authors assert that "This demonstrates that the effect of 3D-NOD is not sensitive to these hyperparameters"?

Authorsrebuttal2023-08-18

Dear Reviewer goao, thank you very much for your careful reading of the rebuttal, and we sincerely appreciate your further reply. We provide the following answers to your further questions: **Q1 Comparison with [15] on the 20 common classes on SUN RGB-D:** We would like to clarify the **significant differences in the settings** between ours and [15]. In [15], they adopt an extra open-vocabulary 2D detection model [34] to generate large-vocabulary 3d pseudo labels on 1k categories. Then their model is trained by these large-vocabulary 3d pseudo labels. While in our settings, rather than using any open-vocabulary 2D detection model, we train a base model with 3D labels of base categories (10 categories). Then we discover more and more objects of novel categories during training. As the distinct difference in the task settings, it is not straightforward to directly compare our model with theirs. In order to provide a fair comparison with [15], we need to adapt our model to their setting. Specifically, we train our base model by **the same 3d pseudo labels** with [15], then discover more and more objects during training. We appreciate the authors of [15] for releasing their codes for generating pseudo labels on ScanNet. This enables us to train our method using the same pseudo labels and evaluate our model under the same setting. As shown in Tab. B of the rebuttal PDF, our method achieves **clearly better mean AP (1.3 points improvement)**, proving the superiority of our method. While the codes of [15] for generating 3D pseudo labels **for SUN-RGBD are not released yet** at the time of our reply. Additionally, the ScanNet and SUN-RGBD are captured using different sensors, which introduces significant gaps in the data and processing for the pseudo-label generation. Thus, the codes for ScanNet cannot be directly transferred to SUN-RGBD without detailed descriptions. To ensure fairness and correctness, we only compare our method on ScanNet for the present. Considering that (i) both our model and [15] do not have dataset-specific designs for either ScanNet or SUN-RGBD, and (ii) our analysis in response to Q5 of the rebuttal highlights the consistency of our method across the two datasets. The already provided comparison on ScanNet can **effectively validate the superiority of our methods**. We will also contact the authors of [15] for the remaining parts of source codes and update the comparison on SUN-RGBD in the final paper. **Q2 The sensitivity of these hyperparameters:** We apologize for the confusion caused. We wanted to express that our 3D-NOD method can produce stable improvements with different thresholds from a wide range, demonstrating that its effectiveness is not reliant on carefully selected thresholds. As shown in Tab. 3 in the main paper, despite the performance variability, all our models trained with different threshold settings can consistently outperform the model trained without 3D-NOD (*i.e.*, ‘0.0&0.0’ in the first row of Tab. 3) by a significant margin of 70% or more, clearly verifying its advantage. We will further revise our statement about the sensitivity in the revision to make this point clearer.

Authorsrebuttal2023-08-21

Dear Reviewer goao, thank you very much for providing your valuable comments. As the deadline is approaching, we would like to kindly inquire if our discussions have addressed your concerns. If you have any further questions, we would be happy to continue our conversation. If our response has cleared your concerns, we would greatly appreciate it if you could consider updating your score and leaving your feedback. Thank you again.

Reviewer zhog2023-08-18

Thanks for the authors' rebuttal

The rebuttal has addressed my concerns well. I raise the rating to weak accept.

Authorsrebuttal2023-08-18

Dear Reviewer zhog, thank you very much for reviewing our paper, providing valuable suggestions, and carefully reading the rebuttal. We appreciate your time and effort. Your recognition of our work is truly encouraging.

Authorsrebuttal2023-08-19

Dear Reviewer Qxx5, thank you sincerely for your time and efforts. We greatly appreciate your valuable comments. If you have any questions regarding the rebuttal, we would be delighted to have a discussion.

Authorsrebuttal2023-08-21

Dear Reviewer Qxx5, thank you very much for providing your valuable comments. As the deadline is approaching, we would like to kindly inquire if our discussions have addressed your concerns. If you have any further questions, we would be happy to continue our conversation. If our response has cleared your concerns, we would greatly appreciate it if you could consider updating your score and leaving your feedback. Thank you again.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC