Reply to reviewer's questions (part 1/2)
Dear Reviewer UDVe,
Thank you for taking the time to respond to our rebuttal, especially given this busy period. We appreciate your feedback and the opportunity to further clarify the points you find unclear. Below we will answer these questions point by point.
**Q1: Concern on technical contributions**
Thanks for your comments. We understand your concern regarding the novelty. In terms of technical contributions, we have the following points to explain.
- Previous DA/DG methods based on relatively small models for 3D point cloud recognition design common feature space among source and target domain or adapt the meta-learning framework to handle the domain shifts. They provide very valuable insights and solutions to advance the development of 3D DA/DG.
- Recent works, especially large 3D models (e.g., ULIP, ULIP-2, Uni3D), demonstrate even much better zero-shot recognition performances across a wide range of target tasks. The results imply that domain gaps can also be more effectively narrowed down by pre-training large 3D models on large-scale datasets (e.g., ULIP-2 is pre-trained on million-scale pointcloud-text-image triplets). You can treat them as different technical routes/stages to solve the 3D DA/DG problem.
- This work is built on the latter route and the idea of improving the 3DDG ability is to exploit the power of large models. To this end,
- (1) we design an active interaction strategy to align with the pre-trained knowledge in large 3D models,
- (2) we deploy LLMs as powerful interfaces to produce high-quality descriptions to various point clouds,
- (3) we synthesize the opinions from different learning stages with a gaussian-weighted voting.
- Then, these three components are effectively incorporated into a unified regulation framework to handle the category/dataset shifts between seen and unseen domains.
- Finally, we verify the effectiveness of distinct components by ablation experiments and validate the proposed framework on multiple large 3D models and multiple benchmarks to reflect the boosted generalization ability and robustness.
- We also want to note that our framework shows promising model-agnostic attribute, which implies that with the increasing abilities of large 3D models and LLMs, the 3DDG gains will increase.
- For the prompt learning part, our design is distinguished from previous works in:
- we conduct multi-modal prompt learning (both on text and 3d branches) while previous works conduct prompt tuning on a single modality. For instance, PPT [52] only tunes text prompts, IDPT [68], DAPT [82], Point-PEFT [53] tune prompts only in 3D. In the beginning of this project, we compare these two solutions and find our strategy achieves better generalization over single-modal on our benchmarks.
**Q2: Improving literature review**
Thanks for your suggestions. We appreciate that the reviewer clarifies there are many 3D DA/DG methods for **lidar data in autonomous driving**.
After receiving the feedback, we maximize our efforts to find related papers (these works are attached in another reply part due to character limitation). Many of them focus on semantic segmentation, object detection, and registration. We will read them carefully later. There is no doubt that these 3D DA/DG methods are prestigious to the field and we will reflect them in our revised related work section by:
* (1) summarizing these works according to the methodology they proposed;
* (2) explaining the differences and relations to our work.
**Q3: Lacks a specific design to adress domain gaps**
Thanks for your comments. This question is similar to Q1 (concern on technical contributions) and we answer the question in corresponding section, kindly referring to that part. About the question related to transfer learning, we have the following points to provide.
First, we agree transfer learning and DG methods share many similarities. They are both techniques used to improve the performance of models on tasks where there is limited or no data.
Second, they approach this problem from different angles/ways. Let's perceive the difference through some specific examples.
- In transfer learning, a model is typically pre-trained on a large dataset (source domain) and then fine-tuned on a smaller, task-specific dataset (target domain). For example, a model is pre-trained on the ImageNet dataset (which contains millions of images across 1,000 classes) and then fine-tuned on a smaller dataset of medical images to classify different types of skin diseases. By this way, the learned useful features from ImageNet can be transferred to the target domain with relatively little additional training.
- In domain generalization, the model is only trained on source domain and directly tested on unseen target domains, without requiring fine-tuning on the target domain data. In our base-to-new class and cross-dataset settings, we do not fine-tune the prompts on new classes or target datasets.