Library

Unified Domain Generalization and Adaptation for Multi-View 3D Object Detection

Recent advances in 3D object detection leveraging multi-view cameras have demonstrated their practical and economical value in various challenging vision tasks. However, typical supervised learning approaches face challenges in achieving satisfactory adaptation toward unseen and unlabeled target datasets (\ie, direct transfer) due to the inevitable geometric misalignment between the source and target domains. In practice, we also encounter constraints on resources for training models and collecting annotations for the successful deployment of 3D object detectors. In this paper, we propose Unified Domain Generalization and Adaptation (UDGA), a practical solution to mitigate those drawbacks. We first propose Multi-view Overlap Depth Constraint that leverages the strong association between multi-view, significantly alleviating geometric gaps due to perspective view changes. Then, we present a Label-Efficient Domain Adaptation approach to handle unfamiliar targets with significantly fewer amounts of labels (\ie, 1$\%$ and 5$\%)$, while preserving well-defined source knowledge for training efficiency. Overall, UDGA framework enables stable detection performance in both source and target domains, effectively bridging inevitable domain gaps, while demanding fewer annotations. We demonstrate the robustness of UDGA with large-scale benchmarks: nuScenes, Lyft, and Waymo, where our framework outperforms the current state-of-the-art methods.

cs.CVComputer ScienceEngineering
Source

Paper

Similar papers

Unsupervised Domain Adaptation with Multimodal Fusion for Monocular 3D Object DetectionJin Jiang, Jidong Dai, Wei Li et al.2026 · Vehicles In Library
UniDA3D: A Unified Domain-Adaptive Framework for Multi-View 3D Object DetectionHong Wu, Cheng Chi, Jinlin Wu et al.2026 · arXiv.org · 0 citations In Library
BEVUDA++: Geometric-aware Unsupervised Domain Adaptation for Multi-View 3D Object DetectionRongyu Zhang, Jiaming Liu, Xiaoqi Li et al.2025 · arXiv (Cornell University) · 5 citations In Library
BEVUDA++: Geometric-Aware Unsupervised Domain Adaptation for Multi-View 3D Object DetectionRongyu Zhang, Jiaming Liu, Xiaoqi Li et al.2024 · IEEE Transactions on Circuits and Systems for Video Technology In Library
MUSDA: Multi-source Multi-modality Unsupervised Domain Adaptive 3D Object Detection for Autonomous DrivingXiaohu Lu, Hamed Khatounabadi, Hayder Radha2026 · arXiv.org · 0 citations In Library
STAL3D: Unsupervised Domain Adaptation for 3D Object Detection via Collaborating Self-Training and Adversarial LearningYanan Zhang, Chao Zhou, Di Huang2024 · IEEE Transactions on Intelligent Vehicles In Library
SSDA3D: Semi-supervised Domain Adaptation for 3D Object Detection from Point CloudYan Wang, Junbo Yin, Wei Li et al.2023 · Proceedings of the AAAI Conference on Artificial Intelligence In Library
SSDA3D: Semi-supervised Domain Adaptation for 3D Object Detection from Point CloudYan Wang, Junbo Yin, Wei Li et al.2022 · arXiv (Cornell University) · 64 citations In Library
CMDA: Cross-Modal and Domain Adversarial Adaptation for LiDAR-Based 3D Object DetectionGyusam Chang, Wonseok Roh, Sujin Jang et al.2024 · Proceedings of the AAAI Conference on Artificial Intelligence In Library
CMDA: Cross-Modal and Domain Adversarial Adaptation for LiDAR-Based 3D Object DetectionGyusam Chang, Wonseok Roh, Sujin Jang et al.2024 · arXiv (Cornell University) · 11 citations In Library

Peer review

Accept (poster)NeurIPS 2024 · via OpenReview
Reviewer 2JqY4/10 · confidence 4/52024-07-12

Summary

The paper proposes an Unified Domain Generalization and Adaptation(UDGA) scheme for multi-view 3D object detection. The main components of the proposed method are (1) depth inconsistency-based constraints from multi-view and (2) an efficient domain adaptation scheme (LEDA). The multi-view depth inconsistency constraints improve the domain gap caused by geometric gaps when it comes to perspective view changes. LEDA offers adaptation with fewer amounts of labels while preserving the knowledge from the source domain.

Strengths

The paper clearly addresses which problem it focuses on, and each proposed module has solid goals to address existing limitations. The experimental results from multiple datasets also seem convincing.

Weaknesses

I have a few concerns about the demonstration of the proposed method as well as addressing existing works. (1) The method part needs to be improved. For instance, specifying every input and output tensor dimension (i.e., depth estimation network, etc.) would significantly help readers understand what each module is doing more quickly. (2) Using the depth inconsistency seems similar to DETR3D's[1] inconsistency constraint on RGB features. In DETR3D, it is even mentioned that using the RGB feature is better than having explicit depth estimation. Based on this, the depth estimation network can introduce additional parameters for optimization while not being very helpful. I cannot see experiments or comparisons with DETR3D. (3) More related works need to be addressed. Currently, the paper focuses on multi-view-based domain generalization for 3D detection. However, there is also another active line of research with LiDAR-based domain adaptation for 3D detection, pioneered by ST3D[2]. Clearly addressing the difference between those researches will make the reader recognize the line of research that the paper focuses on. (4) The effectiveness of LEDA is not well demonstrated, as claimed in the paper, compared to existing approaches. [1] Detr3d: 3d object detection from multi-view images via 3d-to-2d queries, Wang, Yue, et al. CORL, 2022 [2] St3d: Self-training for unsupervised domain adaptation on 3d object detection, Yang, Jihan. et al., CVPR 2021.

Questions

I have a few questions that correspond to the weakness that I mentioned. (1) Is there a justifiable reason why using Depth inconsistency constraint is superior to using deep RGB feature-based inconsistency ( as in DETR3D)? (2) As one of the main proposed contributions claimed, a detailed analysis of LEDA seems missing. How much does LEDA improve the proposed system compared to existing approaches in terms of efficiency and accuracy? Addressing those two questions properly will improve the quality of the paper and, subsequently, the reviewer's rating.

Rating

4

Confidence

4

Soundness

3

Presentation

2

Contribution

2

Limitations

The limitations are addressed

Reviewer 2JqY2024-08-11

I appreciate the authors' effort to answer my questions. I am convinced by the authors' responses about "RGB-features" and "In-depth explanation of LEDA". However, the authors' response to "Status of LiDAR-based cross-domain object detection" made me sceptical about what the authors meant by "(1) These methods were all proposed under the assumption of target-aware conditions, which presents the limitation that they cannot be applied without prior access to the target domain." and (2) "LiDAR-based methods were less affected by domain shifts compared to those of cameras, though their performance did not reach the oracle level." in the rebuttal. Regarding (1), did the authors mean that LiDAR-based self-training methods require target statistics, such as average object size, and self-training requires access to the target domain? From the reviewer's understanding, the proposed method LEDA even requires labels directly from the target domain(although it's a comparably small amount, i.e. 1%- 5%), whereas the LiDAR-based self-training methods do not require such direct labels. In the context that LEDA requires target labels and LiDAR-based methods require self-training, it seems to me that they both require prior access to the target domain. Regarding (2), if the authors meant that LiDAR-based methods haven't met the Orcal performance in Waymo to KITTI adaptation task, I must say that this is false. For instance, DTS [1] and so on already outperformed Oracle with the justifiable reasons in Waymo to KITTI adaptation task. I understand that the multi-view-based and LiDAR-based Domain adaptation/generalization methods have different characteristics and cannot be directly comparable. However, as a similar branch of work, the LiDAR-based domain adaptive 3D detection also needs to be addressed with proper description so that readers can understand the lines of similar works. Specifically, it does not sound very convincing to advertise that multi-view-based domain generalization is more challenging than LiDAR-based domain generalization because of the above-mentioned reasons for (1) and (2). I personally think that just mentioning LiDAR-based domain adaptive/generalization 3D detection works as relevant works without comparing which one is more challenging would make it clearer for readers to know what the relevant lines of research are. [1] Density-Insensitive Unsupervised Domain Adaption on 3D Object Detection, Hu et al. CVPR 2023

Authorsrebuttal2024-08-14

Dear Reviewer 2JqY, We sincerely appreciate your valuable efforts and professional feedback, which have indeed improved the quality of the final version of our manuscript. Above, we've prepared answers to your remaining concerns regarding the **clear description of the relationship between lidar- and camera-based domain adaptation literature.** If it appears to be reasonable, could you please update your rating of our work, as you indicated in your initial review? As the authors-reviewers discussion period is nearing its end, we would like to politely request your final verdict on this matter. Best regards, The Authors

Authorsrebuttal2024-08-12

Revision of the Status of LiDAR-based Cross-Domain Object Detection

We genuinely appreciate the reviewer's thoughtful and detailed feedback. Our proposed generalization technique, **Multi-view Overlap Depth Constraint, does not require prior access to the target**, unlike Statistical Normalization (SN) [1]. However, due to its limited performance, we designed LEDA to efficiently learn novel target knowledge, and we agree with your observation that **LEDA indeed falls under the category of methods requiring direct labels from target domain.** Furthermore, we acknowledge that our lack of insight into UDA for LiDAR 3D Object Detection led to an insufficient comparison in the answer (3) 'Status of LiDAR-based Cross-Domain Object Detection'. To address these issues, we will revise rebuttal phrases (1) and (2) as follows: Regarding (1): “These methods were all proposed under the assumption of target-aware conditions, which presents the limitation that they cannot be applied without prior access to the target domain.” $\rightarrow$ “These methods adopt 'prior access to target' approaches (e.g., SN [1], self-training [2, 3]) to effectively align the domain gaps between the source and target.” Regarding (2): “LiDAR-based methods were less affected by domain shifts compared to those of cameras, though their performance did not reach the oracle level.” $\rightarrow$ “~LiDAR-based methods were less affected by domain shifts compared to those of cameras, though their performance did not reach the oracle level.~” More importantly, we observe that **the challenges of LiDAR-based methods are also aligned with our problems in a certain aspect (i.e., label-efficient learning).** LiDAR-based approaches utilize self-training strategies with pseudo-labeling; however, **our UDGA framework introduces a method for fine-tuning LEDA using a small subset of novel target labels, and applying LEDA to LiDAR-based 3D object detection represents a promising direction for future work.** We will clearly mention this discussion in the final manuscript and agree that this explanation will indeed enhance the readability of our paper. Thanks for your thorough and professional comments on this comparison. ---- [1] Wang, Yan, et al. "Train in germany, test in the usa: Making 3d object detectors generalize." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020. [2] Yang, Jihan, et al. "St3d: Self-training for unsupervised domain adaptation on 3d object detection." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021. [3] Hu, Qianjiang, Daizong Liu, and Wei Hu. "Density-insensitive unsupervised domain adaption on 3d object detection." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

Reviewer gahV5/10 · confidence 4/52024-07-12

Summary

This paper is about Unified Domain Generalization and Adaptation for outdoor 3D object detection. To address the geometric misalignment between the source and target domains, Multi-view Overlap Depth Constraint that leveraging the strong association between multi-views and Label-Efficient Domain Adaptation are proposed. Experiments are done with several cross-dataset settings.

Strengths

1. The domain generalization and adaption is important for 3D object detection. 2. The writing is easy to follow.

Weaknesses

1. In the past works, the comparative methods, such as DG-BEV, and PD-BEV, reported the performance on both source and target domains. Yet it seems that the reported metrics in Table 1 do not align with the reported results in the two papers. What is the reason for such changes? 2. It is noteworthy that the NDS is specific for nuScenes and different for other datasets. It is important to discriminate such differences. 3. The authors claim that this work is toward 3D universal detection, yet there are other works toward 3D universal detection, such as [1][2]. It is better to illustrate the differences over these methods, both in discussion and experimental comparison. Besides, there are other works devoted to multi-dataset training for 3D detection for generalization, also missed in the paper. Can the proposed method be applied in multi-source training and multi-target testing? 4. The Domain Adapter design seems simple, yet the motivation and potential of such design is missed. It is important since it is one of the two components of this work. [1] Towards universal LiDAR-based 3D object detection by multi-domain knowledge transfer. ICCV 2023. [2] Cross-Dataset Sensor Alignment: Making Visual 3D Object Detector Generalizable. PMLR 2023.

Questions

The experimental comparison and the novelty, importance of the work.

Rating

5

Confidence

4

Soundness

2

Presentation

3

Contribution

2

Limitations

The limitation part has been addressed.

Authorsrebuttal2024-08-14

Dear Reviewer gahV, We greatly appreciate your valuable efforts and professional feedback, which have indeed improved the quality of the final version of our manuscript. We’ve provided answers to your remaining concerns above, and it would be great to hear your feedback on our rebuttal so that we can further improve the final version. Although the authors-reviewers discussion period is nearing its end, we are fully prepared to address any further questions you may have. Best regards, The Authors

Reviewer LaL55/10 · confidence 3/52024-07-14

Summary

The paper presents an adaptation of 3D object detectors to varying target environments using two major strategies. The proposed multi-view overlap depth constraint leverages associations across views to learn view-invariant features. Additionally, a LORA-like structure is designed for parameter-efficient adaptation, accommodating scenarios with limited target data. Experiments on three benchmark datasets demonstrate the effectiveness of the proposed approach, with minimal modification of parameters.

Strengths

+ The paper addresses a significant and practical issue in 3D object detection, aiding in the development of robust models for dynamically changing testing environments. + The flowchart clearly illustrates the core components, making it easy for readers to grasp the main idea. + Extensive experiments have been conducted, and the proposed strategy significantly outperforms the pre-trained source model and even surpasses full fine-tuning strategies.

Weaknesses

- The two proposed strategies appear somewhat disconnected. While the multi-view idea is interesting, the technical details are confusing. Equation (5) includes three different objectives, but their importance or sensitivity is not discussed. The latter adaptation strategy resembles existing works and lacks specific design for the 3D detection task. This is also not correlated with the view-transform. The isolated adaptation modules seem more like a stacking of existing techniques rather than a cohesive addition. - More discussion and comparisons with previous multi-view augmentation strategies would help clarify the merits and innovations of the proposed approach. The current claim that existing strategies are poorly generalizable is somewhat unconvincing.

Questions

Please refer to the weakness section.

Rating

5

Confidence

3

Soundness

2

Presentation

2

Contribution

2

Limitations

The authors have not provided a discussion on the limitations of their work.

Reviewer pgwk7/10 · confidence 3/52024-07-16

Summary

This paper focuses on multi-view 3D object detection. The authors proposed a unified domain generalization and adaptation-based detection method. To enhance the detection model for unseen datasets and address the geometric misalignment problem, the authors proposed a multi-view overlap depth constraint module and a label-efficient domain adaptation technique. The comprehensive experiments on lar-scale datasets, including nuScenes, Lyfy and Waymo have demonstrate the effectiveness of the proposed method.

Strengths

- The storyline is clear, the authors provided detailed motivation analysis in the introduction and present their innovation clearly. - The performance is strong. The method shows SOTA performance in variance benchmarks with fast speed. - Most of the figures are clear. This paper is also well-written.

Weaknesses

- There is no source code for review.

Questions

This paper seems well-shaped to me. However, I'm not an expert in this field, so I'm open to discussion with other reviewers if they hold opponent opinions to me.

Rating

7

Confidence

3

Soundness

3

Presentation

4

Contribution

3

Limitations

Please refer to the above.

Reviewer LaL52024-08-10

Thanks for the reponse.

I would like to appreciate the detailed response from the authors and most of my questions have been addressed. As I agree with other reviewers, the proposed multi-view approach does not seem specifically designed for DG problem. However, based on the impressive improvement compared with the commonly seen augmentation strategy, I think the proposed module has the merit to facilitate future work. Thus, I would like to increase my score to borderline accept.

Authorsrebuttal2024-08-10

Thank you for the positive feedback

We are delighted that our responses to your questions have been well received and led to a positive evaluation of our work (4 → 5). As you acknowledged, our practically motivated framework achieves state-of-the-art performance. We also hope that our work provides new perspectives on the unified view of domain generalization and adaptation for future research.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC