DA-Ada: Learning Domain-Aware Adapter for Domain Adaptive Object Detection

Summary

The paper presents a Domain-Aware Adapter (DA-Ada) for DAOD based on VLMs, which aims to improve the model performance when applied to an unlabeled target domain. DA-Ada incorporates two types of adapters, including a Domain-Invariant Adapter (DIA) to learn domain-invariant knowledge and a Domain-Specific Adapter (DSA) for injecting domain-specific knowledge. It also includes a Visual-guided Textual Adapter (VTA) to encode the cross-domain visual feature into a textural encoder to enhance detection. Experiments across various tasks show that DA-Ada significantly outperforms existing methods, demonstrating its effectiveness in eliminating the domain gap.

Strengths

1. The proposed method can significantly improve DAOD performance, surpassing state-of-the-art works by a large margin. 2. The general idea of using an adaptor with VLMs to assist DAOD is moderately interesting. 3. The method is simple yet effective.

Weaknesses

1. Some claims are confusing and lack sufficient justifications. 2. Some designs are similar to existing works and lack sufficient technical contributions. 3. The method is only deployed on Faster RCNN. The adaptability to more advanced detectors is unknown. 4. The paper writing and organization need to improve. Details are in Questions.

Questions

1. In Line 139, are there any theoretical proofs or experimental justifications to support the claim that low-dimensional features have less information redundancy and are more suitable for domain adaptation? How can you ensure that low-dimensional representation is domain invariant? I don't think that the feature with less information redundancy means domain invariant. 2. For the Visual-guided textual adapter, this design seems to follow CoCoop [1], a generic technique for improving model learning. The difference needs to be clarified. 3. I am confused about the domain-specific adapter. The authors use the residual parts of the visual features to represent the domain-specific information. Is there any proof or literature that can justify the correctness of this assumption? 4. Can this method be applied to more advanced detectors, such as DETR and CenterNet v2? 5. The paper writing and organization need to improve. The authors propose DITA and DSTA but don't provide any information about them in the abstract. Additionally, the method descriptions and Fig.2/3 are not well matched, leading to the reading difficulty. For example, in Line 184, where are DITA and DSTA in Fig. 2(b)? [1] Zhou, K., Yang, J., Loy, C. C., & Liu, Z. (2022). Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 16816-16825). Considering the significant performance gain of the proposed method, I'm happy to turn to a positive score if concerns are addressed.

Rating

5

Confidence

5

Soundness

2

Presentation

2

Contribution

2

Limitations

No potential negative societal impact.

Summary

This paper presents a method to tackle the domain adaptive object detection (DAOD) task within the framework of visual-language models (VLM). The authors propose a Domain-Aware Adapter (DA-Ada) to enhance the visual encoder's ability to learn both domain-invariant and domain-specific features. DA-Ada consists of two components: the Domain-Invariant Adapter (DIA) and the Domain-Specific Adapter (DSA). The DIA learns domain-invariant features by aligning the feature distribution between the source and target domains, while the DSA recovers domain-specific knowledge from the differences between the input and output of the visual encoder blocks. Additionally, the Visual-guided Textual Adapter (VTA) embeds cross-domain information into the textual encoder to improve detection head discriminability. Experiments on various DAOD benchmarks indicate that DA-Ada improves performance compared to state-of-the-art methods.

Strengths

This paper is well-written and easy to follow. The motivation for explicitly learning additional domain-specific features to enhance cross-domain performance is clear and straightforward. The disentanglement of domain-invariant and domain-specific features proposed in the Domain-Specific Adapter, along with the regularization loss in Eq. 15, appears to be effective. The ablation study is comprehensive and detailed, and the experimental results indicate that the proposed model achieves significant performance improvements compared to other state-of-the-art methods across different domain shift scenarios.

Weaknesses

Despite the paper’s strengths, certain aspects of the discussion could be further refined for precision and clarity: 1. The performance of the source-only baseline in Table 3 outperforms (or achieves similar performance to) most of the non-VLM-based models in Table 1. For example, on the Cross-Weather domain shift, the source-only model achieves an mAP of 50.4, whereas methods like AT[35], which are backbone ImageNet-pretrained, show a performance of 50.9, and their source-only variants perform roughly >10 mAP lower. While using a VLM vision encoder to generate essential general knowledge is reasonable, it raises the question of whether this task should still be categorized as cross-domain (domain adaptive) learning or as transfer learning (or VLM-based domain adaptation). Additionally, it would be helpful to add a column in Tables 1 and 2 to indicate the pretraining. 2. It would be beneficial if the authors could also provide the comparisons about the computational speed

Questions

1. What are the regression losses used in DA-Ada? Are they the same as those used in [28]? 2. The symbols of down-projectors in Eq.6 and Section 6.7 are different. Could you clarify this discrepancy? Additionally, for the down-up projections, which operate on both the channel dimension (Eq.5) and the spatial dimensions, a clearer explanation would be beneficial. 3. Since DA-Ada modules are added to different visual encoder blocks, the claim in line 139 that “Low-dimensional features have less information redundancy…” seems inappropriate. In my opinion, “low-dimensional” features typically refer to features from the earlier layers. In Section 2, Eq. 5 seems more related to “channel dimension” condensation. Additionally, in “Bottleneck Dimension” in Section 4.3 (also Table 7), it would be helpful if the authors could directly indicate the input channel dimensions. 4. It would be beneficial to add some visualizations for failure cases. Some minor problems: 1. Typos in line 183-184? 2. Legend missing in Figure 3 (the red symbols)

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Yes, the limitations are discussed in Sec 6.11.

Summary

This work focuses on domain adaptive object detection (DAOD) with the vision-language models. The core idea behind this paper is the frozen visual encoder with a domain-agnostic adapter only captures domain-invariant knowledge for DAOD. To this end, this paper proposes a novel Domain-Aware Adapter (DA-Ada) to capture the domain-invariant knowledge and domain-specific knowledge. The experimental results on multiple DAOD tasks show the proposed method clearly outperforms existing DAOD methods.

Strengths

1. DAOD is an important problem, especially, in the large vision-language era, we need to rethink and explore the new paradigm for DAOD. 2. The proposed method is reasonable, effective transfer performance should maintain both the transferability (domain-invariant) and discriminability (domain-specific) between source and target domains. 3. The experimental results over multiple benchmarks show the effectiveness of the proposed method. Extensive ablation studies have been conducted to investigate the proposed method.

Weaknesses

1. Although the proposed method is reasonable, it still has limited novelty for the domain adaptation community. The domain-invariant and domain-specific knowledge have been exploited by many previous works. 2. The function of dec loss is to maximize the distribution discrepancy between DIA and DSA. Why is the cosine similarity calculated by $h_i^I$ and $h_i^I \cdot h_i^S$ instead of $h_i^I$ and $h_i^I$. 3. Lack of details for VTA, what are the structures of DITA, DSTA, and DSTA. Besides, it should be added the text description c in Figure 2 (c), the CLIP Textual Encoder adopts both the projected visual feature and the textual feature for better clarity. 4. The author should add results of source-only baseline, i.e., RegionCLIP, for a fair comparison. 5. The Injection Operation has many ways in Table 6, can the authors provide some explanation between $h_i^I + h_i^S$ and $ h_i^I + h_i^I \cdot h_i^S $ 6. In line 184, the figure reference should be Fig.2(c) instead of Fig.3(b).

Questions

1. Do we really need a source domain for adaptation when we already have a powerful vision-language detector that contains much general knowledge from large-scale data?

Rating

5

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

Nothing.

Summary

This article proposes a Domain-Aware Adapter (DA-Ada) tailored for the DAOD task. The key point is exploiting domain-specific knowledge between the essential general knowledge and domain-invariant knowledge. The DA-Ada framework consists of the Domain-Invariant Adapter (DIA) for learning domain-invariant knowledge and the Domain-Specific Adapter (DSA) for injecting the domain-specific knowledge from the information discarded by the visual encoder. Comprehensive experiments over multiple DAOD tasks show the effectiveness of DA-Ada.

Strengths

1. The paper is well-written. 2. The proposed method achieves significant performance improvement compared to several baselines on commonly used benchmarks. The proposed method not only improves the detection performance on the target domain, but also achieves improvement on the source domain.

Weaknesses

1. The DIA module is a sequence of operations involving mapping, dimensionality reduction, slicing, and dimensionality-raising. It is challenging to understand why such operations can extract domain-invariant knowledge. The author should provide a stronger explanation to clarify the rationale behind this design. 2. In line 251 of the text, could the author elaborate on how the source-only adapter and the domain-agnostic adapter each function? This would help readers better understand. 3. Could the author design a quantitative experiment to further visualize which features represent domain-specific knowledge? Additionally, could they compare the traditional adapter with the method proposed in this paper to further demonstrate the effectiveness of the proposed method? The AP metric alone is insufficient to fully explain the motivation behind this method. 4. Since this paper focuses on adapter work, could the author further review the related work on VLM (Vision-Language Models) adapters in more detail? This would help readers better understand the contribution of this paper. 5. The paper mentions that the domain-agnostic adapter easily causes the model to bias towards the source domain. The latest work [1] also addresses the issue of source bias. In the related work and experiments sections, the author should include a discussion and comparison with the latest work. 6. During this training process, are the original CLIP network components (including the visual encoder and the text encoder) completely frozen, with only the adapter and VTA parts being trained? 7. minor error: There are several places between lines 91 and 102 where spaces are missing between sentences. Reference: [1] DSD-DA: Distillation-based Source Debiasing for Domain Adaptive Object Detection[C]//Forty-first International Conference on Machine Learning.

Questions

See weaknesses

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Adequately addressed