Enhancing Domain Adaptation through Prompt Gradient Alignment

Summary

This paper aims to leverage prompt tuning of vision-language models for Unsupervised Domain Adaptation (UDA) tasks. The authors formulate UDA as a multi-objective problem where each objective is modeled by a domain loss. To resolve conflicts between domains, they manipulate gradients by maximizing their cosine similarity. Additionally, to stabilize the training procedure, they propose using the norm of the gradients as a regularization term. They also provide a generalization error bound for their method. Empirical results demonstrate the effectiveness of the proposed method.

Strengths

The paper is well-organized and the presentation is clear. To my knowledge, the idea of aligning gradients of different objectives by maximizing cosine similarity and regularizing the gradient norm is intuitive and novel. The authors also provide a generalization bound for the proposed method, which is a valuable contribution.

Weaknesses

Major: (1) Recent empirical and theoretical studies [1] show that simply reweighting the loss of different objectives can match the performance of gradient surgery. When the model is under-parameterized (which may be the case for prompt learning, as the prompt parameters are relatively small compared to the CLIP parameters), simple reweighting is sufficient to fully explore the Pareto front. My question is why the proposed gradient manipulation is better. Can the authors provide theoretical analysis or empirical validation? [1] Revisiting Scalarization in Multi-Task Learning: A Theoretical Perspective (2) The backbone CLIP-ResNet-50 is a weak backbone. Please consider conducting some experiments on a CLIP-ViT based model. Minor: (1) Missing references. There have been some studies that use gradient alignment for prompt learning, e.g., [3,4]. Please consider adding them to the related works section. [3] Prompt-aligned Gradient for Prompt Tuning [4] Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models (2) Table 1 in the Introduction is not well explained. It does not specify which dataset and CLIP backbone are used, and the symbols $\rightarrow C, I, P$ are not explained. In addition, Table 2 should be referenced as Figure 1.

Questions

Please refer to the weaknesses, and I am willing to raise the score if the major weaknesses are addressed.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Limitations have been discussed and no negative societal impact has been identified.

Summary

This paper proposed a novel domain adaptation method that tunes the text prompts based on self-training with pseudo labels. The proposed method treats the source training and target training as multi-objective optimization problems, and it introduces to alignment of the gradients from both training (Prompt Gradient Alignment) so as not to cause conflict between the source and target training. The authors validated the proposed method empirically and theoretically.

Strengths

+ This paper is rich in technical novelty. + There are many methods in domain generalization research that aim to align gradients from multiple domains. However, requires heavy hessian calculations. The proposed method successfully avoids this issue by approximation with Taylor expansion. + The proposed method is empirically and theoretically justified.

Weaknesses

+ It is questionable whether the comparison with existing methods is fair. + The old domain adaptation methods fail to show their effectiveness when they are applied to the CLIP model. I suspect that those domain adaptation methods were not properly applied to CLIP. Is there any reason why they fail? + Conversely, would it not be possible to evaluate the proposed method in accordance with existing domain adaptation benchmarks? For example, would it be possible to evaluate the proposed method by updating the model parameters instead of the prompts? + There are some additional related works that should be discussed in the paper. + Bose, Shirsha, et al. "Stylip: Multi-scale style-conditioned prompt learning for clip-based domain generalization." Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2024. This work proposes a prompt training method for the domain generalization tasks. + Zhu, Beier, et al. "Prompt-aligned gradient for prompt tuning." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023. This introduces a gradient matching idea into prompt tuning, which has a similar mind to this paper. + Some of the formulas are a little difficult to follow. + It is very difficult to understand which of eq 9 or eq 11 is the actual loss function. This goes for eq 16 and eq 18 as well. I recommend separating the formulations for easier understanding.

Questions

+ As mentioned in the third bullet of weaknesses, I could not fully understand the formulations. Could you explain which is the actual loss function?

Rating

7

Confidence

3

Soundness

3

Presentation

2

Contribution

3

Limitations

+ I did not found any additional limitation for this work.

Summary

This paper proposes a novel approach called Prompt Gradient Alignment (PGA) for unsupervised domain adaptation (UDA). The key contributions are: (1) Formulating UDA as a multi-objective optimization problem with objectives for source and target domains; (2) Aligning gradients between objectives to encourage consensus; (3) Penalizing gradient norms to improve generalization; (4) Leveraging pre-trained vision-language models through prompt learning. The method is evaluated on standard UDA benchmarks and demonstrates consistent improvements over other prompt-based baselines. A theoretical generalization bound is also provided to justify the approach.

Strengths

The strong empirical results across multiple UDA benchmarks are particularly impressive. The consistent outperformance of other prompt-based methods on datasets like ImageCLEF, Office-Home, and DomainNet (as shown in Tables 3-5) provides robust evidence for the method's effectiveness. The performance gains are substantial in many cases, with improvements of up to 4% on average accuracy compared to state-of-the-art methods. The theoretical analysis offering a generalization bound (Section 4.5) adds credibility to the approach, providing insights into why the method works and under what conditions it can be expected to perform well. This combination of empirical success and theoretical grounding is a significant strength of the paper. The ablation studies (Section 5.4, Table 6) effectively demonstrate the contribution of each component of the proposed method, providing a clear understanding of how different elements (like gradient alignment and norm penalization) contribute to the overall performance. The visualization of gradient similarity evolution (Figure 7) offers additional insights into the method's behavior during training.

Weaknesses

The computational complexity and training time comparisons to baseline methods are notably absent. Without this information, it's difficult to assess the practical trade-offs of implementing PGA compared to existing methods. This is particularly important given the method's use of gradient manipulation, which could potentially increase computational requirements. The paper's heavy reliance on pre-trained CLIP models raises questions about the method's applicability in scenarios where such pre-training is not available or suitable. While the use of pre-trained models is a strength in many cases, it could also be a limitation in domains significantly different from CLIP's training data. The discussion of hyperparameter sensitivity is limited. Given the importance of hyperparameters like ρga and ρgn in the gradient alignment and norm penalization processes, a more thorough exploration of their impact on performance would be valuable. While the method shows improvements over existing approaches, the degree of novelty is somewhat incremental. The core ideas build heavily on existing prompt-based and gradient manipulation techniques, which may limit the paper's impact.

Questions

1. How does PGA's computational complexity compare to baseline methods? Are there significant differences in training time? 2. Have you explored performance on data types beyond image classification? How generalizable do you expect the method to be? 3. How sensitive is the method to hyperparameter choices, particularly for gradient alignment and norm penalization? 4. Given the reliance on CLIP pre-training, how well do you expect the method to perform in domains very different from CLIP's training data? 5. How does the method's performance change as the degree of domain shift varies? Is there a point where simpler approaches become competitive?

Rating

4

Confidence

3

Soundness

3

Presentation

2

Contribution

2

Limitations

The reliance on CLIP pre-training potentially limits applicability to domains very different from CLIP's training data. It's unclear how well the method would perform on specialized domains (e.g., medical imaging, satellite imagery) that are not well-represented in CLIP's training set. The paper does not thoroughly explore potential failure cases or scenarios where the method might struggle. For instance, how does the method perform when there are extreme label distribution shifts between source and target domains? Or when the domain shift is particularly severe? The scalability of the method to very large datasets or high-dimensional data is not addressed. It's unclear how the computational requirements scale with dataset size or feature dimensionality.

Summary

To enhance both transferability and discriminability for prompt learning based domain adaptation, this paper proposes a Prompt Gradient Alignment (PGA) method. PGA encompasses multiple domain-wise classification objectives, cosine similarity maximization regularizers between prompt gradients of different domains, and prompt gradient norm penalty for each classification objective. It can thus achieve both inter-domain gradient alignment and flat minima enforcement. To efficiently and effectively update the designed prompts, a practical gradient update procedure is devised and works under both single-source and multi-source UDA. Empirical results shown the superiority of the proposed method.

Strengths

1. A novel method with an information-theoretic explanation. Aligning prompt gradients across domains moves the shared prompt towards the low-loss region of both domains, such that domain-invariant class-discriminative features can be captured, thus benifiting both domains. 2. A practical, flexible, and efficient gradient update procedure. It has a weighting term on the source signal to control how much emphasis should be put on the target domain and avoids the costly Hessian computation. 3. A well-written article. The paper has a smooth logic and is easy to understand.

Weaknesses

1. The abstract should reflect the technical highlights. For example, UDA is essentially a multiple-objective optimization problem, which is not the technical novelty of this paper. 2. As indicated by Theorem 4.1, the proposed method minimizing source empirical loss, gradient norms, and gradient misalignment, model and reduce the upper bound of the established generalization error. Nevertheless, it is not intuitive why minimizing gradient norms and gradient misalignment are beneficial for the performance on the target domain. 3. In Theorem 4.1, g_t^src and g_t^T are the gradients w.r.t. P_t of source loss and target loss respectively. The two gradients should be derived from the same prompt or different prompts? Can the gradient alignment theory be extended to other spaces like image, feature, and output spaces? 4. Due to use of pseudo labels for the target domain, though they are improved by confidence filtering, the accuracy of pseudo labels may greatly affect the success of inter-domain gradient alignment. It is encouraged to investigate the relationship between them. 5. Some important related works are missing in the discussion of related works, e.g., [a]. [a] Tang et al. Unsupervised domain adaptation via distilled discriminative clustering. PR, 2022. 6. Validation on larger datasets like S2RDA [b] is important for three reasons. First, it is a more large-scale, realistic, challenging multi-domain benchmark dataset. Second, it is tailored for synthetic-to-real transfer, more in line with industrial needs. Third, this paper leverages the powerful generalization capability of large-scale pre-trained vision-language models and fine-tuning with a large amount of training data can avoid the potential risk of overfitting. [b] Tang and Jia. A New Benchmark: On the Utility of Synthetic Data with Blender for Bare Supervised Learning and Downstream Domain Adaptation. CVPR, 2023. 7. What circumstances does the overfitting issue appear? Why does penalizing gradient norms alleviate the overfitting issue? Are there any theoretical reasons? 8. In Eq. (6), L_tgt should be L_T? In Eq. (20), L_s should be L_s^PGA? The authors may re-write the final overall objective for clarity.

Questions

See Weaknesses.

Rating

8

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

N/A