Generalized Logit Adjustment: Calibrating Fine-tuned Models by Removing Label Bias in Foundation Models

Summary

This paper addresses biases in foundation models, such as CLIP, which due to the imbalanced training datasets, are skewed towards frequent semantics. The authors propose a Generalized Logit Adjustment (GLA) method, an optimization-based approach for debiasing these models. Despite inherent challenges, the GLA method demonstrates significant improvements across multiple tasks and datasets, achieving 1.5 pp accuracy gains on ImageNet and large average improvement (1.4-4.6 pp) on 11 few-shot datasets. Notably, the GLA method does not require access to the pre-training dataset, making it practical for fine-tuning scenarios. The paper offers both theoretical justification and extensive empirical evidence for the proposed method.

Strengths

1. **Solid Theoretical Analysis**: The paper provides robust theoretical justification for the Generalized Logit Adjustment (GLA) method. It formalizes the estimation of label bias as a constrained optimization problem, proving that GLA is a Bayes optimal classifier. 2. **Practical Solution**: The proposed method doesn't require access to the pre-training dataset, which makes it practical for fine-tuning scenarios. This approach is useful given privacy and copyright concerns associated with accessing the label distribution of pre-training datasets. 3. **Comprehensive Evaluation**: The paper uses a comprehensive benchmark for evaluation, considering three real-world settings and three fine-tuning paradigms. This diverse approach to evaluation adds credibility to the findings. 4. **Significant Performance Improvement**: The paper reports notable improvements across various tasks, demonstrating the efficacy of the proposed GLA method. The substantial gains across multiple datasets reinforce the value of the method.

Weaknesses

1. **Limited Validation**: The paper only presents results with a limited set of tasks and models. While they used the CLIP model for the zero-shot setup, the validity and effectiveness of the GLA framework with other models (like BERT, GPT, etc.) have not been validated. 2. **Estimating Pre-training Bias**: The estimation of pre-training label bias is done with downstream data. However, this could be problematic if the downstream data distribution significantly differs from the pre-training data distribution, leading to incorrect bias estimation and sub-optimal performance.

Questions

The only two questions are listed in weakness.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

While it's a promising beginning to uncover the biases in foundational models, it's crucial to recognize that the exploration should not be confined to computer vision alone. It is necessary to extend the studies to other fields as well.

Summary

This paper studies the label bias in Foundation Models, like CLIP. Specifically, authors model the estimation of the label bias as a constrained optimization problem, and propose a Generalized Logit Adjustment (GLA) method to debias foundation models. Experimental results demonstrate the effectiveness of proposed method.

Strengths

- Show that the skewness of pre-training distribution affects the performance of downstream tasks, i.e., label bias of foundation models. - Propose a GLA method to ensemble the debiased zero-shot and fine-tuned models. - Extensive experimental results demostrate the superiority of proposed method over conventional fine-tuning.

Weaknesses

- Authors should discuss the difference with existing label bias estimation methods, e.g. [1,2]. - [3] also studied the label bias in CLIP. What's the difference? - There exist some strong assumption for proposed method. [1] Lipton Z, Wang Y X, Smola A. Detecting and correcting for label shift with black box predictors. In ICML, 2018: 3122-3130. [2] Garg S, Wu Y, Balakrishnan S, et al. A unified view of label shift estimation. In NeurIPS, 2020, 33: 3290-3300. [3] Wang X, Wu Z, Lian L, et al. Debiased learning from naturally imbalanced pseudo-labels. In CVPR, 2022: 14647-14657.

Questions

- The Proposition 1 shows that $f_{gla}$ is the best model, or nearly optimal Bayesian classifer. Thus what leads to the error between the realistic results and the ideal performance of optimal Bayesian classifer. - What's the source of the validation data for a specific problem. In all experiments, the compared fine-tuning methods whether or not to use the validation data. - Please illustrate that ''Intuitively, since the zero-shot and fine-tuned models provide diverse predictions, conditioned on the two predictions is equivalent to adding the logits in log space".

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The proposed method can effectively deal with label bias in foundation models. However, it depends on some strong assumptions. E.g., the estimation relies on the validation data. How to guarantee that a good estimation can be obtained with poor validation data.

Summary

This paper proposes a distribution estimation method that estimates the distribution of pre-training data for large VLMs like CLIP. Combined with downstream data distributions, a logit adjustment is applied to the outputs of both the zero-shot model and the fine-tuned model. By using a simple ensemble of the zero-shot and fine-tuned models, improved performance is achieved on the reported benchmarks.

Strengths

The proposed method introduced performance improvements over compared baselines on most of the reported benchmarks.

Weaknesses

Here are some questions: 1) The claim "their predictions are complementary to those of fine-tuned models" in line 30 cannot be supported by the cited references. Neither [3] nor [14] can support fine-tuned and zero-shot models are complementary. 2) In addition, in Section 4.2, where the authors attempt to demonstrate the complementarity of fine-tuned models and zero-shot models, the reasoning provided is not persuasive. The authors argue that fine-tuned models are biased towards downstream distributions, while zero-shot models are robust to distribution shifts. However, it remains unclear why a distribution shift-robust model should be complementary to a model that has already been adapted to a shifted distribution. If the authors claim and can prove that zero-shot models can provide something that is missing in the fine-tuned model and cannot be provided by the shifted downstream dataset, it needs to be explicitly demonstrated and proven. Otherwise, this assumption lacks solidity, which consequently undermines the validity of all subsequent proofs. 3) The rationale behind the method used to estimate the pre-training data distribution is not clearly explained. The paper does not provide sufficient information on why the method is designed in this particular way, nor does it address whether there are any means to verify the effectiveness of this method in accurately estimating the pre-training data distribution. Even though pre-training data is often proprietary, the authors could still perform toy experiments to demonstrate the efficacy of the proposed method. Given that this is a crucial component of the proposed method, it should not be overlooked. 4) The biggest weakness of this paper is the improvements introduced by the proposed method is very limited (~1% on average across all reported benchmarks). Edit: most of my concerns are clarified. I will change my rating.

Questions

Same as above.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

1 poor

Limitations

The biggest limitation is the limited performance improvements.

Summary

Logit adjustment method has been widely used in long-tailed recognition, in which the source dataset is available, thus the class margin can be set in advance. However, the large-scaled foundational data cannot be accessed. To reduce the recognition bias of foundational models on downstream tasks, this paper proposes a generalized logit adjustment (GLA) method. The main contribution of GLA is to estimate the class margins for zero-shot models, after that, the fine-tuned and debiased zero-shot models are assembled together to achieve consistent improvement. The authors demonstrate the effectiveness of the proposed method through extensive experiments, including three tasks and three finetuning approaches.

Strengths

1) Estimate the class margins for foundational models is an important task, especially when the pretrained datasets are not available. This paper proposes an effective generalized logit adjustment method for the zero-shot models. 2) To estimate the class margins, the authors consider it as a simple constrained optimization problem. They also provide theoretical analysis for their GLA method. The proofs seem to be ok. 3) The authors verify their method with extensive experimental results, including many-shot learning, few shot learning and long-tailed recognition tasks. The presented results deem to be good.

Weaknesses

The main weaknesses include the following three aspects. 1) The proposed method is simple, and the theoretical proofs look a little reluctant. a. Section 4.1 sees to be redundant. The Definitions and Lemma are less important for the following analysis. b. The proof and conclusion of Proposition1 are not convincing. c. There are some false assumptions about the discussion of Corollary 1. In case 1, the authors claim that the zero-shot model can not provide further improvement when the fine-tuned model is given. Obversely, this is wrong. Because their combination can further improve recognition accuracy. 2) The authors did not show the algorithm steps when estimating class margins (Eq. 3). They also did not provide the complexity and convergence of this optimization problem, and the choice of hyper-parameter \lambda. 3) Unfair comparisons make results look good. For example, in long-tail learning, the proposed GLA is an ensemble model with larger network parameters. Thus, it is not surprising that it outperforms other baselines. Please provide the model parameters and inference time to show the improvement. Other comments: 1) Typo in line 109: w_k is not model parameter. 2) In Eq.4, v should be bold.

Questions

1) How to initialize q? 2) Your assumption in Lemma 2 does not always hold, the target distribution may be unbalanced. 3) Please provide more analysis for your proofs. The constant C_2 in Eq. 21 and the conclusion of Eq. 30 are not easy to understand.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors discuss the limitations of their method.

Summary

This paper proposes a method for combining the predictions of zero-shot and few-shot classifiers. The method removes the need for a weighting hyper-parameter to interpolate between the predictions. Furthermore, it also removes a source of bias (due to the frequency of words in the pre-training dataset).

Strengths

## Simplicity The proposed method is both simple to understand and implement. This means that it could have a significant impact. Simple methods can easily be deployed by both practitioners and by researchers (as a strong baseline for future work). ## Presentation The presentation of the method is clear, concise, and easy to understand. ## Experimental Results The experimental results show that the proposed method improves over a selection of sensible baselines in a range of settings, with some strong improvements in several cases.

Weaknesses

## Correctness While the proposed method clearly results in improved performance in several settings, it is not clear to me that the reason for the improvement suggested by the authors is correct. That is, I am not convinced that the proposed method to estimate the label bias of the pre-training dataset is working as intended. Does $\log \mathbf{q}$, αs described in eq 3 and sec 4.3, provide a good estimate of $\pi_p$? I do not believe that the paper provides theoretical or empirical evidence that this is the case and that the mechanism behind the success of the method is due to the suggested bias improvement, rather than something else. ############## ### Edit, after rebuttal and discussion. The authors have now addressed my concerns above. They have demonstrated that their approximation is accurate when the downstream and pre-training datasets are similar. They have also shown that when the downstream and pre-training datasets are different, the approximation becomes worse. Nonetheless, the proposed method still works well, and the authors have given intuition for why this might be the case. Given the authors proposed updates to the paper, I believe that the story of the paper will now be accurate and clear. I've raised my score from 5 to 7.

Questions

Additional experimental and theoretical analysis showing that $\log \mathbf{q} \approx \pi_p$ would significantly strengthen the paper. Concretely, if this weakness were addressed, I would happily increase my score by 1-2 points. The true value of $\pi_p$ could, for example, be estimated by using the LAION-400m dataset, which is similar in scale and content to the true pre-training dataset of CLIP and results in similar zero-shot accuracies. This was the approach taken in the "A Simple Zero-shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image Models" paper (https://arxiv.org/abs/2302.06235), which addresses similar biases in the context of prompt selection in zero-shot classifiers. Additionally, including the above-mentioned paper in the related work would be useful.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The theoretical and experimental analysis does not support the main claim of the paper, as discussed above under "weaknesses".

The concerns are not perfectly clarified

Thanks for your detailed response, and most of them are not perfectly clarified. Firstly, the assumption in which test distribution is class-balanced in A1 seems to be impractical. The more reasonable setting is that the test prior is unknown. Secondly, DebiasedPL is general and works in more scene, e.g., semi-supervised learning. Though proposed method has theoretical Bayes optimal classifier, the strong assumption and limited improvement weaken the contribution. Thirdly, though no validation set are used to estimation $\pi_p$, the used balanced training data can not ensure the solution is optimal. E.g., the noisy label or imbalanced test distribution may deteriorate the performance. I tend to decrease my score to 4. Reviewer egNQ