Summary
The authors propose a method for debiasing the representations of Vision-Language Models (VLMs) that can be applied at various layers of the image and text encoders/decoders, and can be used for a variety of downstream tasks such as image generation, 0-shot classification, text to image retrieval, and image captioning.
Strengths
- This work's goal of debiasing VLMs is an important and timely problem that should be of interest to many in the NeurIPS community
- The proposed approach is seemingly effective, lightweight and seems easy to apply
- The method is finetuning-free, which mitigates issues such as catastrophic forgetting
Weaknesses
**Minor issues:**
- The writing could be improved in parts. There are minor grammatical errors in parts, such as "For instance, Hirota et al. [18], Zhao et al. [42] investigated bias in image captioning, where specific genders or races are disproportionately represented **leading to generate** biased caption." I suggest another editing pass or two.
- I find the structure of section 3 "Bias Analysis in VLMs" is a little odd. It mostly reads like background/preliminary material, but it also contains some experimental setup details (such as in line 165). This seems off, as this is before the proposed method section. I suggest moving these experimental details to section 5, with the rest of the experimental setup.
- Figure 4 shows that DeAR fails to produce bias-free representations. Including a figure likes this naturally makes the reader want to see a figure showing that the proposed approach **does** succeed here. It feels incomplete as-is.
**More significant issues:**
- The experimental results don't include confidence intervals. The authors justify this by pointing out that the proposed approach is deterministic. However, CIs still can and should be calculated by an approach such as bootstrapping the test set.
- Relatedly, the text to image generation experiment is **not** deterministic if the text2img generator's random seed is changed, so CIs should definitely be reported here.
- The compared methods are limited to only 2 other debiasing approaches and the baseline model.
- The proposed approach of 1) finding features most associated with the biased attributes and 2) replacing these features with an imputed value from low-confidence samples is, I think, very similar in practice to Chuang et al.'s approach [1] in "Debiasing Vision-Language Models via Biased Prompts". Chuang modifies embeddings by projecting them onto the subspace defined by the linear direction most associated with the biased attribute(s). Intuitively, the direction most associated with the biased attribute should be similar to the direction defined by the features identified in step 1) of the proposed approach. Not that I am **not** saying that this similarity alone is a limitation; I think the proposed approach is different enough to still have sufficient novelty. However, I think that Chuang's approach really needs to be compared against (for all experiments except maybe image captioning), in order to verify that these features are either finding a different direction in the embedding space or that the low confidence imputation performs better than just projecting onto the bias subspace.
[1] Chuang, Ching-Yao, et al. "Debiasing vision-language models via biased prompts." arXiv preprint arXiv:2302.00070 (2023).
Questions
- Is the improvement of the proposed approach over the compared methods significant when CI are considered, where these CI's are obtained from bootstrapping or some other way?
- How does the proposed approach perform when compared against the method proposed in "Debiasing vision-language models via biased prompts."?
Limitations
I think the discussion on limitations could be improved. The authors point to Section ***5.2 Result Analysis** as their discussion of limitations. 5.2 covers the results of their experiments, and does include some discussion on how the experimental results on image generation point to room for improvement. However, I feel like a discussion of the limitations would be greatly improved by dedicating a separate section/subsection to it. I would expect the authors to discuss more fundamental limitations, rather than just experimental performance. For instance, I would include details on how the method assumes access to a validation set with labels for bias/protected attributes (e.g. a dataset with race/gender/etc labels).