Understanding Bias in Large-Scale Visual Datasets

Rebuttal

We thank you again for your valuable feedback and we hope our response can address your questions. If you have any further questions or concerns, we are very happy to answer. [1] Thomee et al, YFCC100M: The New Data in Multimedia Research\ [2] Gadre et al, DataComp: In search of the next generation of multimodal datasets\ [3] Changpinyo et al, Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts\ [4] Li et al, Exploring Plain Vision Transformer Backbones for Object Detection\ [5] Kirillov et al, Segment Anything\ [6] Podell et al, SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis\ [7] Anthropic, Claude 3.5-Sonnet\ [8] Meta, Llama 3.1

Reply

My concerns are mostly addressed, it would be interesting to see how these suggestions could be incorporated to build a new dataset curation method in future work. I increase my score to 6 after reading the rebuttal.

Rebuttal

We thank you again for your valuable feedback and we hope our response can address your questions. If you have any further questions or concerns, we are very happy to answer. [1] He et al, Deep Residual Learning for Image Recognition\ [2] Liu et al, A ConvNet for the 2020s\ [3] Song et al, MPNet: Masked and Permuted Pre-training for Language Understanding\ [4] Reimers & Gurevych, Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks\ [5] Thomee et al, YFCC100M: The New Data in Multimedia Research\ [6] Gadre et al, DataComp: In search of the next generation of multimodal datasets\ [7] Changpinyo et al, Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts\ [8] Liu et al, A Decade’s Battle on Dataset Bias: Are We There Yet?

This is some impressive response.

I greatly appreciate the authors effort in putting up the response. In fact, it's beyond my expectation. I find my 3 major concerns have been adequately addressed thanks to the additional information - I trust the authors will incorporate them into the final version. In retrospect, I believe what hindered my initial impression was the title of the paper, which led me to believe this was some incremental work over investigating *'intra-dataset biases'*, such as imbalanced distribution of object classes or visual features and etc. I highly recommend, if applicable, that the authors explicitly open up with **'INTER-dataset biases'** in the title or somewhere early-on in the revised paper. Overall, I am willing to update my rating owning to this thorough rebuttal.

Thank you

Thank you again for your helpful comments and for reviewing our response! We are glad to hear that the concerns have been addressed. We will incorporate the additional results discussed in our rebuttal into the next version of the paper and emphasize that we're studying the inter-dataset bias in our introduction to better reflect the focus of our work.

Rebuttal

We thank you again for your valuable feedback and we hope our response can address your questions. If you have any further questions or concerns, we are very happy to answer. [1] Zhang et al, Understanding deep learning requires rethinking generalization\ [2] Song et al, MPNet: Masked and Permuted Pre-training for Language Understanding\ [3] Reimers & Gurevych, Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks\ [4] Udandarao et al, No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance \ [5] Dunlap et al, Describing Differences in Image Sets with Natural Language\ [6] Thomee et al, YFCC100M: The New Data in Multimedia Research\ [7] Gadre et al, DataComp: In search of the next generation of multimodal datasets\ [8] Changpinyo et al, Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts

Response to rebuttal

I thank the authors for their added experimental results and comments. (1) The random labels experiment sufficiently answers my question regarding the specificity of these dataset-level biases. (2) Thanks for running the sentence embeddings and segmentation mask experiments, those high stable accuracies clarify my concern. (3) The cross-validation results are very strong. I would encourage the authors to somehow incorporate these results into the main paper perhaps as a mean +/- std or something similar, I think this would significantly increase the confidence in the paper's findings. (4) Thanks for the clarifying discussion b/w your work and the No Zero-shot paper. You being able to reproduce the findings of that paper are very interesting. (5) Great that the VisDiff results corroborate your analysis. (6) Your response to w7 is very insightful, thank you for adding this. I would encourage to expand on this discussion and if possible add some sample images from each of the datasets to that section in the appendix. Would be super useful! Overall, the author rebuttal has significantly increased my confidence in the paper and the main results and insights provided in the paper are very useful. I am updating my score to an 8.

Thank you

Thank you for your time and feedback on the paper! We’re glad that your concerns have been resolved. We will make sure to include the additional results from the rebuttal in the next version of our paper. The review has been very helpful in enhancing our paper.

Reply to "build more diverse and representative datasets in the future"

Thank you for your clarifications. Indeed, our previous response focused more on past efforts. **Here we provide several ways our framework can be used to help build more diverse and representative datasets in the future:** - When considering adding a new set of images (e.g., from another website) into a data collection, we can first treat new images as a separate dataset, use the transformation and classification framework to tell where and how much they differ from the existing image collections. This can help us decide whether to join them. If the goal is to enhance diversity of the dataset in a certain aspect (e.g., object types), we should only join them when they are sufficiently different. - Our language-based analysis provides textual descriptions for any new dataset. This text description directly gives the data curators the intuition on the gist of the dataset, especially when compared with reference datasets. It can help curators refine text terms for image search and tag filtering, in search engines or other platforms. - Our framework can identify bias and distribution imbalance in the image statistics (e.g., colors and object distribution). This can help guide adding/removing images with desired/undesired statistics for more balance. - The result of a dataset classifier trained on the transformed datasets can serve as a measure of the image's "typicality" within the dataset. For example, images in dataset A that are misclassified as images in dataset B are considered "not typical" within A, for that transformation/attribute. If needed, images with less typical attributes can be then oversampled in collecting data and/or training to enhance representation. **In addition to how to use the framework, we list a few direct lessons we learned from our analysis on YCD, that can also help build more diverse and representative datasets in the future:** - Filtering by embedding similarity to images of a reference dataset could inherit bias of that dataset. We observe that DataComp has the lowest number of unique objects per image (Fig 12). This potentially resulted from DataComp filtering for images with high embedding similarity to ImageNet training examples, most of which feature object-centric images [1]. To mitigate this, dataset curators should be mindful of the inherent biases in their reference datasets (e.g., ImageNet). Concretely, to enhance the object diversity within DataComp, one could consider using a reference dataset with a higher per-image object diversity (e.g., COCO) during filtering. - The source website's image collection mechanism can introduce bias. We also noted that YFCC is heavily skewed towards outdoor scenes and human interactions (Sec 4.2). This bias likely stems from its reliance on a single data source, Flickr, where user-generated content often focuses on personal photos, landscapes, and social interactions. Dataset curators should recognize that the collection methods (e.g., user uploads) of data sources (e.g., Flickr) can introduce biases into the resulting dataset (e.g., YFCC). - Web-scraped images would naturally contain more digital graphics. Since CC and DataComp are crawled from the Internet, they feature results from search engines [2, 3]. This prioritizes professionally created content like advertisements, infographics, and digital media. Curators should evaluate whether this composition aligns with their downstream goals. Thank you for your quick reply/clarification. We will add a discussion on this to the paper and see it as an important improvement. We also hear your suggestion on moving experiments to the appendix for the space. Given that NeurIPS policy allows an additional page for accepted publications, we would be able to add the discussion while maintaining the current experiments. We are happy to address any further concerns. References:\ [1] Barbu et al, ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models\ [2] Changpinyo et al, Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts\ [3] Gadre et al, DataComp: In search of the next generation of multimodal datasets