Understanding Contrastive Learning via Distributionally Robust Optimization

Summary

This paper tackles the sampling bias problem in contrastive learning, where negative samples are usually sampled randomly from the marginal distribution and may contain false positive samples. The authors provide a connection between the contrastive learning objective and distributionally robust optimization as a first part of the contributions: By an optimal choice of the temperature parameter, the InfoNCE loss coincides with maximizing the loss for negative samples, leading to the worst-case negative sample distribution. Consequently, it is revealed that the temperature parameter admits a trade-off between the generalization error and the DRO radius. Secondly, the authors point out the two issues of standard contrastive learning: It tends to overweigh negative samples with large distances (called over-conservatism) and be sensitive to outliers. To mitigate these issues, the authors propose an alternative weight distribution over negative pairs to prevent overweighting of too-far negative samples.

Strengths

- New insights on contrastive learning: One of the main contributions of this paper, the equivalent reformulation of contrastive learning with distributionally robust optimization, provides a new perspective to contrastive learning. In this picture, contrastive learning can be interpreted as optimizing the worst-case distribution over the negative samples. This interpretation can benefit us in developing a new method, as seen in Section 5. - An new interpretation of the temperature parameter: As a result of the reformulation, we observe that the temperature parameter in contrastive learning serves as the Lagrange multiplier for the DRO constraint, and lower temperature leads to stronger DRO constraint. This insight again contributes to the development of algorithms.

Weaknesses

- Meaning of the connection between contrastive learning and DRO is not clear enough: Theorem 3.2 connects the DRO objective and the InfoNCE loss. Although it is very interesting and insightful, and the authors argue that contrastive learning can be regarded as DRO and thus mitigate the sampling bias, we could argue that DRO is merely contrastive learning with the sampling bias in the opposite way. This would undermine the robustness property of DRO can still suffer from the sampling bias residing in contrastive learning. - DRO interpretation may not explain hard mining: In Section 3.3.C and Section 5.1, the authors observe that the DRO equivalent formulation explains how contrastive learning puts larger weights to negative samples with larger $f\_\\theta$. Although this seems correct, this phenomenon does not correspond to what is so-called "hard mining." Indeed, Robinson et al. [38] (and many other works) define hard negative samples as "points that are difficult to distinguish from an anchor point," which can be put in the current context as "negative samples with small $f\_\\theta$." This observation may explain why a "strange" trade-off is observed in Section 3.3.A such that stronger DRO constraint leads to looser generalization error, unlike what we expect.

Questions

Major comments are listed first. - In Corollary 3.4 and Theorem 3.5, it is better to explain exactly what the "approximation" indicates. - With the proposed method ADNCE, can you discuss how we should choose the parameters $\\mu$ and $\\sigma$ in Section 5.2? Minor comments follow. - Typo in l.42: "we proof" -> "we prove" - Typo in l.82: "Donsker-Varadah" -> "Donsker-Varadhan" - Typo in footnote 1: The expectation looks strange. - Typo in l.142: $\\mathbb{E}\_Q$ -> $\\mathbb{E}\_{Q\_0}$ - In Theorem 3.2, the definition of $\\mathbb{Q}$ lacks. In Eq. (4), can $\\mathbb{E}\_{Q\_0}[Q/Q\_0]$ be simply written as $\\int Qdx$? - Typo in l.156: "satisfied" -> "satisfies" - Typo in l.173: "theorem 3.2" -> "Theorem 3.2" - Typo in l.177: "subsection 3.4" -> "Subsection 3.4" - Typo in l.190 and l.191: $E\_{Q\_0}$ -> $\\mathbb{E}\_{Q\_0}$ - In Theorem 4.2, it is better to mention $\\phi^\*$ indicates the convex conjugate because this is the first place where it appears. - In Corollary 4.3, explaining what two random variables $X$ and $Y$ refer to make the statement complete is better. - In l.251, what do you compare CL-DRO with by the word "tighter"? By the way, the word "tight" indicates that equality can be attained for a given inequality, so the comparative "tighter" seems strange. - In l.257, I don't understand how "DRO bridges the gap" in the following paragraph. - In l.259, it is better to explain what $\\mathcal{I}\_{DV}$ indicates.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

The authors have adequately discussed the limitations in the conclusion. Societal impacts do not apply to this work because this work is mostly theoretical.

Summary

This paper starts from the question why the naive form of CL is robust to sampling bias issue, resulting in empirical success in various areas. T this end, the authors first present the relationship between CL and DRO theoretically, where the DRO-constrained CL objective is conceptually equivalent to the objective of CL itself. They further show that the temperature in CL acts as a (inverse of ) robust radius in DRO constraint. Based on these findings, the authors finally propose ADNCE, where the importance weight of negative samples follow Gaussian distribution. The proposed methods are validated via experiments on various domains.

Strengths

- The paper reveals the relationship between CL and DRO in a more comprehensive way to address the bias-tolerant behavior of CL. - Theoretical findings regarding \tau are validated empirically. - The proposed framework and the ablative model based on the approximation of CL-DRO (Eq. 8) achieve meaningful performance improvement on the datasets.

Weaknesses

- The design choice of ADNCE needs to be justified. Why such Gaussian-like weights are the most reasonable choice and can address the weakness mentioned in Sec. 5.1? - Missing sensitivity analysis of \mu and \sigma in (12), comparing to those of \tau in InfoNCE. How much the results change as the hyperparameters vary?

Questions

- Why controlling the variance of negative samples contributes to the success of CL?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Limitation has been addressed.

Summary

The paper proposes a novel theoretical framework for understanding contrastive learning (CL) via distributionally robust optimization (DRO). Under this framework, the paper derives that the InfoNCE loss can be interpreted as DRO with KL ball around the negative sampling distribution, and the paper leverages DRO insights to derive the optimal temperature parameter. Experiments are conducted which show that a novel modification of InfoNCE can lead to better sample complexities.

Strengths

1. The framework that the paper proposes is quite elegant mathematically and explains the InfoNCE loss from a rigorous mathematical perspective, which endows us with many insights. 2. Using these insights, the paper proposes a new ADNCE loss that overcomes the issues of conservatism in DRO and shows some possibly promising experiments. (I am not an expert in CL so I cannot gauge how convincing these empirical improvements are).

Weaknesses

1. The proposed approach introduces more hyperparameters, which define the weighting distribution. These hyperparameters can be a hassle to tune in practice, and it is unclear if the reported gains are simply from better hyperparameter tuning with these newly introduced hyperparameters.. It would be informative to show whether the new ADNCE is better for a large number of hyperparameters, or only a few. Please also answer my questions.

Questions

1. In Corollary 3.4, how is optimality measured? In other words, what objective does the setting of Eqn 6 optimize, and which assumptions are needed for it to hold? 2. Besides the math, is there any intuition on why DRO in the negative samples is what CL should strive for? In other words, why do we want to maximize the distance for all possible distributions of negative samples around the truth? (rather than the one we see data from). I think illustrating this intuition would be important to making this paper stronger. 2b. Does this DRO insight applies to other types of CL losses, besides InfoNCE? 3. What is Q^ideal? It is undefined before Line 154. 4. Can you please shed more light on what is hard-mining and how DRO gives us insights that hasn't been uncovered before? The paper seems to assume that readers know what hard-mining is, but I am not aware of this phenomenon for CL. 5. Can ADNCE also be explained with some modification of DRO? It would be nice to see if the new method is theoretically motivated as well, or simply a heuristic to avoid conservatism of DRO.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

See weaknesses/questions.

Summary

In this work the authors demonstrate a connection between contrastive learning, in particular InfoNCE, and distributionally robust optimization. In contrastive learning algorithms it is typical during training that samples which are similar are treated as being different, ie a negative pair, since contrastive learning typically does not include class labels. Intuitively this would be a hinder the ability of contrastive learning algorithms to learn useful representations. Interestingly this doesn't seem to happen in practice, choosing a temperature parameter correctly can cause standard CL algorithms to perform as good as, or better than, robust variants, in the presence of false negative pairs. In this work it is theoretically proven that InfoNCE is naturally distributionally robust, which explains the phenomenon from the last paragraph. This is done by proposing a distributionally robust version of InfoNCE and demonstrating that the loss is equivalent to InfoNCE with scaling and shifting. This work goes on to analyze the effect and selection of the temperature parameter, giving further insight into the behavior of InfoNCE. It is demonstrated that the distributionally robust InfoNCE is an estiamtor of mutual information. This work then proposes ADNCE as a way to overcome shortcoings of InfoNCE, in particular its sensitivity to outliers, i.e. the hardest negative pairs. This is done by weighting samples in the loss so that the most outlying negative pairs have lower weight in the ending risk. It is experimentally demonstrated that this method brings improvements.

Strengths

- The theoretical result of this work is useful, reasonably nontrivial, and regards a topic that is of very large interest to the ML community. Very good! - ADNCE is an easily algorithm for a problem of signifiant interest, that brings consistent improvements. Also very good!

Weaknesses

I spent a fair amount of time with this paper and I think its contributions are of high significance and importance, however I really feel that the exposition needs some improvement. I spent some time with the proofs and, while I was always able to decipher what was happening and found no errors, but they are written in a way that I found very hard to parse. While its far from the worst writing I have seen, there is quite a lot of the main text that was also difficult to precisely understand. I don't think the topicality here is so mathematically dense or advanced that this couldn't be written in a much clearer way. If this paper were very well written I would give this an 8, I think. Some example issues are listed below: - Line 63: What is "the ultimate distribution"? - Line 101 & Proof of Theorem 3.2 : it would be much clearer if $f_\theta$ was instead $f_\theta(x,y)$. This wouldn't take more space and it would be much easier to parse whats going in in the Proof of Theorem 3.2 if this were the case. - Theorem 3.2: It should be made clear that $\alpha^*$ is a function of $\eta$, this seems quite important. One could simply write $\alpha^*(\eta)$. - appx. line 10: I think $P$ in this line is supposed to be $Q$ - appx. (17): I think its worth writing the original constrained optimization before applying strong duality. - appx. (17): $\max_L$ is again a bit vague. I'm assuming whats actually happening is that we are optimizing over $Q$ in the $L$ definition. - appx. line 21: "fine definition" is nonstandard, something like "always exists" or "is well-defined" - appx. line 29: this should be an $\arg \min$. Text like "$\alpha^*$ represents the optimal value of ..." is needlessly confusing, just say "$\alpha^* = \arg \min \cdots$." - Line 148, 165: Maybe I'm missing something, but I don't quite get what "optimal" means in these lines. - Theorem 3.3: What is $Q^{ideal}$? In what sense is it "ideal"? If this work used more clear and precise notation I think it would be great, right now it looks like a rough draft. As an example improvement, I would rewrite in appx (17) the fist term in the last line $\mathbb{E}_{P_0} [f_\theta]$ as $\mathbb{E}_{(X,Y)\sim P_0}[ f_\theta(X,Y)]$ Some other errors: - "and graph" should be "and graphs" - Give the definitions of acronyms, eg DCL HCL RPL - There are many missing periods for sentences that end with equations, eg at the end of (1), (16),(17) - Line 70: "downstrea" [sic] - Page 3 footnote: sloppy ]

Questions

I don't have any questions really, the paper just needs more polish

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

4 excellent

Limitations

None

Response

Thank you for the clarifications. I specifically appreciate the clarification of M2: "a larger $f\_\theta$ value indicates a higher degree of similarity," which I misunderstood at the initial review phase. Given this, I increase the evaluation score from 4 to 5.