Summary
This paper introduces a novel formulation of Optimal Transport (OT), named Curriculum and Structure-Aware Optimal Transport, for generating pseudo labels by considering both inter- and intra-distribution structures of samples. Moreover, to efficiently estimate the distribution's structure, the authors adopt a curriculum paradigm to progressively train the proposed denoising and relabeling allocator. Additionally, they present a computation method for the proposed CSOT that ensures faster processing speeds, reducing computational overhead. Experimentally, this paper achieved SOTA performance on various benchmarks.
Strengths
1.Estimating the intra- and inter-structure coherence of samples is a convincing and reliable method for improving relabeling accuracy. The proposed prediction-level and label-level consistency constraints seem also interesting and plausible.
2.The combination of the proposed OT and curriculum learning for solving INL is also smooth and makes sense.
Weaknesses
1.The effectiveness for alignment of global and local structures between samples and classes is not fully convincing. The ablation results for OT in Table 3 seem weak. The comparison between row (a) with 78.07 and row (b) with 78.65 suggests that the performance improvement brought by the proposed prediction-level and label-level constraints is limited. Moreover, the introduction of two additional constraints adds complexity to the optimization. Similarly, row (e), CSOT w/o Ω^{L}, achieves the best performance, indicating that the benefits brought by the prediction-level and label-level constraints are unstable.
2.As the part that readers are most concerned about, the section 4.3, the loss function needs to be able to reflect the integrity of the method and the specific combination with its own innovation points. I can’t see the innovation of this article in the loss function here, and each loss term is an existing work. The work in this paper seems to be only used to build a dataset $\mathcal{D}_{\text{clean}}$ and $\mathcal{D}_{\text{corrupted}}$ for training? This structure and method of writing can greatly weaken the contribution of this paper. Besides, I would like to suggest the authors provide an overall alghrothim to show the whole training process, where the proposed methods would have been used during each training epoch and the training objectives are not the key point in this paper.
3.Considering the complexity of the proposed algorithm, and its marginal improvement over previous methods on two real datasets in Table 2, especially compared to NCE. The effectiveness of this work is questionable. Additional experiments are suggested, especially on Clothing1M. Besides, some related works[1] should be discussed which are published recently.
[1] OT-Filter: An Optimal Transport Filter for Learning with Noisy Labels (CVPR 2023)
4.The results of row (g) in Table 3 are not sufficient to show that the performance improvement is brought about by the method in this paper. For example, we need a more detailed ablation study to explain the role of NCE loss and the role of CSOT.
5.The structure of the article is confusing, which weakens the contribution of this article. Secondly, the introduction of some tool concepts is quite abrupt. For example, the proposal of Eq. (3) and (6) gives people a new form of OT formulation and will give specific solutions later. This expectation is affected by the entropy regularization in the following text, whose introduction is abrupt. This results in the final Eq. (16) which does not look significantly different from the original sinkhorn algorithm. It is recommended to introduce sinkhorn from the beginning and to emphasize that there are new constraints based on it.
6.The characters in the figures in the experimental part are too small, which affects reading. At the same time, why are there two figures, and their labels are figure 2? Moreover, Fig. 4 is quoted in the description of the text, but there is no figure 4 in fact.
Questions
1.For more precision, I would like to suggest providing a more detailed description of Equation (6). What does the left constraint in Equation (6) signify? Why should $Q\mathbb{1}_{C}$ not exceed $\frac{1}{B}\mathbb{1}_{B}$? Furthermore, it seems that $\frac{m}{c}\mathbb{1}_{C}$ is not a simplex. Could you please explain the rationale behind this relationship and provide additional descriptions or citations to support it? Additionally, the introduction of curriculum into SOT appears to be too direct and lacks smoothness.
2.The identification of selected samples in Equation (7) is confusing. What does "topK(W,⌊mB⌋)" mean? Based on my understanding, it implies that a sample pair (x_i, y_i) will be considered clean if it belongs to the top k convincing samples in the current training batch. I would suggest adding a description for the round-down symbol ⌊ and ⌋ to clarify its purpose.
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Limitations
The authors describe its limitations.