Summary
The paper presents a novel approach to dealing with entropic affinities (EAs) used in machine learning for dimensionality reduction tasks, specifically in the popular t-SNE algorithm. It addresses the limitations of current symmetrization methods applied to EAs, which can compromise the entropy and stochasticity properties of the affinity matrix. The proposed method uses optimal transport to achieve a natural symmetrization, leading to a new affinity matrix. This new matrix is then leveraged in a new dimensionality reduction algorithm called SNEkhorn, which is demonstrated to outperform state-of-the-art methods on both synthetic and real-world datasets.
Strengths
1. The paper introduces a novel approach to symmetrizing entropic affinities by formulating them as an optimal transport problem. This brings a fresh perspective to the problem and could open up new avenues for research and application in machine learning.
2. The proposed symmetrization method claims to maintain the constant entropy and stochasticity properties of the affinity matrix while being computationally efficient through dual ascent. This combination of robustness and efficiency is essential for practical applications.
3. The development of the SNEkhorn algorithm based on the new affinity matrix offers a practical implementation of the proposed approach. The demonstrated superiority of SNEkhorn over other baseline methods on various datasets further strengthens the paper's claims.
4. The paper evaluates the proposed SNEkhorn algorithm on both synthetic and real-world datasets, providing a comprehensive assessment of its performance and general applicability.
Weaknesses
Clarity of Presentation: While the abstract provides a high-level overview, some concepts, such as entropic affinities, optimal transport, and dual ascent, might be challenging for readers not already familiar with the domain. The paper should provide a clear and concise introduction to these concepts, ideally with intuitive explanations and illustrative examples. For example, what is P and C on paper 3 "Symmetric Entropy-Constrained Optimal Transport"
Robustness Analysis: Since the proposed method claims to be robust to varying noise levels, it would be valuable to include a thorough analysis of its performance under different levels of noise in the datasets instead of two level. This could strengthen the claim of robustness and highlight the algorithm's practicality in real-world scenarios.
Questions
1. What sensitivity analysis was performed on the hyperparameters introduced by the new symmetrization approach? How robust is the SNEkhorn algorithm to changes in these parameters?
2. Can you provide additional insights into the interpretability of the reduced representations obtained by SNEkhorn? How well does it preserve meaningful structure in the data compared to other methods? Is the visualization in Figure 6 a special case?
3.Could you elaborate on the limitations or potential failure cases of the proposed approach? What are the scenarios where SNEkhorn might not perform as well or could face challenges?
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Limitations
While the paper proposes a novel approach to symmetrizing entropic affinities and introduces the SNEkhorn algorithm with promising results, it is essential to consider the potential limitations and shortcomings of the work. Some possible limitations include:
Scalability: Dimensionality reduction algorithms often face challenges with scalability when dealing with large datasets. The paper should address the computational efficiency and scalability of the proposed SNEkhorn algorithm, especially when applied to high-dimensional and massive datasets commonly encountered in real-world applications.
Parameter Sensitivity: It is essential to investigate the sensitivity of hyper parameters and assess how they impact the performance of the SNEkhorn algorithm, e.g. perplexity.