Hyperbolic Space with Hierarchical Margin Boosts Fine-Grained Learning from Coarse Labels

Summary

The paper adresses the issue of coarse to fine learning, _i.e._ coarse labels are present during training but the metric are computed on fine-grained labels. The paper introduces a method Poincaré embeddings with hierarchical cosine margins (PE-HCM). The paper suggests embedding the image representation in the hyperbolic space for coarse label classification to take advantage of its natural hierarchical organisation. It has a second branch that clusters with k-means instances that share the same coarse labels to simulate fine-grained pseudo-labels. It uses a hierarchical loss that aims at making the cosine distance between pairs match their relation, i.e. same instance label (same instance with different data augmentation), same cluster label (assigned by k-means), same coarse label (supervision), different coarse label (supervision). The paper validates experimentally the method on several benchmarks, CIFAR-100 and BREEDS - a collection of 4 datasets. The paper conducts ablation studies of the different components of the method.

Strengths

S1. The method is well justified. The adaptive hierarchical distance is interesting and works well experimentally. S2. Experimental results are encouraging. S3. Ablation studies are present.

Weaknesses

W1. Some works are missing in the related works: MaskCon (MaskCon: Masked Contrastive Learning for Coarse-Labelled Dataset, CVPR 2023) and Grafit (Grafit: Learning fine-grained image representations with coarse labels, ICCV 2021) that both tackle the coarse to fine task. And UnHyp (Unsupervised Hyperbolic Metric Learning, CVPR 2021) that uses hyperbolic space to create a hierarchy. W2. The paper lacks comparison to state-of-the-art methods such as MaskCon and Grafit. It could conduct experiments on the large scale iNaturalist-18 dataset. W3. The paper lacks comparison to unsupervised baselines, such as UnHyp or more recent state-of-the-art self-supervised-learning methods, such as Dino (Emerging Properties in Self-Supervised Vision Transformers, ICCV 2021), iBot (iBOT: Image BERT Pre-Training with Online Tokenizer, ICLR 2022) etc. Minor: W4. The paper could remind the reader, in the benchmark datasets section, what classes are present in the different datasets, and what are the coarse classes on the different datasets.

Questions

Q1. Line 145: The paper gives notations for coarse and fine-grained labels. Could the method take advantage of a hierarchy of classes rather than solely a coarse label? Q2. What is the impact of the granularity of the coarse labels, _i.e._ how much will the performance deteriorate when the coarse labels become more and more broad and on the contrary what happens when given the fine-grained labels? Q3. Line 227: the paper defines a distance that is used in the HCM module. It uses the Euclidian cosine distance, this is surprising as the paper states that it compares instances in the hyperbolic space. The paper should elaborate this design choice. Q4. Table 2: reports results on the BREEDS benchmark. I am not sure to understand what are the results indicated as “5-way” and “all-way”. Typos: - Table 1: Coarase —> Coarse - Table 5: European —> Euclidian

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

N/A

Summary

This paper proposes a method developed in the hyperbolic space to embed visual embeddings to deal with the task of fine-grained learning from coarse labels. By discovering the advantages of hyperbolic space, the authors design the hierarchical cosine margin manner and an adaptive hierarchical cosine distance, which favour modelling fine-grained objects. Experiments are conducted on five benchmark datasets. Evaluations show that the proposed method achieves superior recognition accuracy over competing solutions on these datasets.

Strengths

+ The proposed method is novel and makes sense, which has great technical contributions, especially for the introduction of hyperbolic space into the fine-grained learning from coarse labels. + The fine-grained learning from coarse labels task has good practical potential, which also is a challenging problem in computer vision and machine learning. Such a task deserves further exploration. + The hierarchical cosine margins and the adaptive hierarchical cosine distance are interesting. These modules are crucial for the whole framework of the proposed method and they are also tailored designs for the task studied in this paper. + The experiments are comprehensive and convincing. Overall, they can validate the effectiveness of the proposed method, as well as the proposed modules. + The paper is well-written and easy to follow.

Weaknesses

Although the motivation of Figure 1 is relatively clear, it can be further explained.

Questions

- As stated in the section of Hyperparameters, different numbers of clusters lead to different accuracy. How to choose an appropriate hyperparameter for a specific task for users? - Some minor issues of paper writing should be fixed, e.g., “Fig” should be fixed as “Fig.” or “Figure”.

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

4 excellent

Contribution

4 excellent

Limitations

This paper does not explicitly discuss its limitations.

Summary

The paper presents a novel method, PE-HCM, for fine-grained learning from coarse labels. The authors propose the use of the hyperbolic space for sample embedding and introduce a hierarchical cosine margins manner to enhance discriminative ability. The method combines supervised learning with coarse-grained labels and instance-level labels obtained through data enhancement and clustering. Experimental results on benchmark datasets demonstrate the effectiveness of the proposed approach.

Strengths

* The paper addresses an important problem in the field of fine-grained learning, which is the challenge of learning from coarse labels. Fine-grained recognition is crucial in various applications, and the proposed method offers a solution to improve the performance in this context. The significance of the problem adds value to the research presented in the paper. * The paper introduces several novel concepts and methods. The use of the hyperbolic space for sample embedding is a unique approach that has shown promise in other related works. Additionally, the introduction of the hierarchical cosine margins manner to enhance discriminative ability is an innovative strategy that contributes to refining the hierarchical divisions for fine-grained recognition. The combination of these concepts and methods in the proposed PE-HCM method demonstrates the originality of the research. * The paper exhibits good writing quality, characterized by clarity and soundness. The paper effectively communicates the motivation, methodology, and experimental results to the readers. The descriptions of the proposed method and the experimental setup are clear and well-presented. The writing style contributes to the overall understanding of the research and enhances the readability of the paper. * The paper demonstrates a high level of experimental rigor. The authors provide detailed descriptions of the benchmark datasets used, including their characteristics and sizes. They also compare the proposed method with several state-of-the-art models, covering a wide range of relevant approaches in the field of embedding learning. By conducting extensive experiments on multiple benchmark datasets, the authors validate the effectiveness of their proposed method and show its superiority over existing techniques. The inclusion of statistical measures such as mean accuracy and confidence intervals further enhances the reliability of the experimental results. * The paper showcases the adaptive ability of the proposed method through the introduction of the Adaptive Hierarchical Cosine Distance (AHCD) strategy. The authors track the update processes of the distance parameters and show that the method adjusts these parameters during training to align with the actual data distribution. Furthermore, the paper provides a thorough analysis of the trade-off hyperparameter α, demonstrating the sensitivity of the method to its value and offering insights into finding an appropriate balance between discriminative hierarchical embedding and generalization ability.

Weaknesses

* While the paper includes theoretical illustrations of sample pair distances, it would be beneficial for the authors to provide empirical evidence by visualizing the feature distances between real samples. * The paper focuses on the evaluation of the proposed method on specific benchmark datasets but lacks a broader discussion on the generalizability of the results. Providing insights into the transferability of the proposed approach to other fine-grained recognition tasks or datasets would strengthen the overall impact of the paper.

Questions

The paper introduces the use of the hyperbolic space for embedding samples and applies the hierarchical cosine margins to enhance the discriminative ability. While the results demonstrate improved performance in fine-grained recognition, I would like to inquire about the interpretability of the learned embeddings. Can the authors provide more insights into the interpretability of the hyperbolic embeddings and how the hierarchical cosine margins contribute to the separability of fine-grained categories? It would be helpful to include visualizations or case studies that illustrate how the learned embeddings capture meaningful hierarchical relationships and contribute to the discriminative power.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The paper concludes by discussing potential directions for future research, highlighting the importance of refining category-specific semantic hierarchical distance constraints. This suggestion indicates the authors' awareness of the limitations of the proposed method and opens up opportunities for further exploration and improvement in fine-grained learning from coarse labels.

Summary

This paper addresses the challenge of learning fine-grained embeddings from coarse labels, where detailed distinctions required for fine-grained tasks are often lacking. The authors propose a novel method that embeds visual embeddings into a hyperbolic space and enhances their discriminative ability using a hierarchical cosine margins approach. The hyperbolic space offers advantages such as capturing hierarchical relationships and increased expressive power, making it suitable for modeling fine-grained objects. By enforcing relatively large/small similarity margins between coarse/fine classes in the hyperbolic space, the proposed hierarchical cosine margins approach is introduced. While enforcing similarity margins in the regular Euclidean space is popular for deep embedding learning, extending it to the hyperbolic space is non-trivial and valuable for coarse-to-fine generalization. Experiments are conducted on five benchmark datasets.

Strengths

1. Novel Method: The paper introduces a novel method, Poincare embedding with hierarchical cosine margins (PE-HCM), which addresses the challenging task of fine-grained learning from coarse-grained labels. This innovative approach bridges the gap between coarse-grained and fine-grained labels, enabling the transfer of knowledge from coarse-grained categories to fine-grained recognition tasks. 2. Technical Contribution: The proposed method incorporates key technical contributions. It utilizes the hyperbolic space to capture hierarchical relationships and provide increased expressive power. Additionally, the hierarchical cosine margins enforce proximity relationships among sample pairs, enabling the model to learn fine-grained features and distinctions. The adaptive strategy for updating target distances further enhances the model's ability to capture underlying relationships and adapt to the fine-grained characteristics of the data. 3. Paper Writing: The paper is well-written and effectively communicates the motivations, challenges, and proposed solutions. The introduction provides a clear background and problem statement, leading to a comprehensive description of the proposed method. The technical details, including the use of hyperbolic space and hierarchical cosine margins, are explained concisely, making the paper accessible to readers. 4. Experimental Performance: The proposed method is thoroughly evaluated on five benchmark datasets, demonstrating its superior performance compared to competing solutions. By achieving state-of-the-art recognition accuracy, the experimental results highlight the effectiveness and potential practical applications of the proposed method in fine-grained recognition tasks.

Weaknesses

1. Limited Analysis of Failure Cases: The paper could include a discussion of the failure cases of the proposed method. Analyzing scenarios where the method does not perform well would provide insights into potential weaknesses and opportunities for future improvements. 2. Minor Issues: There are several typos throughout this paper, e.g., in Table 1, Coarase should be Coarse.

Questions

I am curious about how does the proposed method perform when the available coarse-grained labels are noisy or contain inaccuracies?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

N/A

After rebuttal

After reviewing the author's response and considering the feedback from other reviewers, I have decided to maintain my initial score of "strong accept". I believe this is a solid paper.