Meta-Exploiting Frequency Prior for Cross-Domain Few-Shot Learning

Summary

This paper introduces a novel framework for cross-domain few-shot learning. The input images are first decomposed into low-frequency content and high-frequency structure using FFT. Then, the PRM-Net includes three branches, low-frequency, high-frequency, and main branch. The PRM-Net includes two priors to regularize the feature embedding network. The approach shows state-of-the-art results on multiple benchmarks.

Strengths

1. The use of frequency decomposition to address CD-FSL task is easy to understand and implement. 2. The method is thoroughly evaluated on multiple benchmarks. 3. The method achieves state-of-the-art results on multiple benchmarks. 4. The paper provides a clear and detailed description of each component of the method.

Weaknesses

1. The method avoids additional inference costs, however, the decomposition and additional branches (high-low frequency branches) could introduce significant computational overhead. Moreover, the improvement by incorporating the reconstruction prior seems marginal according to Table 3. 2. Lack of proper citation for the datasets mentioned in section 3.1, e.g., CropDisease, CUB, ISIC. 3. The reliance on fixed decomposition strategies like FFT might not be optimal for some scenarios, e.g., the foreground objects have similar colors to the background, or the background is complex. The whole model cannot be trained end-to-end due to FFT.

Questions

1. How do you see the value of research in the (cross-domain) few-shot learning while the foundation models can perform well in zero-shot tasks? 2. Is it possible to combine this approach with fine-tuning strategies to further enhance performance in certain domains?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

n/a

Summary

The paper introduces an innovative framework that leverages the concept of frequency priors for cross-domain few-shot learning. The novel idea of decomposing images into high and low-frequency components and integrating these into the meta-learning process is a creative advancement in the field. Built upon established image transformation theories such as the Fourier Transform, the proposed method is grounded in a solid theoretical foundation, which enhances its credibility and applicability across various domains. The empirical validation through extensive experiments on multiple cross-domain benchmarks further substantiates the effectiveness of the framework, showcasing its superiority over state-of-the-art methods. And the experiment, along with the supplementary materials, has been very comprehensive, addressing most of my questions.

Strengths

The paper is commendably structured, with a clear and concise presentation of ideas, and it exhibits a high degree of reproducibility, further enhanced by the open sourcing of some of the code. The abstract and introduction effectively encapsulate the motivation and key contributions of the work, providing readers with a quick yet comprehensive understanding. The methodology is explained in a step-by-step fashion, making it accessible to readers who may not be intimately familiar with meta-learning or frequency domain analysis, thereby demonstrating the paper's strength in both clarity and comprehensiveness.

Weaknesses

(1) While the paper highlights the efficiency advantages of the proposed method, it falls short of offering a detailed analysis of computational complexity. A critical evaluation extending beyond inference times to include the total floating-point operations (FLOPs) during both training and inference phases is essential. Understanding the complete computational costs provides insights into the method's practicality, scalability, and suitability for deployment across varying computational environments. (2) The paper could be improved by including more extensive ablation studies on the choice of backbone network and loss functions. The current presentation lacks a thorough exploration of why specific architectural decisions and loss formulations were made, which is crucial for understanding the contribution of these choices to the overall performance. (2) Although the paper demonstrates strong empirical results, there is a need for a more profound theoretical analysis underpinning the effectiveness of frequency priors in cross-domain generalization. A deeper theoretical understanding would bolster the paper's claims and offer insights into the broader applicability of the proposed method.

Questions

(1) Is there a hyperparameter used during the FFT process to determine the high and low-frequency information? If so, how was the threshold for decoupling these frequencies determined, and how might it influence the experimental results? (2) How does the model perform over a longer period of training, and are there any signs of degradation or improvement in performance? Can the loss optimization curve and loss landscape be provided? (3) The content task and structure task are measured in a decoupled manner. Are these tasks entirely independent, and theoretically, could the low-frequency information be used as the query set to predict within the raw image support set, and vice versa for high-frequency information? And so on. (4) Besides the EMA (Exponential Moving Average) update method, what other update mechanisms have been considered or could be applicable to the model's training process? (5) There seems to be a limited ablation study regarding the choice of loss functions. Why was MSE chosen for the Feature Reconstruction Loss, and could cosine similarity loss be a viable alternative? Similarly, why was KL divergence chosen for the Prediction Loss, and was this decision based on empirical results or theoretical analysis? (6)The experiments were conducted using ResNet-10. Could the authors provide insights into whether the proposed method could be applied to larger models such as ResNet-101? Additionally, could the method be beneficial when applied to models like CLIP from OpenAI or the DINO model, and are there any explorations in this direction? (7) Some of the experimental content in the supplementary materials could be incorporated into the main text, such as the sections on 'low' and 'high' in Table 4. This is just a suggestion.

Rating

7

Confidence

5

Soundness

4

Presentation

4

Contribution

3

Limitations

The authors have transparently addressed the limitations of their method, particularly its performance on the Chest dataset, and have responsibly considered the broader societal impacts, confirming no negative social effects. They also acknowledge the increased training time associated with their approach. Proactively, they suggest future improvements and responsibly guide how their work can contribute to the field's progression, aligning well with the checklist for addressing limitations and societal impacts.

Summary

The paper introduces a novel framework called Meta-Exploiting Frequency Prior for Cross-Domain Few-Shot Learning, which aims to improve meta-learning's generalization by decomposing images into high- and low-frequency components. This method leverages these components to guide the feature embedding network, enhancing category prediction consistency and feature reconstruction. The framework achieves state-of-the-art results on various cross-domain few-shot learning benchmarks, demonstrating its effectiveness and efficiency.

Strengths

1. The paper presents a novel approach by introducing the concept of exploiting cross-domain invariant frequency priors for few-shot learning. The idea of using cross-domain invariant frequency priors is interesting. 2. The proposed framework is rigorously evaluated on multiple cross-domain few-shot learning benchmarks, demonstrating its effectiveness. 3. Good writing and clear presentation.

Weaknesses

1. The method relies heavily on the strong prior assumption that using low and high-frequency components can effectively mimic the distribution shift encountered during testing. However, this assumption may not hold true when the test and training datasets are entirely unrelated, as observed in the EuroSAT dataset where the performance of the method is not as impressive. The authors need to clarify and justify this assumption. 2. The paper does not provide an evaluation of the method's performance on within-domain few-shot learning tasks. Understanding how the proposed framework performs in a scenario where the training and testing data come from the same domain could provide a more comprehensive view of its robustness and generalizability. 3. The computational complexity of the proposed framework during the training phase is not thoroughly discussed. While the paper claims no additional inference cost, the added steps of image decomposition and feature reconstruction during training could introduce significant computational overhead. Detailed analysis and discussion on the computational requirements and efficiency would be beneficial. 4. The impact of the choice of image decomposition method (e.g., Fast Fourier Transform) on the overall performance of the framework is not thoroughly explored. Alternative decomposition techniques may yield better results or be more computationally efficient. A comparison of different decomposition methods and their influence on the performance and efficiency of the framework would provide valuable insights and strengthen the paper's contributions.

Questions

See weakness

Rating

6

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

yes