Lightweight Frequency Masker for Cross-Domain Few-Shot Semantic Segmentation

Summary

The paper presents a novel approach to cross-domain few-shot semantic segmentation (CD-FSS) by introducing a lightweight frequency masker that aims to improve performance by filtering different frequency components for target domains. The authors propose an amplitude-phase-masker (APM) module and an adaptive channel phase attention (ACPA) module to reduce inter-channel correlations and enhance feature robustness against domain gaps.

Strengths

1. The paper addresses a relevant and challenging problem in the field of few-shot semantic segmentation, particularly in cross-domain scenarios where performance typically suffers due to domain shifts. 2. The proposed lightweight frequency masker, including the APM and ACPA modules, introduces a novel perspective on feature disentanglement in the frequency domain, which is a promising direction for improving generalization across domains. 3. The authors provide a thorough interpretation of the phenomenon of frequency filtering and its impact on feature channel correlations, which is well-supported by mathematical derivations and empirical evidence. 4. The paper includes extensive experiments on four target datasets, demonstrating the effectiveness of the proposed method in reducing domain gaps and improving segmentation performance.

Weaknesses

1. The novelty of the approach among existing work could be better established. It would be better to give a more detailed comparison with state-of-the-art methods that also attempt to address domain shifts in few-shot segmentation, especially those through the frequency operations. 2. The paper does not discuss the computational efficiency of the proposed method, which is an important consideration for practical applications. It would be beneficial to include details on the runtime and resource requirements of the approach. 3. Since overfitting is an important issue in the few-shot finetuning, I wonder how this method could benefit the model in reducing the overfitting.

Questions

NA

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

NA

Summary

This paper discover a phenomenon that simpy filtering different frequency components for target domains can lead to a significant performance improvements. Then the paper delve into this phenomenon for an interpretation, and propose an approach based on this phenomenon, which achieves futher performance improvements. The proposed method includes an amplitude-phase-masker (APM) module and an Adaptive Channel Phase Attention (ACPA) module, which are lightweight but effective as validated by experiments.

Strengths

1. The paper identifies an intriguing phenomenon where frequency filtering leads to performance gains in CD-FSS, which is a novel contribution to the field. 2. The proposed lightweight frequency masker introduces minimal additional parameters (0.01%) yet achieves significant performance improvements (over 10% on average), which is a strong practical contribution. 3. The paper includes extensive experiments on four target datasets, demonstrating the effectiveness of the proposed method.

Weaknesses

1. Can this method be applied to other domains or tasks such as Cross domain few shot learning, domain generalization? 2. It would be helpful to include a sensitivity analysis on the choice of frequency components to filter, to understand the robustness of the method to different filtering strategies. 3. Some existing methods such as GFNet also applied filtering on the frequency domain, it would be better to compare with these methods, both in the related work and in the experiments.

Questions

1. How this method can help other tasks such as cross domain few-shot learning, domain generalization, or few-shot object detection? 2. How is this work compared to other works applying frequency operation?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

This paper addresses the CDFSS task from the aspect of frequency analysis, however, solely the frequecy analysis is not an novel aspect. This paper lacks discussion about the difference with previous works regarding the frequency analysis. But in all, I still recognize the novelty and contribution of this paper.

Summary

This paper presents a novel approach to cross-domain few-shot semantic segmentation (CD-FSS) by introducing a lightweight frequency masker. This masker aims to enhance the robustness of models against domain gaps by filtering different frequency components during the testing phase. The authors claim that their method significantly improves performance, sometimes by as much as 14%, without the need for extensive retraining or parameter tuning.

Strengths

This paper introduces a novel frequency masker that is lightweight and does not require training on the source domain. The authors provide a clear explanation of the phenomenon where frequency filtering improves performance, supported by mathematical derivations and experiments. This paper demonstrates significant performance improvements on multiple target datasets, which is a strong empirical contribution. The proposed APM and ACPA modules are innovative and show promise in addressing the domain gap problem in few-shot segmentation.

Weaknesses

While the paper claims to reduce inter-channel correlation, why not directly constrain the model to reduce such correlation? How is this work compared with [27]? The paper could benefit from a more thorough comparison with state-of-the-art methods, particularly those that also employ frequency domain techniques. The authors might consider providing more details on the experimental setup, including data preprocessing and model training procedures, to ensure reproducibility.

Questions

Could other frequency-based works also achieve the correlation reduction?

Rating

8

Confidence

5

Soundness

4

Presentation

4

Contribution

3

Limitations

An important part of the analysis is the correlation reduction. However, the paper only includes a comparison with the reduction method by mutual information. Many other methods can also achieve this goal. In my opinion, the mathematical deduction only proves that the frequency operation is able to reduce the correlation, but does not show its advantages in such reduction. Therefore, I would like to see the author provide more comparison and analysis regarding this problem, such as directly comparing with [27].

Summary

This paper makes several notable contributions to the field of cross-domain few-shot segmentation (CD-FSS). The authors discover that filtering different frequency components for target domains can lead to significant performance improvements, attributing this to reduced inter-channel correlation in feature maps, which enhances robustness against domain gaps and expands activated regions for segmentation. Building on this insight, they propose a lightweight frequency masker comprising an Amplitude-Phase-Masker (APM) module and an Adaptive Channel Phase Attention (ACPA) module. These components effectively reduce channel correlations and further enhance segmentation performance. The proposed method demonstrates significant advancements over current state-of-the-art CD-FSS approaches, highlighting its potential impact on the field.

Strengths

This paper presents several notable advantages in the field of cross-domain few-shot segmentation (CD-FSS). It identifies a significant performance improvement by filtering different frequency components for target domains, which reduces inter-channel correlation in feature maps and enhances robustness against domain gaps. The proposed lightweight frequency masker, consisting of the Amplitude-Phase-Masker (APM) and Adaptive Channel Phase Attention (ACPA) modules, effectively reduces channel correlations and improves segmentation performance with minimal additional parameters. The authors also provide relevant mathematical derivations to support their findings. The method demonstrates substantial improvements over state-of-the-art CD-FSS methods, making it a significant contribution to the field.

Weaknesses

- Performing frequency domain filtering on features is likely to result in some loss of information, potentially damaging the original structure of the features. Moreover, the mask weights required for different domains should vary. Are the authors training the APM on the source domain and then directly testing it on different target domains? - The novelty of the method is limited. The idea proposed by the authors is very similar to [1] and seems to merely apply cross-domain techniques to cross-domain few-shot segmentation. - The method proposed by the authors shows very limited improvement on some datasets and even performs worse than the existing state-of-the-art (SOTA) methods. [1] Deep Frequency Filtering for Domain Generalization CVPR2023

Questions

1. The authors claim that filtering certain frequency components can lead to significant performance improvements. How sensitive is this improvement to the specific frequency components chosen? Is there a systematic way to determine the optimal frequency components for a given domain? 2. The paper introduces the Amplitude-Phase Masker (APM) module. How does the initialization of the APM affect the final performance? Have the authors explored different initialization strategies? 3. The Adaptive Channel Phase Attention (ACPA) module uses phase information for attention weights. What is the rationale behind using only phase information rather than both phase and amplitude? How would the results change if amplitude information were incorporated? 4. The paper claims that the proposed method reduces inter-channel correlation in feature maps. How does this reduction in correlation compare to other feature decorrelation methods in the literature, such as those based on orthogonality constraints or whitening? 5. The authors use mutual information to measure inter-channel correlation. Are there other metrics that could provide additional insights into the nature of the feature disentanglement achieved by this method?

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

1. Computational overhead: The paper does not adequately address the computational cost of the proposed method. Frequency domain operations and the additional modules (APM and ACPA) likely introduce significant computational overhead, which should be quantified and compared to existing methods. 2. Ablation studies: The paper would benefit from more comprehensive ablation studies. For instance, the individual contributions of APM and ACPA are not clearly delineated, and the impact of different design choices within these modules is not thoroughly explored.

Discussion phase ends soon, please consider participate ASAP

Dear Reviewer jSMF, Please be reminded that the Author-Reviewer discussion phase will end very soon (in ONE day). Please take a look at the authors' rebuttal, see if they addressed your concerns. If you have any further questions/concerns, please post them ASAP, so that the authors may have time to respond to them! Thanks, AC

The discussion phase ends soon, please consider participate ASAP

Dear Reviewer 79M1, Please be reminded that the Author-Reviewer discussion phase will end very soon (in ONE day). Please take a look at the authors' rebuttal, see if they addressed your concerns. If you have any further questions/concerns, please post them ASAP, so that the authors may have time to respond to them! Thanks, AC

The discussion phase ends soon, please consider participate ASAP

Dear Reviewer H2xr, Please be reminded that the Author-Reviewer discussion phase will end very soon (in ONE day). Please take a look at the authors' rebuttal, see if they addressed your concerns. If you have any further questions/concerns, please post them ASAP, so that the authors may have time to respond to them! Thanks, AC