StableFDG: Style and Attention Based Learning for Federated Domain Generalization

Summary

This paper proposes a style and attention based learning strategy for federated domain generalization which consists of two stages. The first stage shares style information among clients and then explorates and augments styles, and the second stage highlights the essential characteristics of images with attention mechanism. Experiments show that the proposed method achieves a new state of the art.

Strengths

- The style-based learning strategy is novel and intuitive. - The paper is well written and easy to follow - The proposed method is technically sound with superior results on the 5 datasets.

Weaknesses

The detail of oversampling is not fully covered in the paper.

Questions

- Why can eq.7. avoid performance degradation when there is little commonality between key and query images ? - How is oversampling implemented. What if the dataset itself is not class balanced ?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

The paper discusses its limitation in sec. 5.

Summary

The authors observe that existing domain generalization (DG) algorithms face the issue in the setting of federated learning (FL) because of the lack of samples in each client's local datasets/domains. They propose StableFDG, which utilizes both style-based learning which allows the clients to explore styles different from that of the original source domain, and an attention-based feature highlighter, which captures the similarities and common characteristics of features of samples of the same class. The authors conduct their evaluation on datasets such as Office-Home and Digits-DG.

Strengths

+ The evaluation is thorough, considering most of the popular benchmarks including the more challenging ones such as DomainNet. + The visuals are helpful and illustrative for readers to understand the proposed method

Weaknesses

- It would seem that the work could greatly benefit from works in the field of style transfer, which also involves extracting domain knowledge. It would be better for the authors to remark on the connections between their style sharing and exploration with style transfer. - It is not clearly explained what would happen after the style information is obtained by Equation 3. It's evident that this piece of information is used for style-shifting, etc., but how does such an operation relate to the intuition/big picture behind the approach?

Questions

+ Have you considered style transfer approaches to boost your current style-based learning? Style transfer approaches also strive to extract the domain representation (style); would this field help or hinder the performance of StableFDG?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

3 good

Limitations

- The authors argue that, because the label information is not included in the style parameters, the proposed approach doesn't have privacy issues. However, style statistics/parameters, derived from the domains, still contain domain knowledge. It's not clear and not elaborated why the privacy issue is resolved simply because the labels aren't shared.

Summary

In this paper, the authors present StableFDG, where a style-based strategy enables the model to reach a variety of new styles beyond each client's source domain, and an attention-based approach captures and emphasizes the important/common features of each category.

Strengths

The StableFDG method proposed in the article has achieved good experimental results. The experimental details in the article are particularly well articulated and richly illustrated. The essay writing is well-organized and readable.

Weaknesses

The idea of enabling the model to reach a variety of new styles beyond the source domain of each client has been proposed by other method [1], and what the authors have done is more of an incremental improvement. In addition, the authors in a related work evaluate [1] to improve the computational and memory costs, but it itself takes up more memory than [1]. [1] Junming Chen, Meirui Jiang, Qi Dou, and Qifeng Chen. Federated domain generalization for image recognition via cross-client style transfer. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 361–370, 2023. The method proposed in section 3.1 of the article is more like a combination of the methods mentioned in the authors' text. In the "Where to apply style-based modules" subsection, the authors directly follow the way other methods apply modules, where more detailed ablation experiments could be done or the benefits or reasons for such application could be explained in the article. The authors propose a new experimental setup, are the other methods in the paper based on this experimental setup, and what are the advantages of such a new setup over the previous ones?

Questions

See the weakness.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The authors could have elaborated more fully on their new experimental setup.

Summary

This paper presents a novel domain generation method in the FL framework. The authors make two key contributions to enhance domain generalization in FL. Firstly, they introduce style-based learning, which empowers individual clients to explore new styles by leveraging both their own local statistics and those of other clients. This approach promotes domain diversity. Secondly, the authors propose an attention-based feature highlighter that captures similarities among the features of data samples within the same class. This component enables better learning of domain-invariant characteristics. To validate the efficacy of their proposed method, extensive experiments were conducted. The results demonstrate the superiority of the approach compared to existing methods on various benchmark datasets.

Strengths

This paper addresses an important practical problem within the FL setup. It provides comprehensive coverage of related works and is well-written for easy understanding. The extensive experiments conducted effectively demonstrate the effectiveness of the proposed method. The supplementary material is valuable.

Weaknesses

The adaptation of mixing styles and exploration to FL frameworks is straightforward. Also, the attention-based feature highlighter can be viewed as a variation of cross-attention.

Questions

1) In this paper, statistics are recalculated at each round. Instead, could the running mean and variance of Batch Normalization (BN) be used as a substitute? Also, I am curious about the relationship between style augmentation and BN. 2) The attention-based feature highlighter combines self-attention and cross-attention for feature refinement. However, it remains unclear why this specific method is tailored to Federated Domain Generalization (DG). Further explanation is needed to clarify this aspect. 3) During training, both self-attention and cross-attention are used in Eq. (7), but during testing, only self-attention is employed. It would be beneficial to confirm if using self-attention alone during training is not sufficient.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Due to page constraints, some answers to certain questions were provided in the supplementary material, particularly in Supplementary Appendix E. It would be beneficial to include relevant content from the supplementary material into the main body of the paper.