FedSSP: Federated Graph Learning with Spectral Knowledge and Personalized Preference

Summary

This paper focuses on cross-domain Federated Graph Learning, where graph data stored in clients exists negative domain structural shifts. Authors observe the presence of spectral biases as a reflection of structural shifts. Thereafter, this work proposes the Generic Spectral Knowledge Sharing (GSKS), which allows clients to share certain components containing generic spectral knowledge. Moreover, it proposes Personalized Graph Preference Adjustment (PGPA) to satisfy preferences of clients for more suitable local applications. Extensive experiments on different scenarios demonstrates the superiority of the proposed methods.

Strengths

1. Well-motivated. In particular, it first observes the spectral biases as a reflection of structural shifts and further provides a targeted solution to address the biases. Moreover, the authors acknowledge the unique preferences of the dataset in this scenario may lead to the global message passing being not fully applicable to local data and accordingly propose appropriate solutions for the particularity of datasets. 2. Very easy to follow. The Figure is well presented. In particular, the authors provide a well-defined motivation visualization, helping with the understanding of specific challenges and the framework details. In addition, the precise framework presentation aids in the comprehension of each module’s role. 3. Structural heterogeneity is a common challenge in federated graph learning. The innovative solution presented in this paper offers a new perspective for future advancement by promoting improvements in spectral GNNs instead of traditional ones. It is not necessarily limited to cross-domain scenarios but can also bring new ideas for addressing structural heterogeneity in other non-IID federated graph learning contexts.

Weaknesses

1. Lacking of comprehensive comparative analysis of existing FGL methods. For instance, FGSSL appears in the performance table but is not comparatively analyzed in the introduction. The authors are supposed to clarify the shortcomings of this method in the given scenario.

Questions

I expected a more detailed comparative analysis of FGL baselines mentioned in the weakness. Could the authors provide a discussion on it?

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

4

Limitations

This work solves the problems in cross-domain settings well. I wonder whether it is generalizable enough to be extended to other non-iid problems, such as each client having a portion of the same graph dataset. It’s necessary to add relevant experiments to demonstrate that this method is not limited.

Summary

The paper introduces a novel method FedSSP designed to address the current limitations of personalized Federated Graph Learning methods. The author highlighten that existing methods fail to deal with domain structural shift and ignore the uniqueness of datasets in cross-dataset scenarios. To address the limitations, FedSSP innovatively utilizes spectral GNNs and shares key layers in them to facilitate generic communication. Meanwhile, the authors propose preference module for the preferences derive from different graph datasets.

Strengths

1. Addressing problems in pFGL from a spectral perspective is novel and interesting, setting a new direction for future research in federated graph learning. 2. The motivation is clear and compelling. The sharing strategy tackles the issue of knowledge conflict and spectral bias. Preference adjustment is employed to cater to the structural characteristics and preferences of individual datasets, thus ensuring that the graph features extracted are highly relevant and suitable. 3. The experiment is comprehensive, covering a wide range of scenarios to thoroughly evaluate the proposed methods. The ablation studies are thoroughly conducted, providing clear insights into the contributions of each component and demonstrating its effectiveness.

Weaknesses

1. Since the method of spectral encoder sharing essentially belongs to the personalized layer methods, I believe the authors need to discuss the differences between this method and other personalized layer methods in this paper, which they did not, such as APPLE. 2. How the node-level method FGSSL is used here in the experiments? What is the core model of APPLE here in the federated graph learning scenario? Could the authors provide the implementation details of the baselines? 3. The author changed the model architecture here but has not compared its communication cost with baselines.

Questions

Please refer to the Weakness part.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

Authors should discuss communication cost for this potential limitation.

Summary

This work proposed a personalized federated graph learning framework for federated graph classification. The framework includes strategies for sharing generic knowledge and satisfying personalized preferences. The authors evaluated six different cross-dataset and cross-domain settings and showed good performance.

Strengths

1. The writing in this paper is coherent, making the proposed concepts and methodologies easy to follow. The logical flow and concise explanations enhance overall readability and comprehension. 2. The methods are thoughtfully designed with a deep understanding of the specific challenges inherent in cross-domain scenarios. They effectively address these targeted issues, ensuring good performance and adaptability across diverse datasets and domains.

Weaknesses

1. In the motivation, the authors do not clearly explain the relationship between the two spectral bias metrics and the graph structure. 2. The scalability of the method is not well analyzed in the experiment. 3. Absence of a detailed discussion on the necessity of the shared filter encoder in the proposed method, particularly in relation to its specific architecture.

Questions

Q1) I have some confusion regarding the two spectral bias metrics used in the motivation. Could the authors specifically point out the relationship between these two spectral bias metrics and graph structure? Q2) Could the authors add experiments on the scalability of the method, such as performance with varying numbers of clients? Q3) The global consensus in the method is similar to the prototype. If it differs from prototype, authors should provide distinctions and explanations.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

2

Limitations

Yes

Summary

FedSSP tackles structural heterogeneity well in personalized Federated Graph Learning. When it comes to cross-domain scenarios, the structural heterogeneity becomes more negative than usual. It is crucial to mitigate the impact of domain structural shifts. FedSSP proposes two strategies to address these challenges from two directions. It firstly overcomes knowledge conflict by generic spectral encoder weights to seek better collaboration across various datasets. Besides, considering the uniqueness of clients in this scenario, it then satisfies the special preferences of the graphs by feature adjusting.

Strengths

- Studies on cross-domain federated graph learning are crucial for the applications of federated learning in the real world. This would help increase the generalizability in practical applications. This study promotes the generalizability of federated graph learning and demonstrates superior performance in various simulated settings, proving its advantage and broad applicability in practical applications. - The authors present a novel perspective and a spectra-based solution to tackle the issue of structural heterogeneity among clients. The proposed approach effectively addresses the variations in graph structures across clients, thereby enhancing the overall coherence and performance of the federated learning framework. - The generic spectral knowledge sharing and preference modules work harmoniously and complement each other effectively. After addressing knowledge conflicts through spectral bias mitigation, personalized adjustments act as further personalized optimizations, enhancing generic message passing. They together enable adaptive collaboration across diverse datasets and domains.

Weaknesses

The client scale considered in the experiments is quite small. Cross-device federated learning typically involves a significantly larger client scale to ensure the robustness and generalizability of the results. Therefore, it would be beneficial to consider expanding the client scale in experiments to better reflect practical scenarios and to validate the scalability of the proposed method.

Questions

Please refer to the weaknesses.

Rating

7

Confidence

4

Soundness

3

Presentation

4

Contribution

3

Limitations

The experiment should include performance under varying conditions to ensure the method has no limitations in this aspect, such as different numbers of training rounds.