Summary
The paper introduces a new mechanism called Cluster Information Transfer (CIT) to improve the generalization ability of GNNs to various and unknown test graphs with structure shift. The CIT mechanism enhances the diversity of nodes by combining different cluster information with the nodes, which helps GNNs learn invariant representations. The paper presents experimental results on several benchmark datasets, demonstrating that the proposed CIT-GNN model outperforms existing GNN models in terms of accuracy and macro-F1 score.
Strengths
- The paper is easy to understand.
- While the theory analysis is straightforward, it provides theoretically support for the proposed method.
- The experiments are extensive and the results look promising.
Weaknesses
- **Fair Novelty:** The proposed method is essentially an embedding transformation function that aims to eliminate the statistic variance between clusters that causes bias and hinders generalization. The novelty comes from the clustering mechanism and the augmented training using adjusted node embeddings. However, the spectral clustering mechanism has been introduced previously, and the transformation is relatively straightforward.
- **Assumptions and scope of the solution:** My second concern is about the assumptions of this paper. The paper generally assumes that there exist cluster / structural shifts between training and testing datasets. While structure could change across environments, the node labels remain invariant. The paper also implicitly assumes the node embeddings in different clusters are subject to Multivariate Gaussian distributions. These assumptions should be made explicit and formal. Moreover, the scope of structural shifts needs to be clarified. I understand that the variant of graph density or node degree can be one factor, but do other factors, e.g., edge homophily, also contribute to the shifts? A more general scope should be discussed in order to ensure its applicability.
- **More validation about the claim:** The examples in the introduction considers the shifts between sparse and dense graphs. One simple baseline to mitigate it is to augment the datasets or clusters via DropEdge and AddEdge compared to modifying node embeddings. While I understand that the graph structure is complex in the real world, as the authors claimed (Line 57-59), it would be good to experimentally validate the effectiveness of the method with a simple baseline that modifies the graph structure.
- **Lack of real datasets with natural shifts**: The experiments are somewhat synthetic in the sense that the training and testing sets are splitted with strong structural shifts. I wonder the performance of this method on datasets with more natural shifts, e.g., ogbn-arxiv, where the training and testing sets are splitted based on the time difference.
- **Missing Related Works:** It is unclear how the proposed clustering objective differs from the existing group clustering methods, e.g., Zhou et al. [1].
**Minor:**
- I feel Section 2 is not very informative since the setting is simple and the conclusion is not surprising. It would be good to demonstrate the structural shifts in more real settings or just combine this section to the introduction.
- In Line139 and 149, $m$ and $M$ are both used to indicate the number of clusters. It's better to make it consistent.
In general, the angle of mitigating structure shifts is interesting. However, several things above should be solved or clarified to understand its generality and effectiveness.
[1] Towards Deeper Graph Neural Networks with Differentiable Group Normalization. Kaixiong Zhou, Xiao Huang, Yuening Li, Daochen Zha, Rui Chen, Xia Hu.
Questions
- The method considers the embedding space only. Thus, it seems that the method can be extended to other domains, e.g., for images. If that is the case, are there any graph-specific properties that make this solution remains on the graph domain rather than other general domains?
- What's the method's performance when removing/adding 5%, 20% edges and not removing/adding any edges? Does the improvement of the method decline with the decreasing ratios?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Limitations
The authors discussed the limitations on the fact that the method could only apply to node-level tasks directly and the theory remain to be fully developed, as well as the limitation that the transfer remains on embedding space but graph topology.