Summary
This paper shows fundamental limitations of transductive adversarial training of GNNs for node classification and advocates that adversarial training of GNNs should be in an inductive setting. Therefore, this paper proposes a more flexible message-passing scheme using learnable diffusion to gain robust representations under adversarial training, and uses additional local constraints for structural perturbation and an efficient attack (LR-BCD) that contains both local and global perturbations.
Strengths
This paper studies an important problem on the GNN adversarial robustness and provides some interesting observations and insights on this topic. In particular, the analysis and discussions of theoretical and empirical limitations of adversarial training in GNN transductive learning are insightful and useful for the community.
The proposed robust diffusion framework for adversarial training appears efficient in a fully inductive GNN setting with flexible GNN architectures and refined local/global topology perturbations.
Weaknesses
1. The technical contributions of the work seem a bit limited. It appears the proposed method is a combination of existing works, although it is a simple yet effective solution. The claimed novelty lies in the handling of the new local constraints with LR-BCD, but it does not seem to be technically different from PR-BCD. In addition, there is a lack of theoretical justification why the new projection imposes sensible global and local constraints.
2. Given the limited technical contributions of the methodologies, I would like to see more extensive experimental evaluations of the proposed framework. More evaluations on other homophilic and heterophilic datasets with more powerful adversarial attacks (e.g., adaptive attacks [1]) under inductive setting should be conducted to demonstrate the universality of the proposed methods. In addition, the comparison with other robust GNN models (e.g., ProGNN [2], GRAND [3], GARNET [4]).
[1] Felix Mujkanovic, Simon Geisler, Stephan Günnemann, and Aleksandar Bojchevski. Are defenses for graph neural networks robust? NeurIPS’22.
[2] Wei Jin, Yao Ma, Xiaorui Liu, Xianfeng Tang, Suhang Wang, and Jiliang Tang. Graph structure learning for robust graph neural networks. SIGKDD’22.
[3] Wenzheng Feng, Jie Zhang, Yuxiao Dong, Yu Han, Huanbo Luan, Qian Xu, Qiang Yang, Evgeny Kharlamov, and Jie Tang. Graph random neural network for semi-supervised learning on graphs. ICML’21.
[4] Deng et al. GARNET: Reduced-Rank Topology Learning for Robust and Scalable Graph Neural Networks. LoG'22.
3. Some minor points:
* It does not seem the equation (2) correctly reflects the principles of self-training, because theta’ appears predefined in the expression, while it is actually trained from labelled data.
* In Equation-(5), the sum operation in the constraint is not well defined.
* The authors may want to make the title more specific to better reflect the focus of this work, avoiding potential misleading and vagueness.
Questions
1. Please clarify how L/PR-BCD corresponds to adaptive attacks.
2. Please justify how the additional local constraints could preserve graph semantics.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.