Transfer Learning of Graph Neural Networks with Ego-graph Information Maximization

Graph neural networks (GNNs) have achieved superior performance in various\napplications, but training dedicated GNNs can be costly for large-scale graphs.\nSome recent work started to study the pre-training of GNNs. However, none of\nthem provide theoretical insights into the design of their frameworks, or clear\nrequirements and guarantees towards their transferability. In this work, we\nestablish a theoretically grounded and practically useful framework for the\ntransfer learning of GNNs. Firstly, we propose a novel view towards the\nessential graph information and advocate the capturing of it as the goal of\ntransferable GNN training, which motivates the design of EGI (Ego-Graph\nInformation maximization) to analytically achieve this goal. Secondly, when\nnode features are structure-relevant, we conduct an analysis of EGI\ntransferability regarding the difference between the local graph Laplacians of\nthe source and target graphs. We conduct controlled synthetic experiments to\ndirectly justify our theoretical conclusions. Comprehensive experiments on two\nreal-world network datasets show consistent results in the analyzed setting of\ndirect-transfering, while those on large-scale knowledge graphs show promising\nresults in the more practical setting of transfering with fine-tuning.\n