Catastrophic Forgetting in Deep Graph Networks: an Introductory Benchmark for Graph Classification

In this work, we study the phenomenon of catastrophic forgetting in the graph\nrepresentation learning scenario. The primary objective of the analysis is to\nunderstand whether classical continual learning techniques for flat and\nsequential data have a tangible impact on performances when applied to graph\ndata. To do so, we experiment with a structure-agnostic model and a deep graph\nnetwork in a robust and controlled environment on three different datasets. The\nbenchmark is complemented by an investigation on the effect of\nstructure-preserving regularization techniques on catastrophic forgetting. We\nfind that replay is the most effective strategy in so far, which also benefits\nthe most from the use of regularization. Our findings suggest interesting\nfuture research at the intersection of the continual and graph representation\nlearning fields. Finally, we provide researchers with a flexible software\nframework to reproduce our results and carry out further experiments.\n