On the Ability of Graph Neural Networks to Model Interactions Between Vertices

Graph neural networks (GNNs) are widely used for modeling complex interactions between entities represented as vertices of a graph. Despite recent efforts to theoretically analyze the expressive power of GNNs, a formal characterization of their ability to model interactions is lacking. The current paper aims to address this gap. Formalizing strength of interactions through an established measure known as separation rank, we quantify the ability of certain GNNs to model interaction between a given subset of vertices and its complement, i.e. between the sides of a given partition of input vertices. Our results reveal that the ability to model interaction is primarily determined by the partition's walk index -- a graph-theoretical characteristic defined by the number of walks originating from the boundary of the partition. Experiments with common GNN architectures corroborate this finding. As a practical application of our theory, we design an edge sparsification algorithm named Walk Index Sparsification (WIS), which preserves the ability of a GNN to model interactions when input edges are removed. WIS is simple, computationally efficient, and in our experiments has markedly outperformed alternative methods in terms of induced prediction accuracy. More broadly, it showcases the potential of improving GNNs by theoretically analyzing the interactions they can model.

Paper

References (100)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer uCaB6/10 · confidence 3/52023-07-04

Summary

The paper studies how to characterize the ability of Graph Neural Networks (GNNs) to model the interaction given a partition of vertices. They quanlify the interaction strength with separation rank, and prove that it is governed by walk index (number of walks starting from the partition boundary). This relation is empirically demonstrated in their experiments, where a GNN with higher walk index beats the one with lower Walk Index. Lastly, Walk Index Sparsification is proposed to perform edge deletion based on Walk Index, and it markedly outperforms other baselines in terms of preserving prediction accuracy.

Strengths

- An overall interesting theoretical work that supports the intuition: the interaction between two parts in a graph depends on how much interconnected their boundary is. - The orginization is good, the theoretical analysis looks sound and the experiments are solid. The application to edge sparsification is impressive.

Weaknesses

- Walk Index Sparsification is unable to scale up to large graphs with deep GNNs, given its $O(|E||V|^3\log(L))$ complexity. Correspondingly, the experiments only covered 2-WIS and 1-WIS. To further improve, the authors could discuss some potentially efficient implementations.

Questions

See Weaknesses.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

As stated in Weaknesses, the computational overhead is a concern.

Reviewer 2FCe7/10 · confidence 3/52023-07-05

Summary

The paper proposes a new way to analyze the capacity of GNNs based on the complexity of interactions they can model across a partitioning of the nodes. In particular, this complexity is quantified via *separation rank*, and, for a certain kind of GNN with product aggregation, this rank is proved to scale with *walk index*, the number of random walks originating from a vertex on the partition's boundary. Experiments indicate that this finding generalizes beyond the particular GNN in the proofs. First, three different GNNs are shown to perform better at an image classification task when the data are converted to a graph so as to maximize walk index across the relevant partition. Second, the paper proposes WIS, a way to reduce the number of edges in the graph while maintaining GNN performance, based on maintaining high walk indices. WIS is shown to outperform prior methods on several real-world networks.

Strengths

- Conceptual and theoretical explanations on the capacity of GNNs are an important topic, and this paper takes a fresh approach as compared to much of the WL-isomorphism-based work. - The paper is well-organized, and the writing is grammatical and easy-to-read. - Experiments show the superior efficacy of the proposed WIS method for graph sparsification relative to prior methods.

Weaknesses

- While the machinery of the proof appears to be quite complex, I would appreciate some description of the proof concept in the main paper, even if it is at a very high level. - Related to the prior point on the proof concept, I am not totally convinced that walk index is the fundamental quantity here. Particularly the results in Figure 3 for Cora, where 2-WIS's performance is essentially matched by 1-WIS (which simply relates to node degrees), suggest that there may be a simpler explanation. To convince the reader, perhaps there could be more conceptual description of walk index and separation rank, and why the two are fundamentally linked. Or perhaps there could be more experiments showing that walk index is more predictive of performance as compared to other notions of connectivity.

Questions

- Do you see this work as totally orthogonal to WL-isomorphism-based limitation work? As I see it, those results are all-or-nothing, in the sense that they show GNNs either can or cannot distinguish two kinds of graphs. The bound here instead is somehow softer and more nuanced, showing that the complexity at which the GNN can model some interaction (as quantified by separation rank) scales with walk index. Is there some sense in which, or some specific cases for which, the WL results are subsumed by your new results? - Related to the second 'weakness' above, do you see walk index as the fundamental notion here? If so, why?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Some limitations are noted throughout the work -- for example, it is noted that the theoretical results only formally apply to GNNs with product aggregation. A more collected discussion of limitations of the current analysis/understanding, and possible future directions, would be helpful.

Reviewer CGQv5/10 · confidence 4/52023-07-22

Summary

This paper proposes to analyze the expressivity of graph neural networks (GNNs) by their ability to model interactions. The authors consider GNNs with product aggregation scheme, and analyze their ability to model interactions for a given partition of the graph through the notion of separation rank. The authors show quantitatively that these GNNs can model interactions better for partitions with higher walk index (Theorem 1). Motivated from their theoretical analysis, the authors propose an edge sparsification algorithm --- Walk Index Sparsification (WIS, Algorithm 1, 2) --- to drop edges that have least amount of impact on GNN's ability to model interations for a given partition in a given graph. The utility of WIS is demonstrated on both graph-level prediction and vertex-level prediction tasks.

Strengths

Strengths: 1. The proposal to study the expressivity of GNNs via their ability to model vertex interactions is interesting. 2. The theoretical analysis is novel: by focusing on GNNs with product aggregation, the authors connect them to tensor networks, of which the notion of separation rank can be used to study the network's ability to model interactions. 3. The paper is overall well-written and easy to follow, where important technical details and illustrative examples are explained clearly in the supplementary materials.

Weaknesses

Weaknesses: 1. The theoretical analysis focus on a special kind of GNNs with product aggregation, whereas the empirical evaluations are all done using standard GNNs with mean aggregation with ReLU nonlinearity. Although the authors meant to show that theory results can carry over to standard GNNs, they do not explain explicitly how such connections can be drawn. If the main contribution of the paper is... (a) to show that using separation rank to understand GNNs is relevant, then the authors should outline how the analysis carries forward to standard GNNs (b) to show that walk index of partitions in a graph determines GNN performance, then the authors should examine the case where the original graph is replaced with a fully-connected graph, and discuss GNNs with Graph Transformers (c) to propose an edge sparsification algorithm, then comparisons with stronger baselines are needed, such as DropEdge [1] and curvature-based graph rewriting methods [2]. 2. The conclusion that GNNs can model interactions better on partitions that have high walk index (i.e. more interconnected) seems to crucially depend on the choice of aggregation function is tied to the graph topology. However, it seems possible to use different aggregation/pooling function that delineates from the graph topology to capture interactions among partitions with low walk index. More explanations on how to interpret the results are needed. References: [1] Rong, Yu, et al. "Dropedge: Towards deep graph convolutional networks on node classification." arXiv preprint arXiv:1907.10903 (2019). [2] Topping, Jake, et al. "Understanding over-squashing and bottlenecks on graphs via curvature." arXiv preprint arXiv:2111.14522 (2021).

Questions

1. The analysis focus on GNNs with product aggregation, i.e. multiplication of the neighborhood node's embedding vectors $\mathbf{h}^{(l, i)}=\odot_{j \in \mathcal{N}(i)}\left(\mathbf{W}^{(l)} \mathbf{h}^{(l-1, j)}\right)$, where $\mathcal{N}(i)$ depends on the graph topology. In this set-up, the authors show that such GNNs can model interactions among two partitions in a graph better if they are more inter-connected. Yet, earlier work by Cohen and Shashuwa [3] show that for the case of convolutional neural networks (CNNs), their ability to model interactions depend on the choice of pooling functions. In light of this result, it seems plausible for GNNs to model partitions that are less-connected (e.g., Fig 2 -left) if one delineates the aggregation function from the graph topology, which is also commonly used in the literature of graph rewriting (e.g. [2]). Can the authors provide more explicit interpretations of their results and comparison with related work using separation rank for CNNs/RNNs? 2. As a thought example: for GNNs with product aggregation, do they achieve maximally separation rank for a fully-connected graph? How would the WIS algorithm compare to a simple baseline: replacing the original graph with a fully-connected graph, followed by randomly dropping edges (as opposed to (i) edges are removed randomly from the original graph)? This baseline is motivated from [4], where the authors show that replacing the original graph with a fully-connected graph at the output layer can boost GNN performance. 3. In line 264-266, the authors explain that the edge sparsification algorithm should first choose the partitions of interest, whereas in Algorithm 1 and 2, such choice is default to partitions induced by singletons and the rest of the nodes. It seems possible that the optimal partition choice depends on the downstream tasks (e.g., graph-level clustering versus node-level prediction). Can the authors provide more details and recommendations? References: [3] Cohen, Nadav, and Amnon Shashua. "Inductive bias of deep convolutional networks through pooling geometry." arXiv preprint arXiv:1605.06743 (2016). [4] Alon, Uri, and Eran Yahav. "On the bottleneck of graph neural networks and its practical implications." arXiv preprint arXiv:2006.05205 (2020).

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The limitations were discussed, but can be explained more thoroughly through (1) detailed interpretations of the theoretical results (assumptions and applicability); (2) comparison with prior works that make use of separation rank to analyze expressivity of deep learning architectures.

Reviewer CGQv2023-08-12

Explicit comparison of stable rank bounds obtained from this paper versus prior works

Thank you very much for your detailed response. I have the follow-up question on your point of: *CNNs and RNNs can be cast as GNNs with certain input graph structures – grid graph for CNNs and chain graph for RNNs. Accordingly, our separation rank bounds extend those established for CNNs and RNNs to GNNs with arbitrary input graph structures.* Would it be possible to provide an explicit comparison, showing the separation rank bound obtained from your work for GNN, its applications for CNNs and RNNs (viewed as special instances of graphs), along side with previous bounds obtained solely for CNNs and RNNs? I believe this will increase the technical strength of the paper.

Authorsrebuttal2023-08-15

Certainly! We provide such a comparison below. We will elaborate upon this point in the revised manuscript; thank you for the question! For a two-dimensional grid graph with $N$ vertices, a message-passing GNN can be viewed as a CNN. Given $I \\subseteq [N]$, recall that $C_I$ denotes the boundary of the partition induced by $I$, i.e. the set of vertices with an edge crossing the partition. The $L - 1$ walk index of $I$ in the grid graph satisfies $| C_I | \\cdot 3^{L - 1} \\leq WI_{L - 1} (I) \\leq |C_I| \\cdot 5^{L - 1}$ (this is because each vertex has degree $5$ including self-loops, except those on the edge or corner of the grid which are of degree $3$ or $4$ respectively). Hence, Theorem 1 implies that the separation rank of the analyzed GNN with respect to $I$ is at most $D_h^{O (|C_I| \cdot 5^{L - 1})}$, where $D_h$ is the network’s width. Similarly, for a chain graph with $N$ vertices, a message-passing GNN can be viewed as a bidirectional RNN. Given a partition induced by $I \\subseteq [N]$, the $L - 1$ walk index of $I$ in the chain graph satisfies $| C_I | \\cdot 2^{L - 1} \\leq WI_{L - 1} (I) \\leq |C_I| \\cdot 3^{L - 1}$ (this is because each vertex has degree $3$ including self-loops, except those on the endpoints which are of degree $2$). Thus, Theorem 1 implies that the separation rank of the analyzed GNN with respect to $I$ is at most $D_h^{O( |C_I| \cdot 3^{L - 1} )}$. The above results — special cases of our theory — extend those of [1,2,3,4], which study the separation rank of certain CNNs and RNNs. In particular, the results extend the CNN framework of [1,2] by introducing overlaps to convolution windows,$^\dagger$ and the RNN framework of [3,4] by introducing bidirectionality. Given that the models analyzed in [1,2,3,4] are weaker, the corresponding bounds on separation rank developed in these papers are lower. Consider for example the “odd-even” and “low-high” partitions from Figure 1(c) of [1]. Denote the subsets yielding these partitions by $I_{odd}$ and $I_{low}$ respectively. Applying our CNN result to $I_{odd}$ and $I_{low}$ yields upper bounds of $D_h^{O (N \cdot 5^{L - 1})}$ and $D_h^{O( \sqrt{N} \cdot 5^{L - 1} )}$ respectively. In contrast, the bounds in [1,2] (which were derived for a weaker model) are $D_h^{O(N)}$ and $D_h$ respectively. Similarly, for a chain graph, consider the “odd-even” partition induced by $I_{odd} := \\{ 1, 3, …, N \\}$ and the “low-high” partition induced by $I_{low} := \\{1, \ldots, N / 2 \\}$. Applying our RNN result to $I_{odd}$ and $I_{low}$ yields upper bounds of $D_h^{O (N \\cdot 3^{L - 1})}$ and $D_h^{O( 3^{L - 1} )}$ respectively. In contrast, the bounds in [3,4] (which were derived for a weaker model) are $D_h^{O(N)}$ and $D_h$ respectively. We hope the above fully addresses your question. If not please let us know and we will happily expand! --- $\dagger$ Note that CNNs with overlapping windows were studied in [5], but this paper did not provide upper bounds on the separation rank. [1] Cohen, Nadav, and Amnon Shashua. "Inductive bias of deep convolutional networks through pooling geometry." ICLR (2017) [2] Levine, Yoav, et al. "Deep learning and quantum entanglement: Fundamental connections with implications to network design." ICLR (2018) [3] Khrulkov, Valentin, Alexander Novikov, and Ivan Oseledets. "Expressive power of recurrent neural networks." ICLR (2018) [4] Levine, Yoav, Or Sharir, and Amnon Shashua. "Benefits of depth for long-term memory of recurrent networks." (2018) [5] Sharir, Or, and Amnon Shashua. "On the expressive power of overlapping architectures of deep learning." ICLR (2018).

Reviewer CGQv2023-08-15

Clarification on the proof of "stronger" bound

Thank you for your further explanation. I follow the first half but want to clarify the comparison at the end: Since $D_h$ represents the network width, then $D_h > 1$. Given $D_h > 1$, and $L > 1$, shouldn't it be $$ D_h^{O(N)} < D_h^{O(N 5^{L-1})}; D_h^{O(N)} < D_h^{O(N 3^{L-1})} $$ Namely, the upper bounds in [1,2], [3,4] (LHS) seem to be sharper than yours (RHS) for the odd-even partition? What do I miss here?

Authorsrebuttal2023-08-15

The bounds from [1,2] and [3,4] are not sharper but rather apply to weaker architectures that can attain only a smaller separation rank. In particular, the CNNs considered in [1,2] have non-overlapping convolution windows while the analyzed GNN over a grid graph is equivalent to a CNN with overlapping convolution windows, as is typically the case in practice. Furthermore, the RNNs considered in [3,4] are unidirectional while the analyzed GNN over a chain graph is equivalent to a bidirectional RNN. We hope this has clarified why the bounds from [1,2,3,4] are lower (but not sharper). Please let us know if you have any further questions, we will happily elaborate.

Authorsrebuttal2023-08-17

Dear Reviewer, We hope that our latest response has fully addressed your question. Please let us know if you have any additional feedback; there is still some time left in the discussion period and we will gladly address any question or comment you may have. Authors

Reviewer CGQv2023-08-18

Thank you - this clarifies!

Thank you again for the clarification. This makes sense to me and I was confused with the wording "lower" versus "sharper". Maybe consider rephrasing as "their weaker model....tighter bound" to ease the burden of the readers. I encourage the authors to add these bound comparison and discussion to the paper, which will be a good contribution.

Authorsrebuttal2023-08-18

We will indeed include the comparison and discussion above in the paper; thank you for the suggestion and feedback! If by this we have addressed all of your questions, we would greatly appreciate it if you would consider increasing your score. Thank you again for the time and for the thoughtful engagement during the discussion period.

Reviewer 2FCe2023-08-14

Thank you for your response. I will keep my score to support the paper, though I hope that a few limitations will be noted clearly: - The theoretical results only apply to product aggregation, which is not commonly used to my knowledge, and, as Reviewer CGQv also noted, the aggregation function is important. - Empirically, k-WIS has not been shown to be more useful than 1-WIS (degree information). I think some demo to this effect would help corroborate the theory and overcome the point above about the aggregation function, even if the k-WIS variant is not scalable.

Authorsrebuttal2023-08-15

We are adding to the text a concentrated account of limitations and directions for future work. The points you raised (which currently are mentioned throughout) will be noted clearly in this account. Thank you for the support!

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC