A Theory of Link Prediction via Relational Weisfeiler-Leman on Knowledge Graphs

Graph neural networks are prominent models for representation learning over graph-structured data. While the capabilities and limitations of these models are well-understood for simple graphs, our understanding remains incomplete in the context of knowledge graphs. Our goal is to provide a systematic understanding of the landscape of graph neural networks for knowledge graphs pertaining to the prominent task of link prediction. Our analysis entails a unifying perspective on seemingly unrelated models and unlocks a series of other models. The expressive power of various models is characterized via a corresponding relational Weisfeiler-Leman algorithm. This analysis is extended to provide a precise logical characterization of the class of functions captured by a class of graph neural networks. The theoretical findings presented in this paper explain the benefits of some widely employed practical design choices, which are validated empirically.

Paper

Similar papers

Peer review

Reviewer 1Kyu6/10 · confidence 2/52023-06-17

Summary

This paper focuses on graph neural networks (GNNs) for link prediction on knowledge graphs. It introduces the concept of conditional message-passing neural networks, which compute pairwise node representations based on a source node and a query relation. The paper analyzes the expressive power of these networks with a relational Weisfeiler-Leman algorithm and a logical characterization. Experimental analysis is conducted to validate the theoretical findings.

Strengths

1. The paper offers a systematic understanding of conditional message-passing neural networks for link prediction on knowledge graphs. 2. The paper provides insights into the expressive power of conditional message-passing neural networks over the general message-passing neural networks. 3. The paper conducts experimental analysis to empirically validate the theoretical findings. It explores the impact of different model choices, e.g., initialization, global readout functions, providing practical insights into the performance of graph neural networks for link prediction on knowledge graphs.

Weaknesses

1. The title and some lines in the introduction seem to be a little bit overclaimed. For instance, the title named a theory of link prediction seems to be claimed on the general link prediction task while the main discussion in the main paper is about the Knowledge Graph link prediction. besides, it is hard for me to understand lines 32-36. I could see from this paragraph that the paper focuses on the NBFNet-based model with conditional message passing. Nonetheless, it seems not so clear to me what higher-order messaging passing neural network computing the pairwise representation on the node. I would like to suggest authors to strengthen the focus of the paper at the beginning and then detail the difference between the RGCN, and CompGCN models with NBFNet. 2. The second bullet of the contribution part at line 52 claims that the paper designed a relational Weisfeiler-Lemma algorithm. It leads to the potential misleading that the paper is the first to propose the relational Weisfeiler-Lemma algorithm. Nonetheless, as discussed in line 97 in related work, the relationship is already included in the discussion in 'Weisfeiler and Leman go relational'. I would like to suggest the authors for a clear comparison with the related work on existing relational Weisfeiler-Lemma algorithms. 3. The intuition for logical characterization is not so clear. It seems to be hard for me to understand the logical characterization. How it contributes to the expressiveness is not so clear. I would like to suggest the author have more discussion on motivation and why the contribution is significant. 4. Lack of explanation on why only focuses on the inductive Link Prediction setting. Despite the NBFNet shows strong ability typically inductive setting. Nonetheless, the algorithm seems not so relevant in the transductive or inductive setting. I still think the experiments on both transductive and inductive settings should be taken into consideration.

Questions

See in Weakness

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

The authors have adequately addressed the limitations

Reviewer 7NY67/10 · confidence 4/52023-07-04

Summary

The paper investigates graph neural networks (GNNs) for link prediction in knowledge graphs. The authors propose a GNN that generalizes several existing techniques based on the labeling trick and NBFNets and investigate its expressive power by relating it to the Weisfeiler-Leman method and giving a logical characterization. Different design choices of the architecture are explored experimentally.

Strengths

* A highly relevant problem with various applications is addressed and neural architectures for related tasks are thoroughly analyzed, giving new theoretical insights. As the number of papers published in this area grows rapidly, it is of utmost importance to explain the advantage of practical design choices and to work out the benefits of specific architectures. The paper is a solid contribution in this direction. * The authors give a good overview of related work, including several recent results, and place their work well in the context of existing approaches. The paper continues recent work and generalizes it, which is a valuable contribution to fostering future work. * The paper is well-written and enjoyable to read; the presentation is on a high level.

Weaknesses

* The experimental comparison focuses on the design choices of the proposed architecture but does not compare to existing works. This would be desirable, in particular, since higher-order GNNs subsume some of the techniques (as stated in the paper). The trade-off between efficiency and expressivity appears to be crucial but is neither investigated theoretically in terms of time complexity nor experimentally in terms of measured runtimes. * The presentation of conditional messages passing remains abstract. It would be helpful to come up with an illustrative example, if possible. Minor comments: * l176: $\epsilon_u$ should be explained * l186: $\mathbf{z}_q$ as argument of MSG should be just $q$ following the notation introduced earlier * l370: "Thus, thus"

Questions

* How does the approach compare to NBFNets, architecture using the labeling trick, and higher-order WL-type GNNs regarding (i) accuracy and (ii) running time?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

Limitations are addressed sufficiently.

Reviewer kRBD5/10 · confidence 3/52023-07-05

Summary

The authors note that while GNNs are understood well in the context of simple graphs, there is a lack of comprehensive understanding when it comes to knowledge graphs. This study aims to systematically explore the use of GNNs in knowledge graphs, specifically in relation to the task of link prediction. The research includes a unifying view on different, seemingly unrelated models, potentially revealing new ones. The study evaluates the expressive power of different models using the relational Weisfeiler-Leman algorithm. The theoretical findings help clarify the advantages of some commonly used practical design choices, and these theories are supported by empirical validation.

Strengths

I have read the appendix of the paper. The technical details are clearly articulated, and the proof statements are clear and persuasive. The authors introduce conditional message-passing neural networks and define a relational Weisfeiler-Leman algorithm to substantiate that these neural networks can match the expressive power of this algorithm. The initialization progress is interesting.

Weaknesses

In the background section, the author first mentioned G = (V, E, R, c) while change it to G = (V, E, R, x) later. I know the author says it usually should be x instead of c. But can we directly use G = (V, E, R, x) at first to prevent confusion? More GNN baseline method should be involved in the experiments. The proof process is commendable, but could it be possible to draw a conclusion from it and incorporate it into the main text instead of the appendix?

Questions

Can the author add more baselines? Although the authors have already provided two datasets, each with four different versions, I think this is adequate, so I have not listed it as a weakness. However, it would be good if the authors could include one or two additional datasets.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

As the author mentioned, the method is limited to binary task such as link prediction.

Reviewer cjW94/10 · confidence 3/52023-07-31

Summary

This paper explores the expressive power of several GNNs designed for link prediction in knowledge graphs. The authors propose a conditional message passing framework with various designs for each component, a generalization of NBFNets. They also prove that the proposed framework can match the expressive power of a relational Weisfeiler-Leman algorithm. The experimental results demonstrate the impact of different model choices.

Strengths

1. The paper provides a theoretical understanding of some relational GNNs for knowledge graphs, such as NBFNets. 2. The proposed C-MPNN is reasonable, generalizing GNNs that can compute pairwise representations. 3. The authors study the effect of different model architectures, adding depth to the exploration of GNNs.

Weaknesses

1. Using relational asymmetric local 2-WL (rawl2) to measure the expressive power of C-MPNN is a little awkward. The definition of rawl2 is quite similar to C-MPNN, leading to potential confusion. 2. The experiments do not verify some theoretical findings, focusing mainly on different choices of model design. Certain theoretical discoveries, such as logical characterization, are not confirmed. 3. The experiments are primarily conducted on two datasets with differing data splits, leading to inconsistencies. More datasets and deeper analysis may be necessary to solidify the findings.

Questions

Please refer to the weakness.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

N/A

Authorsrebuttal2023-08-10

Adjusting the title

Thanks for your comment! We agree with your comment regarding the practical aspect. We acknowledged in our response that the empirical focus is on KGs. To make this also explicit in the title, we propose the following title: A Theory of Link Prediction via Relational Weisfeiler-Leman on Knowledge Graphs. We hope this addresses your concern and clears up the ambiguity. We also hope that the other concerns are addressed in our rebuttal.

Authorsrebuttal2023-08-14

Follow up

To follow up on this, could you please let us know whether the proposed change in the title addresses your concern (W1)? We will further highlight that the focus of this work is on KGs.

Reviewer 7NY62023-08-16

Thank you for your detailed reply. My questions regarding the trade-off between expressivity and complexity have been addressed sufficiently.

Authorsrebuttal2023-08-17

Thanks for your response, and for raising your score! Our fundamental contribution is to theoretically ground the behaviour of a class of architectures, and, as a result, to concretely explain their superior empirical performance based on formal criteria. By this perspective, our study forms a formal basis for future work which aims to design new architectures at the the trade-off between expressive power and scalability. We will integrate the discussions and the findings from the rebuttal period to the new version of the paper.

Authorsrebuttal2023-08-18

Thank you very much for going through the rebuttal, and for raising your score. Could you please let us know if there are any remaining concerns? This would allow us to hopefully clarify these concerns and get your final feedback before the rebuttal window closes. To the best our understanding, your main concern was regarding the lack of a high-level presentation of the proof sketches in the body of the paper: we are happy to integrate these explanations to the main paper to present our results at different granularities so that the reader gets a better idea of the proofs without going through them in detail. We hope this suggested change is satisfactory and fully addresses your concerns. Of course, we are happy to elaborate based on your input.

Reviewer kRBD2023-08-19

Thank you for your response. I understand your revision and recognize that there is a trade-off between detailed explanation and a summarized main idea. However, I believe that the current representation might still confuse users who do not read the detailed proof. Therefore, I would like to maintain my current score.

Authorsrebuttal2023-08-20

Thanks for acknowledging this trade-off which is almost always present in theoretical papers. To better inform you on how the high-level explanations would look like, we include these explanations for the sake of completeness: - __Overview of Theorem 5.1:__ We first show a correspondent characterization of the expressive power of R-MPNNs in terms of a relational variant of the WL test (Theorem A.1). This result generalizes results from Barceló et al [1]. Then we apply a reduction from C-MPNNs to R-MPNNs, that is, we carefully build an auxiliary knowledge graph (encoding the pairs of nodes of the original knowledge graph) to transfer our R-MPNN characterization to our sought C-MPNN characterization. - __Overview of Theorem 5.2:__ We start by showing a logical characterization for R-MPNNs (without global readout) in terms of a variant of graded modal logic called rGFO$^2_{\text{cnt}}$ (Theorem A.11), which generalizes results from Barceló et al [2] to the case of multiple relations. Then, as in the case of Theorem 5.1, we apply a reduction from C-MPNNs to R-MPNNs (without global readout) using an auxiliary knowledge graph and a useful translation between the logics rGFO$^2_{\text{cnt}}$ and rGFO$^3_{\text{cnt}}$. - __Overview of Theorem 5.3:__ Intuitively, our logic rGFO$^3_{\text{cnt}}$ from Theorem 5.2 only allows us to navigate the graph by moving to neighbors of the “current node”. The logic erGFO$^3_{\text{cnt}}$ is a simple extension that allows us to move also to non-neighbors (Proposition A.15 shows that this logic actually gives us more power). Adapting the translation from logic to GNNs from Theorem A.11, we can easily show that C-MPNNs with global readout can capture this extended logic. _[1] Pablo Barceló, Mikhail Galkin, Christopher Morris, and Miguel Romero. Weisfeiler and leman go relational. LoG, 2022._ _[2] Pablo Barceló, Egor V. Kostylev, Mikaël Monet, Jorge Pérez, Juan L. Reutter, and Juan Pablo Silva. The logical expressiveness of graph neural networks., ICLR, 2020._

Reviewer cjW92023-08-18

Thanks for the clarification from authors. I still have one questions: Does logical characterization means the global readout?

Authorsrebuttal2023-08-18

Thanks for going through our rebuttal! We answer your question below in two parts: ***What does the logical characterization achieve?*** When studying the expressive power of neural networks, one is typically interested in characterizing the class of functions that can be captured (or, approximated) by the neural network. In the context of graph machine learning, we are interested in the exact same question with the difference that the functions are defined over graphs. For the task of link prediction, this means the following: for each knowledge graph G, we are interested in functions of the form $f(G): R \times V \times V \to$ {0,1} , because we want to quantify whether a link $r(a,b)$ is true or not, where $r \in R$, and $a,b \in V$. This is precisely what is achieved by logical characterizations: if we can show that a GNN architecture can capture a logic L, then it means that this GNN architecture can capture all functions which can be expressed in this logic. This is very important, because if a GNN architecture A captures a *strictly larger logic* than another GNN architecture B, then we can conclude that A is *more expressive than* B. One implication is indeed regarding the use of global readout: specifically, we obtain that C-MPNNs with global readout are strictly more powerful than standard C-MPNNs. This means that there is *at least* one function that the former can capture but the latter cannot which is empirically confirmed in Appendix B.1. ***How does this compare to WL-based characterizations?*** Logical characterizations of GNNs date back to the work of Barceló et al [1]. Unlike the characterizations of GNNs based on the WL test, these logical characterizations are uniform, in the sense that they hold over the set of all graphs. Therefore, these studies are harder to obtain and yield more powerful characterizations, because they require a single parametrized model which would apply uniformly to all graphs. Uniformity condition is much more realistic, as it applies to all graphs of all size/structure, and hence desired in theoretical analysis. Non-uniform WL characterisations are indeed weaker than logical characterisations: one example is that they cannot recognise the expressiveness gain of using of global readout which is recognised by the logical characterization, as shown in Barceló et al [1]. Please let us know if this answers your question. These technical differences are intricate and easy to overlook, and we are happy to explain these more in our paper, if you agree these explanations are useful. [1] Pablo Barceló, Egor V. Kostylev, Mikaël Monet, Jorge Pérez, Juan L. Reutter, and Juan Pablo Silva. The logical expressiveness of graph neural networks., ICLR, 2020.

Authorsrebuttal2023-08-20

Could you please let us know whether our response answers your final question? We would also like to hear your opinion after our responses to the items 1-3 from your review. We hope your questions and concerns are adequately addressed - we are happy to elaborate more before the discussion window closes.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC