Calibrate and Boost Logical Expressiveness of GNN Over Multi-Relational and Temporal Graphs

Summary

The paper deals with classifying multi-relational graphs, and specifically, the R-GNN architecture, which deals with more than one type of edge by producing a different neighbor set for each of them, aggregating the messages of each neighbor, and then merging them together in a final update layer. The first results in the paper are that these architectures are limited, as Figure 1 shows, these architectures cannot even count the number of total neighbours of a node. To fix this, the proposal is to maintain the architecture, but apply a graph transformation task, some sort of graph reification. Intersectingly, R-GNNs and R^2-GNNs recover the expressive power of similar architectures in the single-relation scenario, when applied not under the original graphs but in the transformed graphs. The second part of the paper looks to implement these ideas specifically in Temporal Graphs, wherein one has a different relation for each timestamp measured in the temporal part. As multi-relational graphs, this is indeed a nice application of the framework. Results and experiments show that the proposed architecture competes with several other proposals for classification of temporal graphs.

Strengths

* Sound, robust theoretical study relating multi-label GNNs with logic. The proofs seem correct and use a variety of techniques. * Claims are backed up with examples, so that it is easy to quickly grasp the ideas of the paper. * The claims in the paper are backed up with experiments: It really does seem that the power of R-GNNs (and similar architectures) benefits from applying the graph transformation.

Weaknesses

* The results are tailored specifically to a single architecture (R-GNNs). One could think of several other ways to incorporate edge information, I know for instance GATs (Velickovic et al. 2018), but the paper does not discuss any other approach, and we don't know if the weakness shown in the results of section 4 are due to the specific architecture, or if there is a bigger problem underlying multi-relation graphs that must be tackled with the transformation. * The proposed solution involves a linear transformation in graphs, but probably demands much more layers in a GNN, as the length of every path is now multiplied by two. This involves adding extra costs to the learning process, and probably some difficulties in using the node embedding of the graph. * Synthetics experiments do not compare against other GNN architectures capable of dealing with different edges, appart from R-GNN. Real life experiments do compare against other temporal variants, but it is difficult to see if the added power is due to the global readout, the transformation, or the mix of everything in the architecture proposed.

Questions

* Please justify why a study on R-GNN is important, and whether your insights could be valuable when dealing with any other architecture incorporating edges. * I'd tend to think that maybe every multi-labelled GNN would benefit from the graph transformation approach, and like wise for the global readout. What would happen if I run GRU-GCN or TGN over a temporal graph that has already been transformed? Or if instead of using R^2-GNN I use any other GNN with the graph transformation?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

2 fair

Limitations

* Please discuss potential inconveniences regarding using the transformation in graphs. You probably need more layers, with the added cost on training. Also, even if the transformation is linear, it is not free for huge graphs, so your approach is probably better tailored to datasets containing several small graphs.

Summary

This paper justifies the logical expressiveness of R^2-GNN from the perspective of "universal" graph classes, including multigraphs and infinite graphs. Specifically, the R^2-GNN seems to be identical to the previous ACR-GNN in [1]. The engagement of graph classes makes theoretical statements in finer settings than previous ones in [1] which only involve one graph. The thorough inspection of different graph classes captures the expressiveness power of R^2-GNN compared to logical formulas in FOC_2. Moreover, this paper adapts a commonly used graph transformation $F$ to enhance the expressiveness of R^2-GNN to be stronger than FOC_2 on universal graph classes. The theoretical results collapse into existing ones in [1] when the graph classes contain finite graphs, which are naturally bounded. The theoretical results can be also extended to temporal graphs under static representations, which are studied in [2]. To achieve this, the authors investigate the expressiveness hierarchy between the collapse function $H$, two static representations, GNN and TGNN (also defined by the author), and graph transformation $F$. Empirical results support the expressiveness results partially related to $F$, $H$, and TGNN. [1] Barceló, P., Kostylev, E. V., Monet, M., Pérez, J., Reutter, J., & Silva, J. P. (2020, April). The logical expressiveness of graph neural networks. In 8th International Conference on Learning Representations (ICLR 2020). [2] Gao, J., & Ribeiro, B. (2021). On the equivalence between temporal and static graph representations for observational predictions. arXiv preprint arXiv:2103.07016.

Strengths

This paper makes a large amount of work to (1) justify the logical expressiveness of R^2-GNN compared to FOC_2 in universal, bounded, and simple graph classes; (2) empirical technics to improve the expressiveness of GNN. The theoretical results are original. Though some results are still missing to achieve a complete understanding of this problem. The presented ones are significant and inspiring.

Weaknesses

The presentation of this paper should be improved given the dense representation of theorems and proofs. Firstly, the content is not self-contained. The definition of static representation of temporal graphs is not included even in the appendix. Most importantly, the key definition to justify logical expressiveness is missing. In Lines 130-131, the author states that the R^2-GNN is the set of all R^2-GNN-based boolean node classifiers. However, it is far from a satisfactory definition, which might be risky for accepting this paper. Please check the question part for my questions about the definition. AnotherW ambiguous part of this paper is that the graph transformation $F$ for multi-graphs is isolated to FOC_2 classifiers. In fact, $F$ should be orthogonal to classifiers.

Questions

1. What is the definition of one boolean classifier set A as the subset of another boolean classifier set B for a given graph class GC? There are two possible definitions 1. For any a in A, there exists a b(a) in B so that a and b(a) achieve the same results on all G in GC. 2. For any G in GC and any a in A, there exists a b(G, a) in B so that a and b(G, a) achieve the same result on G. 2. $F$ enhanced the power of R^2-GNN classifiers but it never relates the R^2-GNN. Can we state that $F$ also enhances the expressiveness of FOC_2 classifiers? What will Figure 3 be if we include FOC_2 \odot F (or with proper modification of FOC_2 with new predicates introduced)? 3. For the temporal graph, the author draws two lines of hierarchies of expressiveness. What are the relationships between R^2-TGNN and time-and-graph and R^2-GNN\odot H and time-and-graph?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

Yes

Summary

The paper introduces a new connection between the power of the relational GNNs (working on multi-relational graphs) with a class of first-order logic functions. This class of functions named $\mathcal{FOC}_2$ is a subset of first-order logic functions constrained to having only two variables, but includes counting quantifiers. The theoretical analysis considers three classes of multi-relational graphs: universal graph class, bounded size multi-relational graphs, and simple graphs that two nodes can have just one type of connection. Based on these classes paper proves in general none of $R^2-GNN$ and $\mathcal{FOC}_2$ class of functions is a subset of each other, but adding a preprocess to the graph can make a model that is at least as powerful as the union of both classes. They also extend their results to the temporal graphs, where time series graphs can be collapsed into a static graph or considered as separate graphs. The paper also compares two different classes of models, time-and-graph versus time-then-graph, and proves second class is absolutely more powerful. Empirical results on synthetic and real-world graphs have been also conducted for supporting theoretical results.

Strengths

1. Paper for almost all parts is very well-written. 2. Paper finds the exact class of functions that can be learned by most common relational GNNs. 3. Theorems show the theoretical superiority of time-then-graph functions on temporal graphs over the time-and-graph models. 4. Paper finds a hierarchal classification of the power of different models on temporal graphs. 5. Having supporting experiments for showing that theoretical implications have real effects on real-world datasets.

Weaknesses

1. Intuition behind the functions used for synthetic graphs has not quite been explained. A little more explanation on why these functions and if they are just a random selection or one of the most simple functions that could convey a point will be helpful. 2. The paper relies a lot on definitions of time-then-graph and time-and-graph models and refers to previous work for their definitions. These definitions are better to be briefly explained at least as a section in the Appendix. 3. Section 6 starts with the Experiments and it is hard at first to understand that all synthetic are temporal. Particularly, superscripts on the relations were confusing at first without knowing they are temporal indices. It would have been better to clearly indicate that before starting and also include synthetic experiments for the general multi-relational graphs part. 4. Real-world experiment on the Brain-10 dataset is a little inconsistent with theoretical results. Minor errors: 1. Inconsistent use of $FOC$ and $\mathcal{FOC}$ in the notations. 2. Line 110, a space missing between "vectors" and "$\mathbf{x}_v^{(i)}$". 3. Line 339, "somorphism" -> "isomorphism"

Questions

1. Node-level binary classification tasks are limited in the real world. Theoretical results are insightful, however, I was wondering if these results can imply anything about any other type of tasks on the graphs, e.g. multi-class node classifications or graph-level tasks. 2. In Corollaries 4.1 and 9.1, it is interesting to see that both classes have precisely same power. However, this power seems to appear because of $\mathcal{FOC}_2$ being able to enumerate all possible combinations of relation counts. Is there any complexity measurement, e.g. time-complexity of evaluating a function on a graph, on how complex can the function under $\mathcal{FOC}_2$ be as the bound on the number of nodes in the graph or number of possible relations in the class increases?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

4 excellent

Limitations

The paper clearly defines the class of functions analysis. There are limitations on what types of graphs and tasks theoretical results work, but these limitations are clearly indicated in the paper.

Summary

The paper extends the logical characterization of GNN --- node classification, by investigating the relational case i.e. multi-relational graphs. The paper generalizes the work of Barcelo et al. [ICLR 2020], who provided the logical characterization of GNNs for node classification in terms of First Order Logic with two variables and counting quantifiers (FOC2). The paper discuss the R2-GNN model, that extends R-GNN --- a GNN architecture for multi-relational graphs --- with a global read out function. The global readout adds aggregation over node features to the combination function in each layer. Some of the key results of the paper are the following: - R2-GNN are not captured in FOC2 and FOC2 is not captured by R2-GNN (for unbounded graphs) - R2-GNN are captured in FOC2 for bounded graphs (given a known upper-bound on the number of nodes) but not vice versa - R2-GNN = FOC2 for simple graphs The paper then extends the expressivity of R2-GNNs to full FOC2 , by introducing a transformation F that runs in linear time w.r.t the multi-graph size (O(|V|+|E|)). Finally, the authors extend there framework to temporal graphs. This is achieved by simply adding additional predicates indexed by time. The authors then discuss the expressivity of this framework with the previously introduced transformation F applied to each time stamp and another transformation comprising union of each time stamped graph. Post-rebuttal: I have read author's rebuttal, and I am more confident about my rating and will keep it, i.e. acceptance.

Strengths

The paper extends logical characterization of GNNs in an interesting direction i.e. over relational structures. The theory presented is rather intuitive and the authors try to convey the key proof ideas. They also exploit their theoretical investigation to expand the expressivity of GNNs over relational structures and temporal graphs. They provide convincing experiments (although slightly redundant, see weaknesses) over synthetic and real-world data.

Weaknesses

- Quite terse for non-experts - Comparison of all the aggregation functions in tables are not much discussed in the paper and are not really important to the main message of the paper. Maybe this could be removed/shortened to give space for more proof intuitions?

Questions

When using many predicates, it seems that one could encounter some form of curse of dimensionality --- Do you think R2-GNNs suffer such issues? Specially in the temporal/transformation case, when number of predicates are further increased. In general, some discussion on how the complexity of learning changes with number of predicates could be interesting

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

3 good

Contribution

3 good

Limitations

The authors discuss the fact that R2-TGNN*F^{T} do not improve performance over R2-TGNN on real-world dataset, whereas on their synthetic dataset they do observe improvement. This ambivalence between theory and practice is quite interesting and further discussion of this--- as to what exactly is different in real-world data that makes more expressive models inferior --- could be very interesting

Summary

In this paper the authors provide a theoretical analysis of graph neural network (GNN) expressivity on multi-relational and temporal graphs. Specifically, they extend the multi-relational graph convolution network R-GCN to a class of architectures they call R-GNN, which allows for arbitrary aggregation (pooling neighbors with respect to a particular relation) and combination functions (pooling these aggregations with the node representation itself). They further extend this to a class of architectures they call R²-GNN, which adds a global "readout" function which aggregates the feature vectors of all nodes in the graph. They analyze the capability of R²-GNN to classify nodes and compare it to that of $\mathcal{FOC}_2$, a restricted subset of first-order logic which only allows formulas with at most 2 variables but also allows counting quantifiers. They find that neither is a subset of the other, in particular R²-GNN is not able to distinguish whether two neighbors of a given node with different relations are, in fact, the same node. However by first augmenting the graph with a particular transformation $F$ they are able to overcome this restriction, and ultimately show that $R^2-GNNs \circ F = \mathcal{FOC}_2$ on any graph class with a bounded number of nodes. All of this is then extended to temporal graphs, which can also be converted to a multi-relational graph (with a unique set of relations per time-step). A hierarchy of expressivity with respect to current temporal graph frameworks is presented. Finally, experimental results validating the theoretical expressivity on synthetic and real-world datasets are presented.

Strengths

The paper is, to the best of my knowledge, original, and theoretical results analyzing the expressivity of graph neural networks in the multi-relational setting are certainly of significant interest to the graph neural network community. The work is of a high quality, and presented clearly. Despite the highly technical nature of the results, the authors do a good job in motivating their work and providing intuition for the proofs.

Weaknesses

In general, a point of caution with any strongly theoretical work is whether it applies in practice. In this setting the largest concern comes from the graph transformations. It is not unusual for a graph transformation to extend capabilities in the way the authors have done. A classic example is to extend a method which works on undirected graphs to the setting of directed graphs by creating a new graph with twice as many nodes, one representing "head" and "tail" for each node, however this can make some relationships (eg. transitivity) very difficult to observe. In the author's setting, while the graph augmentation may afford the ability to prove theoretical expressivity results, one might be concerned that the resulting graph is augmented to the point that a GNN architecture may require greater depth to perform the same sort of tasks it did previously. Of course the authors have provided empirical evidence that this augmentation is not problematic in at least three graphs, which is quite reasonable to support their approach, however it is something to bear in mind.

Questions

1. What is the initialization used for the nodes in $V' \setminus V$? To the best I can tell, they do not have any unary predicates, so they would seem to be initialized with a vector of all zeros. 2. Not a question as much as a comment - the R²-GNN architecture is equivalent to the R-GNN architecture if we first augment the graph to have a new binary relation which forms a complete graph on all nodes. Given that the work already includes a graph augmentation step, an alternative presentation could potentially avoid the R²-GNN architecture altogether, and simply express the entire thing as R-GNN on an augmented graph. I'm not sure it is clearer to take this approach, but it might be worth noting. Most of the paper was very well written, but I did find the following typos (mostly in the intro): L37: authors -> the authors L45: authors -> the authors L46: in -> in the L47: edges in graphs -> edge in the graph L48: of -> (delete) L48: graph needs -> graphs need L51: as -> as a L57: be -> can be L60: to -> to the L60: Besides -> Moreover L61: extensively -> extensively in L69: in -> in the L70: power -> the power L71: such -> such a L72: mutli -> multi L91: to -> to denote L96: node Boolean -> Boolean node L111: vectorsx -> vectors x L338: do somorphism -> perform an isomorphism A minor suggestion: move the sentence in lines 170-171 to around line 85, to motivate the definition of bounded and simple graph classes.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

I did not see much discussion of the limitations of the proposed model. Some of the concerns outlined in the weaknesses section above are good candidates for this.

Summary

This work calibrate the logic expressiveness of R2-GNNs as node classifiers on multi-relational graphs. Motivated by some negative results, authors boost R2-GNNs with a graph transformation, which enables R2-GNN to capture FOC2 formulas. They further extend expressiveness results and graph transformation to temporal settings and derive an expressiveness hierarchy of temporal GNNs.

Strengths

1. Clear theoretical results on static and temporal settings. 2. Proposed transformation is straightforward and scalable.

Weaknesses

1. Nearly no related work. There is no related work section. Only the closest related work is mentioned in introduction and some related works are missing. For example, besides [1], some works also analyze GNN's expressivity as boolean classifier. Weifeiler-Leman test have been connected to boolean graph classifier for a long time [2]. [3] also analyze boolean link classifier. 2. The experiments are only conducted on small datasets. You can use larger DGS and AM in RGCN paper [4]. [1] Pablo Barceló, Egor V. Kostylev, Mikael Monet, Jorge Pérez, Juan Reutter, and Juan Pablo Silva. The logical expressiveness of graph neural networks. ICLR, 2020 [2] Jin-yi Cai, Martin Fürer, Neil Immerman, An Optimal Lower Bound on the Number of Variables for Graph Identification. FOCS 1989: 612-617. [3] Xingyue Huang, Miguel A. Romero Orth, Ismail Ilkan Ceylan, Pablo Barceló, A Theory of Link Prediction via Relational Weisfeiler-Leman. CoRR abs/2302.02209 (2023). [4] Michael Sejr Schlichtkrull, Thomas N. Kipf, Peter Bloem, Rianne van den Berg, Ivan Titov, Max Welling: Modeling Relational Data with Graph Convolutional Networks. ESWC 2018.

Questions

1. There is no scalability comparison between the proposed method and baselines. Please add it.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Limitation is not addressed.

Summary

This paper proposes the R^2-GNN that captures the logic features in the graph. The proposed methodology largely resembles the previous work of ACR-GNN, where a GNN variant is proposed to capture the logic feature for graphs with no relation types. The proposed R^2-GNN extends the previous method by considering the graphs that involve a set of unary and binary predicates and potentially temporal information. To do so, the authors apply the same readout function to the base R-GNN model and show that it can capture the full logic classifiers with an additional graph transformation F. Also an temporal variant R^2-TGNN is proposed to handle the temporal graphs.

Strengths

The paper is overall easy to read, however, I didn't check all the proof in detail.

Weaknesses

## Novelty As mentioned in summary, the authors follow the same method in ACR-GNN in modifying R-GNN into R^2-GNN to enable the model to capture the FOC_2 logic family. It seems to me the main methodological differences are the additional graph transformation F and the discussion on temporal graphs. - The former helps the model to distinguish nodes with different relations, while it seems to work empirically, it is more or less an incremental modification to the framework. - For the temporal graphs, the author propose to "temporalize" the graph by mapping the graph snapshots into one single static graph and process it with a slightly modified model, namely R^2-TGNN. That said, this paper is not very novel methodology-wise. ## Quality The definition of a multi-relational graph is more or less redundant and confusing: this is effectively the graph with both unary and binary predicates, and many existing KGs already contain facts of both these types. One can refer to it as a real-world graph or just an ordinary graph instead of as "multi-relational". Some model and design choices lack justification: - It is unclear why the authors pick the R-GNN as the base model, provided that: (1) most existing GNNs can handle graphs with different relations and (2) the proposed modifications such as readout function and transformation F and H are orthogonal to the choice of the model. The authors may want to consider evaluating their method on different base models empirically to better demonstrate the difference - It is also unclear why the task is limited to node classification only: this leads to a narrow choice of the benchmarks in the experiments (L329-L330) and leaves some popular datasets untested such as GDELT and ICEWS. I'm also concerned about the real-world datasets experiments. The authors show the R^2-(T)RGNN achieves better performance than other GNN variants. While empirically this does have some merits, it does not provide any insights into the proposed model. Since the model focuses on learning the latent FOC_2 classifiers. One should first inspect the datasets and see what are the learnable FOC_2 rules in there. And if the model performs better, the authors should take the opportunity and analyze if this indeed resulted from the extra capabilities introduced to the model. ## Clarity The paper is overall easy to read, however, I didn't check all the proof in detail. ## Significance Apart from the lack of novelty, this paper also seems to lack significance. The FOC_2 logic is a small subset of first-order logic. Being able to capture these classifiers leads to minor benefits to the GNN literature: it does not provide sufficient explanation of the GNN model, nor does it tackle the difficult logical reasoning problems on graphs such as multi-hop reasoning, induction, and so on. In the experiments, the model is tested on only synthetic and three real-world datasets, and with no comparison to the SOTA models, so the empirical significance is also unclear.

Questions

See above

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

2 fair

Contribution

1 poor

Limitations

Yes

Please respond to the rebuttal

Dear reviewers Please read the rebuttals and respond, even if only briefly. I would specifically expect a detailed response from the more negative reviewers who are in the minority here. Thanks

Thank you and brief follow-up

Thanks for your clarifications. Could you provide any comment related to my comment related to the equivalent interpretation of R²-GNN as R-GNN with an augmentation? I have copied it below: > Not a question as much as a comment - the R²-GNN architecture is equivalent to the R-GNN architecture if we first augment the graph to have a new binary relation which forms a complete graph on all nodes. Given that the work already includes a graph augmentation step, an alternative presentation could potentially avoid the R²-GNN architecture altogether, and simply express the entire thing as R-GNN on an augmented graph. I'm not sure it is clearer to take this approach, but it might be worth noting. Is my interpretation here correct? If so, would it have been perhaps simpler to present the entire thing as simply R-GNN with a graph augmentation? As some feedback on the comments from (at this time) the remaining reviewer providing a negative score: Reviewer auoR has some concerns with the term "multi-relational" graph, and proposes to use "real world graph" or just "graph". The term "multi-relational" is quite standard in the literature, and I see no particular issue with the author's definition of it. Using "real world graph" would be highly ambiguous, and using just "graph" would be incorrect, as a graph is a tuple $G = (V, E)$ where $E \subseteq V \times V$ but a multi-relational graph is a triple $G' = (V, E, R)$ where $E \subseteq V \times R \times V$. The authors here have chosen to split out binary and unary predicates, which is fairly standard and can be shown to be isomorphic to the multi-relational structure presented here.

Thanks for the response

After reading the response and other reviews, I decided to keep my score and I'm still inclined to reject, as it does not address my concerns about the novelty and the significance of this work. **Technical novelty**. I'm a bit frustrated that the authors did not attempt to defend the technical novelty issues I raised in the initial comment, and my concerns that (1) *"the proposed method largely resembles the previous work of ACR-GNN"* and (2) *"the additional graph transformation F is more or less an incremental modification"* remain unaddressed. Still, I'm happy to be proven wrong in this regard. **R-GNN**. I appreciate the clarification and I strongly recommend authors revise the draft according to make clear that this is for a family of GNN instead of a particular example. **Limited significance**. Apart from the novelty issue, my biggest concern is the limited significance of this work, and the author's response seems to acknowledge it rather than rebut it: - **The authors confirmed that R^2-GNN is limited to the node classification task only**. This significantly limits the scope of the work, as other tasks such as link prediction are critical to most of the graph-based systems and applications such as graph reasoning, entity resolution, QA, and recommendation. - **The authors also acknowledge that real-world graph datasets are difficult to be modeled with FOL classifiers**. While I fully understand the difficulty here, this limitation inevitably undermines the empirical evaluation and consequentially the claims that are supposed to be supported. As far as I'm concerned, the aim of this work is to enable GNN learning FOC2 classifiers rather than pursuing SOTA performance, then if real-world datasets are inherently bad for showcasing this, then why would one want to include them in the first place? Showing results on real-world datasets that are not SOTA distracts the audience and does not add to the main claim of the work. - **Finally, the authors acknowledge that even though FOC2 is already a fragment of FOL, R^2-GNN still cannot fully capture it**. I appreciate the authors' motivation and understand this is a challenging task, but the fact is that such a framework is indeed far from being practically useful and has yet to advance the explainability of GNN by a significant degree.