Summary
1. The paper introduces Neural Probabilistic Logic Learning (NPLL), a rule-based method for knowledge graph reasoning. NPLL represents knowledge using a Markov Logic Network (MLN), enhancing the expressiveness of embedding networks. Through variational inference, NPLL accurately infers unknown facts and introduces a scoring module to improve reasoning accuracy in knowledge graphs.
2. The authors conduct extensive experiments across various benchmark datasets, including YAGO3-10, YAGO37, Codex-L, WN18RR and FB15k-237. The results demonstrate that NPLL achieves superior reasoning performance, surpassing other methods on large-scale and domain-specific datasets. This validation highlights NPLL’s effectiveness in complex knowledge graph reasoning tasks.
Strengths
1. The methodological improvement introduced in this paper that leverages an embedding-based scoring module is straightforward. However, this simplicity contributes to the model’s robustness and ease of implementation, and the results achieved are notably impressive.
2. The experimental evaluation is thorough, covering a wide range of benchmark knowledge graphs. This comprehensive testing approach not only underscores the model’s versatility but also consistently demonstrates superior performance across diverse datasets, reinforcing the robustness and broad applicability of the proposed method.
Weaknesses
## 1 Novelty Issue
1. The proposed methodology closely resembles the approach used in ExpressGNN [1], with the primary difference being the addition of a scoring module on factual triples $(e_h, l, e_t)$. It remains unclear what further distinctions, if any, exist between this model and ExpressGNN, raising concerns regarding the novelty of this contribution.
## 2 Insufficient and Unjustified Experimentation
1. Two model variants, NPLL-basic and NPLL-GNN, are proposed, with reasoning results provided for each (Table 2). However, no explanation is given for the significantly lower accuracy of NPLL-GNN compared to NPLL-basic, leaving questions about model performance unaddressed.
2. The evaluation of data efficiency (Table 4) is limited to the FB15k-237 dataset, replicating results that have already been demonstrated in ExpressGNN. Since this work (NPLL) and ExpressGNN [1] follow the same framework (MLN), results on additional datasets would help clarify the model’s data efficiency. Additionally, the impact of varying data sizes on NPLL-basic performance is minimal (Table 4), but this observation is neither analyzed nor discussed.
3. In the Related Work section, the authors claim that compared to embedding-based methods, NPLL enhances both interpretability and reasoning quality. However, no experimental evidence is provided to substantiate this claim.
## 3 Representation
1. Improper use of mathematical symbols: The mathematical formulations are often imprecise, with symbols inconsistently defined or unclear. For example, the definitions of “fact” in the preliminary section are ambiguous, and the MLN setup and representation of unknown facts are incomplete. Including concrete examples would improve clarity and reader comprehension. In the Model section, certain notations (e.g., $u_g$, $u_k$ in Equation 8) are confusing and inadequately defined, making this section challenging to follow.
2. The text and curves in the all figures, especially Figure 2 and 3, are difficult to read due to their small size, limiting accessibility to critical information.
3. Significant details are missing in the methods section, such as specifics on model training and the E-step and M-step processes. This lack of detail, particularly compared to ExpressGNN, further underscores the concerns about novelty in this work.
[1] Zhang, Y.; Chen, X.; Yang, Y.; Ramamurthy, A.; Li, B.; Qi, Y.; Song, L. Efficient Probabilistic Logic Reasoning with Graph Neural Networks. arXiv February 4, 2020. https://doi.org/10.48550/arXiv.2001.11850.
Questions
1. Novelty: Beyond the addition of the scoring module, what are the other key differences between this approach and ExpressGNN? It would be helpful if the authors could elaborate on any unique elements or improvements this model brings, especially regarding interpretability, efficiency, or theoretical grounding.
2. Impact of different data sizes on model performance: The experiments with varying data sizes in the FB15k-237 dataset reveal minimal impact on model performance, but the reasoning behind this result is not addressed. Could the authors provide an analysis of why this might be the case? It would also be valuable if additional insights could be given on the robustness of the model in data-scarce environments or if alternative metrics could show nuanced performance variations.
3. Training time comparisons: Since NPLL utilizes rules derived from a pre-trained Neural-LP model on specific datasets, I suggest that the reported training time should also include the time required to train the Neural-LP model initially (Table 6).