Improving Temporal Link Prediction via Temporal Walk Matrix Projection

Temporal link prediction, aiming at predicting future interactions among entities based on historical interactions, is crucial for a series of real-world applications. Although previous methods have demonstrated the importance of relative encodings for effective temporal link prediction, computational efficiency remains a major concern in constructing these encodings. Moreover, existing relative encodings are usually constructed based on structural connectivity, where temporal information is seldom considered. To address the aforementioned issues, we first analyze existing relative encodings and unify them as a function of temporal walk matrices. This unification establishes a connection between relative encodings and temporal walk matrices, providing a more principled way for analyzing and designing relative encodings. Based on this analysis, we propose a new temporal graph neural network called TPNet, which introduces a temporal walk matrix that incorporates the time decay effect to simultaneously consider both temporal and structural information. Moreover, TPNet designs a random feature propagation mechanism with theoretical guarantees to implicitly maintain the temporal walk matrices, which improves the computation and storage efficiency. Experimental results on 13 benchmark datasets verify the effectiveness and efficiency of TPNet, where TPNet outperforms other baselines on most datasets and achieves a maximum speedup of $33.3 \times$ compared to the SOTA baseline. Our code can be found at \url{https://github.com/lxd99/TPNet}.

Paper

References (33)

Scroll for more · 21 remaining

Similar papers

Peer review

Reviewer YNEN7/10 · confidence 2/52024-07-10

Summary

The paper introduces a framework for analysis of relative encodings as a function of random walk matrices, and a new model for temporal link prediction. The new model offers SOTA performance on multiple link prediction datasets, and achieves this performance more efficiently than current best models. The design is of the model is explained in detail.

Strengths

This is an overall good paper which proposes and evaluates a new SOTA approach for temporal link prediction. It also introduces an original framework for unifying a range of existing methods. 1. The provided code is sufficiently commented, and detailed instructions for setup are provided, along with utility functions to start the training process. 2. Extensive evaluation has been done of the proposed model, and SOTA performance across a range of datasets has been demonstrated. 3. Current methods have been posed in the newly proposed framework, which is an interesting analytical contribution. Clearly presented and significant contribution to the field of temporal link prediction.

Weaknesses

1. The limitations of the approach are only briefly discussed in the appendix. 2. Although the code is well presented, it did not run out of the box for me. Some effort was required to install additional dependencies not listed in the requirements.txt, and to fix a runtime error in the sampling algorithm of the DataLoader. Minor points / text linting suggestions: Line 81: What is a "unified formation", did you mean "formulation"? Line 94: Style "all the interactions happen" -> "all the interactions that happen" Line 128: Doesn't A^(i) aggregate the i-step temporal walks, instead of k-step? Typo? Line 187: "The value of score function.." -> "The value of the score function". Line 202: What is d? The node degree or dimensionality? Please define. Also in line 2, Algorithm 1. Line 203: Typo enumerating the H vectors. The index 1 is repeated, whereas it should be followed by 2. Line 205: Style "Pseudocode code" -> "Pseudocode" Line 211: Having the integer "i" in the exponent of e could be misleading for a reader who is skimming to get an overview of the paper. This letter is usually reserved as the imaginary unit when exponentiating e, so ideally pick another letter as the index. Line 261: "and obtained" -> "and obtain"

Questions

1. Line 265: Why does ReLU reduce estimation error? Is this an empirical result, or an architectural consideration specific to your data? Please clarify. 2. Line 43: The second argument sounds a bit vague. Could specific examples be mentioned for methods that use and don't use temporal information in constructing their embeddings? 3. Minor point on code: distutils has been removed in python 3.12 and above. To run the code I've had to install the following dependencies in addition to the ones listed: setuptools, numba, sklearn. I've had to fix the following issue. ```{python} set(random.sample(test_node_set, int(0.1 * num_total_unique_node_ids))) ``` Had to be changed to: ```{python} set(random.sample(sorted(test_node_set), int(0.1 * num_total_unique_node_ids))) ``` This happened installing your code in a new python 3.12 environment.

Rating

7

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

The limitations are briefly discussed in the appendix.

Reviewer SMMz6/10 · confidence 1/52024-07-11

Summary

Based on the analysis of traditional methods, this paper proposes a unified framework for relative encoding and introduces a new temporal neural network, TPNet. This model not only addresses the high time complexity issues of traditional methods but also enhances relative encoding by incorporating factors such as time decay effects.

Strengths

1、The authors propose a unified framework for relative encoding, treating them as a function of temporal random walk matrices. 2、The temporal random walk matrix not only takes into account temporal and structural information but also incorporates the effects of time decay. 3、To reduce computational complexity, the authors proposed an approximation scheme for the temporal random walk matrix, detailed in Algorithm 1. 4、The authors conducted thorough axiomatic proofs, providing solid theoretical support for the effectiveness of the TPNet network

Weaknesses

1、If the authors could provide a comparison of memory usage, it would make the model more convincing. 2、This paper does not provide an analysis of the impact of different functions g on relative encoding. 3、The author did not compare with the latest models (published in AAAI 2024 or WWW 2024) in Table 1.

Questions

please see weaknesses

Rating

6

Confidence

1

Soundness

3

Presentation

3

Contribution

3

Limitations

n/a

Reviewer vApy8/10 · confidence 2/52024-07-11

Summary

The article investigates the application of relative encodings in the task of link prediction over temporal networks. Initially, the authors formally unify previously used relative encodings, such as DyGFormer, PINT, NAT, and CAWN, within a unique framework. Subsequently, they propose a novel model, TPNet, and conduct comparative evaluations using several datasets and competitors.

Strengths

The provided framework offers a clearer and broader perspective on existing approaches based on temporal walks. The source code is well-commented and likely to be highly usable. The experimental setup investigates various competitors, datasets, and negative sampling techniques.

Weaknesses

A fundamental concept in the article is "relative encoding." However, the intuition behind this concept is not introduced until line 119. I recommend presenting the meaning of "relative encoding" earlier in the article for better clarity and understanding.

Questions

If I understand correctly, the authors introduce a weight decay mechanism to exponentially decrease the weight of older interactions. While this decision is well-motivated by previous studies, it is well known that the recurrence of interactions is fundamentally important in social networks (e.g., sociopatterns.org). I am curious whether the temporal periodicity of interactions is maintained with this approach. Could the authors discuss this aspect in more detail?

Rating

8

Confidence

2

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors discussed the limitations of the model in the appendix.

Reviewer vApy2024-08-08

Thank you for the detailed response. I’ve read it and you’ve clarified my doubt, so I’ve increased my score.

Reviewer SMMz2024-08-11

Response

Thanks the authors for the rebuttal, and I have no more concerns.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC