Intensity Profile Projection: A Framework for Continuous-Time Representation Learning for Dynamic Networks

Summary

The paper proposes a method to learn low-dimensional continuous-time representations of network nodes, based on the collection of interaction events among them. More precisely, the events are in the form of $(i,j,t)$, where $(i,j)$ is the pair of nodes involved in the interaction event, and $t$ is the occurrence time. The proposed method first estimate the intensity function $\lambda_{i,j}(t)$ of events between each pair of nodes $(i,j)$ at every time instant $t$, then project the intensities of each node at time $t$ onto a learned lower dimensional subspace to obtain a representation. Theoretical results on the recovery error of the representation is provided. Numerical experiments using real data shows the effectiveness of the proposed method.

Strengths

The paper proposes to estimate the representation of nodes using continuous-time events, which seems to be a novel type of data.

Weaknesses

I find the presentation of the paper generally vague and hand-wavy. See the following. 1. The introduction is way too high-level. The authors should be more specific about the problem setting in this paper, for example, why we care about dynamic models, continuous-time event data, low-dimensional representation of nodes etc. 2. The related work is not specific. The authors should use a sentence to summarize the contribution of the mentioned papers and explain the difference from your work. 3. Lemma 1 is not correct. $\widehat U_d$ minimizes the residual sum of squares at $B$ chosen time instants, but not the integrated one. 4. In Section 3, notation part, what is the difference between $\gg$ and $\gtrsim$? Also is the universal constant multiplicative or additive? 5. It's not clear what `$\approx$' means in Section 4.

Questions

.

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

.

Summary

The paper presents a framework called Intensity Profile Projection (IPP) for continuous-time representation learning in dynamic networks. The authors aim to address the challenge of capturing temporal dynamics and evolving relationships in dynamic networks with both high statistical precision and interpretability. The model leverages the concept of intensity profiles, which encode the temporal changes and interactions between nodes in a network. The model provides a uniform error bound for learned node representations and preserves a novel "temporal coherence" property compared to existing baselines. Empirical results on real-world dynamic network datasets demonstrate that IPP outperforms existing methods in various tasks such, highlighting its ability to capture continuous-time representations and uncover temporal patterns in dynamic networks.

Strengths

1. The paper introduces the Intensity Profile Projection (IPP) framework, which offers a unique and innovative approach to continuous-time representation learning for dynamic networks. It introduces the concept of intensity profiles and effectively utilizes them to capture temporal dynamics. 2. Theoretical analysis towards the model shows that the model can achieve high statistical precision and preserve interpretability in terms of ""temporal coherence". 3. The paper is in general easy to follow.

Weaknesses

1. Lack of comparison with state-of-the-art methods: Although the paper claims improved performance over existing methods, it does not provide a comprehensive comparison with some existing continuous models such as GraphODEs[1,2,3,4] which combines neuralODE with GNNs to model network evolution over time. 2. Scalability: The scalability of the IPP framework is not extensively discussed. It would be valuable to address the computational requirements and scalability limitations of the proposed approach, especially when dealing with large-scale dynamic networks. 3. The related work section is too short to provide a comprehensive background of the research topic. [1] Huang, Zijie, Yizhou Sun, and Wei Wang. "Learning continuous system dynamics from irregularly-sampled partial observations." Advances in Neural Information Processing Systems 33 (2020): 16177-16187. [2] Song Wen, Hao Wang, and Dimitris Metaxas. 2022. Social ODE: Multi-agent Trajectory Forecasting with Neural Ordinary Differential Equations. In Computer Vision–ECCV 2022: 17th European Conference. [3]Zijie Huang, Yizhou Sun, and Wei Wang. Coupled graph ode for learning interacting system dynamics. In 401 ACM SIGKDD Conference on Knowledge Discovery and Data Mining, page 705–715, 2021. [4] Zang, Chengxi, and Fei Wang. "Neural dynamics on complex networks." In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 892-902. 2020.

Questions

1. What would be the time complexity of the proposed method? 2. How would the model performance be affected by different network topology?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The authors have discussed the limitations of their work.

Summary

To represent the continuous dynamic network, authors provide the framework based on the intensity profile. First, the intensity between nodes is estimated, which produces the intensity profile. Low dimension reduction via SVD is applied on the intensity, and then each node embedding is obtained by the low dimensional subspace. Author also provide various theoretical analysis about the error bound and the bias-various trade-off. Theoretical analysis as well as empirical analysis on the simulated data demonstrates that the proposed method capture structural preserving and temporally coherent properties. Case study on the real data is conducted to explain the outcome of the proposed framework qualitatively

Strengths

- Simple but powerful method is proposed - Based on the mathematical model, theoretical bound is analyzed and explained. - IPP can capture the behavior of a bifurcating block model.

Weaknesses

- The proposed method is not novel enough. SVD decomposition is a very common technique for the reduction of dimensions, and it often suffers from the long-tailed singular values. - Comparison is too limited. The analysis has been made only for the simulated data with figures. More experiments as well as some qualitative results would be great to have. - SVD decomposition does not prevent producing negative values at the reconstruction. - The proposed projection space is very dependent on the fixed dataset. At least, how to leverage the given embeddings for predictions is not straightforward. Given this, the potential application value is not very clear.

Questions

- Figure numbers are all wrong. - Section 4 is true for any global subspace projection. Also, both properties could be debatable, not necessarily ideal. For instance, when \Labmda_{i}(s) = \Lambda_{i}(t), X_{i}(s) = \alpha * X_{i}(t) could be more ideal, depending on the interactions among the other nodes. - It would be great if authors compare the embedding trajectory for more real data, beyond the specific simulated ones.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

2 fair

Limitations

Often, the meaning of each dimension from the SVD decomposition is not clear. This interpretability is not necessarily required for the representation, but this should be addressed when presenting the case study.

Summary

The authors propose an approach for learning time-varying node embeddings from continuous-time dynamic network data, which consist of a set of instantaneous timestamped relational events between nodes (e.g., messages from one social media user to another). Their proposed approach learns a projection that minimizes reconstruction error of the pairwise intensities between nodes and comes with theoretical guarantees on estimation error. They also show that their approach generates embeddings that both preserve network structure at a given time and is temporally coherent. They demonstrate strong empirical performance on simulated data compared to other dynamic network embeddings. Furthermore, they use their approach to analyze a real network data set on face-to-face interactions of primary school students, which is quite enlightening due to the interpretability of their model. *After rebuttal:* The authors have clarified the one question I had about the meaning of "inductive" in their setting. I continue to strongly support the paper.

Strengths

- Proposed approach learns time-varying node embeddings from continuous-time networks with theoretical guarantees, which is among the first, if not the first, in the literature. - Proposed embeddings can satisfy two good properties of structure preservation and temporal coherence. - Very well written and organized paper that provides highlights of theoretical analysis in the main paper followed by details, including proofs, in the supplementary.

Weaknesses

- There's a large body of related literature on probabilistic generative models for continuous-time networks using point process models such as Hawkes processes that should be discussed. Many of these models are based on stochastic block models or latent space models and are thus also learning node embeddings. See suggested references below. - No quantitative evaluation. This is only a minor weakness in my opinion because I view the main contribution to be theoretical. Typos and minor issues: - Supplementary Section C heading: Visualsation -> Visualisation References: - Arastuie, M., Paul, S., & Xu, K. S. (2020). CHIP: A Hawkes process model for continuous-time networks with scalable and consistent estimation. In Advances in Neural Information Processing Systems 33 (pp. 16983-16996). - Corneli, M., Latouche, P., & Rossi, F. (2018). Multiple change points detection and clustering in dynamic networks. Statistics and Computing, 28(5), 989-1007. doi:10.1007/s11222-017-9775-1 - Huang, Z., Soliman, H., Paul, S., & Xu, K. S. (2022). A mutually exciting latent space Hawkes process model for continuous-time networks. In Proceedings of the 38th Conference on Uncertainty in Artificial Intelligence (Vol. 180, pp. 863-873). - Junuthula, R. R., Haghdan, M., Xu, K. S., & Devabhaktuni, V. K. (2019). The Block Point Process Model for continuous-time event-based dynamic networks. In Proceedings of the World Wide Web Conference (pp. 829-839). - Matias, C., Rebafka, T., & Villers, F. (2018). A semiparametric extension of the stochastic block model for longitudinal networks. Biometrika, 105(3), 665-680. doi:10.1093/biomet/asy016 - Yang, J., Rao, V., & Neville, J. (2017). Decoupling homophily and reciprocity with latent space network models. In Proceedings of the Conference on Uncertainty in Artificial Intelligence.

Questions

1. The authors mention several times that their approach is inductive, allowing one to obtain a node representation profile outside of the training sample. If the task is to obtain the node representation for the future, how would the Intensity Profile Projection approach handle it? Would it require some data from other nodes at that future time?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

Limitations are thoroughly discussed in Section 6. I commend the authors for being very forthcoming with these limitations. I don't view the limitations as weaknesses, because they are mostly limitations that apply to all unsupervised problems.

Thanks for your response.

Thanks for the authors' response and my concern about the scalability issue has been addressed. One questions I still have is that I do not know why GraphODE methods cannot be directly comparable as they are also continuous-time networks as mentioned in your general response. Illustrating the performance comparison among continuous dynamic network models can be beneficial to show how well your model performs.