DECRL: A Deep Evolutionary Clustering Jointed Temporal Knowledge Graph Representation Learning Approach

Summary

This paper proposed an interesting model to integrate high-order clusters into the TKG representation learning. A cluster-aware unsupervised alignment mechanism is introduced to ensure the alignment of soft overlapping clusters across timestamps. An implicit correlation encoder is also proposed to capture latent correlations between clusters. Experimental results show the effectiveness of the proposed model.

Strengths

1. Integrating high-order structure in TKG representation learning is interesting. 2. The proposed model seems technically reasonable.

Weaknesses

1. Entity graph and cluster graph are two important concepts for understanding the proposed method. However, they are never mentioned in the introduction and their definitions in Section 3 are very brief, making the intuition behind the proposed method unclear to me. 2. The authors use a relatively simple task future relation prediction to evaluate the quality of the learned TKG representations. However, the future entity prediction (i.e., [s, r, ?, t]) is more challenging due to the large size of the entity set and the evolution of entity semantics. Intuitively, modeling high-order correlation among entities can also be beneficial for entity prediction. It is better to also evaluate the performance of the learned representations on this task. 3. The authors cluster entities only based on their representations. However, in knowledge graphs, entities are connected via various relations and different relations may indicate different correlations. For example, the relation "leave from" means an athlete will not interact with a club, but "transfer to" means they will have more interactions. How can the proposed method handle such correlations brought by relation semantics? 4. Only two benchmark datasets are used for evaluation, making the experimental results unconvincing. ICEWS14C and ICEWS18C datasets are actually subsets of ICEWS14 and ICEWS18, and they are both derived from the same resource (i.e., ICEWS). More TKGs especially from different resources such as GDELT, YAGO, and Wikidata should also be used for evaluation.

Questions

See weakness

Rating

5

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

Yes

Summary

The paper addresses Temporal Knowledge Graph (TKG) representation learning, which aims to embed temporally evolving entities and relations into a continuous low-dimensional vector space. Existing methods struggle to capture the temporal evolution of high-order correlations in TKGs. The authors propose a novel approach called Deep Evolutionary Clustering jointed temporal knowledge graph Representation Learning (DECRL). DECRL is the first to integrate deep evolutionary clustering with TKG representation learning to capture the temporal evolution of high-order correlations.

Strengths

1 The author clearly describes the motivation for the paper and the methods used. 2 The experimental results demonstrate that DECRL achieves state-of-the-art (SOTA) performance.

Weaknesses

1 Most event prediction models are often tested on ICEWS05-15 and GDELT datasets. The author does not give experimental results on these two datasets.

Questions

1 When clustering nodes using fuzzy clustering, have the authors considered incorporating domain knowledge for node classification? Additionally, have they compared their approach with other node classification methods? 2 Some nodes may not have clear relationships with other nodes. Will clustering them together affect the behavior prediction of these nodes?

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

Please refer to the weaknesses.

Summary

This paper studies the temporal knowledge graph representation learning task and proposes a temporal evolution-aware framework DECRL. By assigning different entities to distinct clusters at each timestamp and modeling the evolution and shifts of these clusters, cluster-aware information is explicitly incorporated into both entity and relation embeddings. This enables better temporal intelligence for making precise predictions. Extensive experiments demonstrate promising results on several benchmarks.

Strengths

* This paper is well-written and easy to follow. * Technical details are well presented and, to some extent, clearly explained. * The experiments are extensive, covering various benchmarks, comparing diverse baselines, and demonstrating effectiveness through both qualitative and quantitative analyses.

Weaknesses

* The motivation for introducing a cluster graph at each timestamp to capture temporal shift information at the clustering scale remains unclear. The authors should further elaborate on this overall motivation. Additionally, the authors mentioned that “some researchers have leveraged derived structures, e.g., communities, entity groups, and hypergraphs, to model high-order correlations among …” (Lines 29-30). In my understanding, the modeling of communities and groups is similar to this paper’s clustering, and hypergraphs can be regarded as another approach for group modeling since a hyperedge connects different nodes. The differences in motivation and technique between DECRL and these methods should be briefly discussed. * The methodology section of DECRL incorporates many minor techniques without corresponding ablation studies to demonstrate their effectiveness or detailed explanations of the rationale. For example, the effectiveness of temporal attentive pooling (Section 4.5) lacks ablation study evidence. Besides, detailed operations for each variant in the ablation study should be provided. * For Figure 2 in the case study, additional textual explanations are needed, such as clarifying what the red dots represent. Additionally, the authors should consider including baseline methods in the figure, rather than only showing DECRL and its variants.

Questions

Please see weaknesses

Rating

6

Confidence

3

Soundness

2

Presentation

3

Contribution

2

Limitations

NA

Summary

The paper proposed a deep evolutionary clustering method for TKGE to capture the temporal evolution of high-order correlation in TKGs. A cluster-aware unsupervised alignment mechanism is introduced to ensure the precise one-to-one alignment of soft overlapping clusters across timestamps. Extensive experiments on four real-world datasets demonstrate the remarkable improvement of the proposed method compared to other baselines.

Strengths

1. The experimental results are remarkable for the improvement of the ICEWS datasets. 2. The paper is well-organized and clearly presented.

Weaknesses

1. The paper proposed to capture the temporal evolution of high-order correlation, while no intuitional case study is provided. 2. Though extensive experiments are conducted, two other main TKGE datasets are omitted, namely Wikidata and YAGO.

Questions

1. Will the proposed method still outperform other baselines by large margins on Wikidata and YAGO datasets? The type of time information in these two datasets is different from ICEWS datasets.

Rating

7

Confidence

4

Soundness

4

Presentation

4

Contribution

3

Limitations

The proposed method mainly focuses on improving the accuracy of link prediction for TKGs, while other aspects such as efficiency and transparency are not discussed.