Multiple Run Ensemble Learning with Low-Dimensional Knowledge Graph Embeddings

Among the top approaches of recent years, link prediction using knowledge\ngraph embedding (KGE) models has gained significant attention for knowledge\ngraph completion. Various embedding models have been proposed so far, among\nwhich, some recent KGE models obtain state-of-the-art performance on link\nprediction tasks by using embeddings with a high dimension (e.g. 1000) which\naccelerate the costs of training and evaluation considering the large scale of\nKGs. In this paper, we propose a simple but effective performance boosting\nstrategy for KGE models by using multiple low dimensions in different\nrepetition rounds of the same model. For example, instead of training a model\none time with a large embedding size of 1200, we repeat the training of the\nmodel 6 times in parallel with an embedding size of 200 and then combine the 6\nseparate models for testing while the overall numbers of adjustable parameters\nare same (6*200=1200) and the total memory footprint remains the same. We show\nthat our approach enables different models to better cope with their\nexpressiveness issues on modeling various graph patterns such as symmetric,\n1-n, n-1 and n-n. In order to justify our findings, we conduct experiments on\nvarious KGE models. Experimental results on standard benchmark datasets, namely\nFB15K, FB15K-237 and WN18RR, show that multiple low-dimensional models of the\nsame kind outperform the corresponding single high-dimensional models on link\nprediction in a certain range and have advantages in training efficiency by\nusing parallel training while the overall numbers of adjustable parameters are\nsame.\n