While hyper-parameters (HPs) are important for knowledge graph (KG) learning, existing methods fail to search them efficiently. To solve this problem, we first analyze the properties of different HPs and measure the transfer ability from small subgraph to the full graph. Based on the analysis, we propose an efficient two-stage search algorithm KG-Tuner, which efficiently explores HP configurations on small subgraph at the first stage and transfers the top-performed configurations for fine-tuning on the large full graph at the second stage. Experiments show that our method can consistently find better HPs than the baseline algorithms within the same time budget, which achieves 9.1% average relative improvement for four embedding models on the large-scale KGs in open graph benchmark. Our code is released in https://github. com/AutoML-Research/KGTuner. 1
Paper
Full text
Efficient Hyper-parameter Search for Knowledge Graph Embedding
OpenAlex · Advanced Graph Neural Networks · 2022
Abstract
While hyper-parameters (HPs) are important 001 for knowledge graph (KG) embedding, exist- 002 ing methods fail to search them efficiently. To 003 solve this problem, we first analyze the proper- 004 ties of different HPs and quantize the transfer- 005 ability from small subgraph to the large graph. 006 Based on the analysis, we propose an efficient 007 two-stage search algorithm, which efficiently 008 explores HP configurations on small subgraph 009 at the first stage and transfers the top configura- 010 tions for fine-tuning on the large whole graph 011 at the second stage. Experiments show that our 012 method can consistently find better HPs than 013 the baseline algorithms with the same time 014 budget. We achieve 10.8% average relative im- 015 provement for four embedding models on the 016 large-scale KGs in open graph benchmark. 017