A Unified Framework for Rank-based Evaluation Metrics for Link Prediction in Knowledge Graphs

The link prediction task on knowledge graphs without explicit negative\ntriples in the training data motivates the usage of rank-based metrics. Here,\nwe review existing rank-based metrics and propose desiderata for improved\nmetrics to address lack of interpretability and comparability of existing\nmetrics to datasets of different sizes and properties. We introduce a simple\ntheoretical framework for rank-based metrics upon which we investigate two\navenues for improvements to existing metrics via alternative aggregation\nfunctions and concepts from probability theory. We finally propose several new\nrank-based metrics that are more easily interpreted and compared accompanied by\na demonstration of their usage in a benchmarking of knowledge graph embedding\nmodels.\n