The STEM-ECR Dataset: Grounding Scientific Entity References in STEM Scholarly Content to Authoritative Encyclopedic and Lexicographic Sources

We introduce the STEM (Science, Technology, Engineering, and Medicine)\nDataset for Scientific Entity Extraction, Classification, and Resolution,\nversion 1.0 (STEM-ECR v1.0). The STEM-ECR v1.0 dataset has been developed to\nprovide a benchmark for the evaluation of scientific entity extraction,\nclassification, and resolution tasks in a domain-independent fashion. It\ncomprises abstracts in 10 STEM disciplines that were found to be the most\nprolific ones on a major publishing platform. We describe the creation of such\na multidisciplinary corpus and highlight the obtained findings in terms of the\nfollowing features: 1) a generic conceptual formalism for scientific entities\nin a multidisciplinary scientific context; 2) the feasibility of the\ndomain-independent human annotation of scientific entities under such a generic\nformalism; 3) a performance benchmark obtainable for automatic extraction of\nmultidisciplinary scientific entities using BERT-based neural models; 4) a\ndelineated 3-step entity resolution procedure for human annotation of the\nscientific entities via encyclopedic entity linking and lexicographic word\nsense disambiguation; and 5) human evaluations of Babelfy returned encyclopedic\nlinks and lexicographic senses for our entities. Our findings cumulatively\nindicate that human annotation and automatic learning of multidisciplinary\nscientific concepts as well as their semantic disambiguation in a wide-ranging\nsetting as STEM is reasonable.\n