KARL-Trans-NER: Knowledge Aware Representation Learning for Named Entity Recognition using Transformers

The inception of modeling contextual information using models such as BERT,\nELMo, and Flair has significantly improved representation learning for words.\nIt has also given SOTA results in almost every NLP task - Machine Translation,\nText Summarization and Named Entity Recognition, to name a few. In this work,\nin addition to using these dominant context-aware representations, we propose a\nKnowledge Aware Representation Learning (KARL) Network for Named Entity\nRecognition (NER). We discuss the challenges of using existing methods in\nincorporating world knowledge for NER and show how our proposed methods could\nbe leveraged to overcome those challenges. KARL is based on a Transformer\nEncoder that utilizes large knowledge bases represented as fact triplets,\nconverts them to a graph context, and extracts essential entity information\nresiding inside to generate contextualized triplet representation for feature\naugmentation. Experimental results show that the augmentation done using KARL\ncan considerably boost the performance of our NER system and achieve\nsignificantly better results than existing approaches in the literature on\nthree publicly available NER datasets, namely CoNLL 2003, CoNLL++, and\nOntoNotes v5. We also observe better generalization and application to a\nreal-world setting from KARL on unseen entities.\n