Transformer-Based Approach for Joint Handwriting and Named Entity Recognition in Historical documents

The extraction of relevant information carried out by named entities in\nhandwriting documents is still a challenging task. Unlike traditional\ninformation extraction approaches that usually face text transcription and\nnamed entity recognition as separate subsequent tasks, we propose in this paper\nan end-to-end transformer-based approach to jointly perform these two tasks.\nThe proposed approach operates at the paragraph level, which brings two main\nbenefits. First, it allows the model to avoid unrecoverable early errors due to\nline segmentation. Second, it allows the model to exploit larger bi-dimensional\ncontext information to identify the semantic categories, reaching a higher\nfinal prediction accuracy. We also explore different training scenarios to show\ntheir effect on the performance and we demonstrate that a two-stage learning\nstrategy can make the model reach a higher final prediction accuracy. As far as\nwe know, this work presents the first approach that adopts the transformer\nnetworks for named entity recognition in handwritten documents. We achieve the\nnew state-of-the-art performance in the ICDAR 2017 Information Extraction\ncompetition using the Esposalles database, for the complete task, even though\nthe proposed technique does not use any dictionaries, language modeling, or\npost-processing.\n