FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction

Sequence modeling has demonstrated state-of-the-art performance on natural\nlanguage and document understanding tasks. However, it is challenging to\ncorrectly serialize tokens in form-like documents in practice due to their\nvariety of layout patterns. We propose FormNet, a structure-aware sequence\nmodel to mitigate the suboptimal serialization of forms. First, we design Rich\nAttention that leverages the spatial relationship between tokens in a form for\nmore precise attention score calculation. Second, we construct Super-Tokens for\neach word by embedding representations from their neighboring tokens through\ngraph convolutions. FormNet therefore explicitly recovers local syntactic\ninformation that may have been lost during serialization. In experiments,\nFormNet outperforms existing methods with a more compact model size and less\npre-training data, establishing new state-of-the-art performance on CORD, FUNSD\nand Payment benchmarks.\n