CopyNext: Explicit Span Copying and Alignment in Sequence to Sequence Models

Copy mechanisms are employed in sequence to sequence models (seq2seq) to\ngenerate reproductions of words from the input to the output. These frameworks,\noperating at the lexical type level, fail to provide an explicit alignment that\nrecords where each token was copied from. Further, they require contiguous\ntoken sequences from the input (spans) to be copied individually. We present a\nmodel with an explicit token-level copy operation and extend it to copying\nentire spans. Our model provides hard alignments between spans in the input and\noutput, allowing for nontraditional applications of seq2seq, like information\nextraction. We demonstrate the approach on Nested Named Entity Recognition,\nachieving near state-of-the-art accuracy with an order of magnitude increase in\ndecoding speed.\n