Incremental Processing in the Age of Non-Incremental Encoders: An Empirical Assessment of Bidirectional Models for Incremental NLU

While humans process language incrementally, the best language encoders\ncurrently used in NLP do not. Both bidirectional LSTMs and Transformers assume\nthat the sequence that is to be encoded is available in full, to be processed\neither forwards and backwards (BiLSTMs) or as a whole (Transformers). We\ninvestigate how they behave under incremental interfaces, when partial output\nmust be provided based on partial input seen up to a certain time step, which\nmay happen in interactive systems. We test five models on various NLU datasets\nand compare their performance using three incremental evaluation metrics. The\nresults support the possibility of using bidirectional encoders in incremental\nmode while retaining most of their non-incremental quality. The\n"omni-directional" BERT model, which achieves better non-incremental\nperformance, is impacted more by the incremental access. This can be alleviated\nby adapting the training regime (truncated training), or the testing procedure,\nby delaying the output until some right context is available or by\nincorporating hypothetical right contexts generated by a language model like\nGPT-2.\n