Hierarchical Multi-Task Natural Language Understanding for Cross-domain Conversational AI: HERMIT NLU

We present a new neural architecture for wide-coverage Natural Language\nUnderstanding in Spoken Dialogue Systems. We develop a hierarchical multi-task\narchitecture, which delivers a multi-layer representation of sentence meaning\n(i.e., Dialogue Acts and Frame-like structures). The architecture is a\nhierarchy of self-attention mechanisms and BiLSTM encoders followed by CRF\ntagging layers. We describe a variety of experiments, showing that our approach\nobtains promising results on a dataset annotated with Dialogue Acts and Frame\nSemantics. Moreover, we demonstrate its applicability to a different, publicly\navailable NLU dataset annotated with domain-specific intents and corresponding\nsemantic roles, providing overall performance higher than state-of-the-art\ntools such as RASA, Dialogflow, LUIS, and Watson. For example, we show an\naverage 4.45% improvement in entity tagging F-score over Rasa, Dialogflow and\nLUIS.\n