Do Response Selection Models Really Know What's Next? Utterance Manipulation Strategies for Multi-turn Response Selection

In this paper, we study the task of selecting the optimal response given a\nuser and system utterance history in retrieval-based multi-turn dialog systems.\nRecently, pre-trained language models (e.g., BERT, RoBERTa, and ELECTRA) showed\nsignificant improvements in various natural language processing tasks. This and\nsimilar response selection tasks can also be solved using such language models\nby formulating the tasks as dialog--response binary classification tasks.\nAlthough existing works using this approach successfully obtained\nstate-of-the-art results, we observe that language models trained in this\nmanner tend to make predictions based on the relatedness of history and\ncandidates, ignoring the sequential nature of multi-turn dialog systems. This\nsuggests that the response selection task alone is insufficient for learning\ntemporal dependencies between utterances. To this end, we propose utterance\nmanipulation strategies (UMS) to address this problem. Specifically, UMS\nconsist of several strategies (i.e., insertion, deletion, and search), which\naid the response selection model towards maintaining dialog coherence. Further,\nUMS are self-supervised methods that do not require additional annotation and\nthus can be easily incorporated into existing approaches. Extensive evaluation\nacross multiple languages and models shows that UMS are highly effective in\nteaching dialog consistency, which leads to models pushing the state-of-the-art\nwith significant margins on multiple public benchmark datasets.\n