Investigating Robustness of Dialog Models to Popular Figurative Language Constructs

Humans often employ figurative language use in communication, including\nduring interactions with dialog systems. Thus, it is important for real-world\ndialog systems to be able to handle popular figurative language constructs like\nmetaphor and simile. In this work, we analyze the performance of existing\ndialog models in situations where the input dialog context exhibits use of\nfigurative language. We observe large gaps in handling of figurative language\nwhen evaluating the models on two open domain dialog datasets. When faced with\ndialog contexts consisting of figurative language, some models show very large\ndrops in performance compared to contexts without figurative language. We\nencourage future research in dialog modeling to separately analyze and report\nresults on figurative language in order to better test model capabilities\nrelevant to real-world use. Finally, we propose lightweight solutions to help\nexisting models become more robust to figurative language by simply using an\nexternal resource to translate figurative language to literal (non-figurative)\nforms while preserving the meaning to the best extent possible.\n