Human-like informative conversations: Better acknowledgements using conditional mutual information

This work aims to build a dialogue agent that can weave new factual content\ninto conversations as naturally as humans. We draw insights from linguistic\nprinciples of conversational analysis and annotate human-human conversations\nfrom the Switchboard Dialog Act Corpus to examine humans strategies for\nacknowledgement, transition, detail selection and presentation. When current\nchatbots (explicitly provided with new factual content) introduce facts into a\nconversation, their generated responses do not acknowledge the prior turns.\nThis is because models trained with two contexts - new factual content and\nconversational history - generate responses that are non-specific w.r.t. one of\nthe contexts, typically the conversational history. We show that specificity\nw.r.t. conversational history is better captured by Pointwise Conditional\nMutual Information ($\\text{pcmi}_h$) than by the established use of Pointwise\nMutual Information ($\\text{pmi}$). Our proposed method, Fused-PCMI, trades off\n$\\text{pmi}$ for $\\text{pcmi}_h$ and is preferred by humans for overall quality\nover the Max-PMI baseline 60% of the time. Human evaluators also judge\nresponses with higher $\\text{pcmi}_h$ better at acknowledgement 74% of the\ntime. The results demonstrate that systems mimicking human conversational\ntraits (in this case acknowledgement) improve overall quality and more broadly\nillustrate the utility of linguistic principles in improving dialogue agents.\n