Improving Conversational Question Answering Systems after Deployment using Feedback-Weighted Learning

The interaction of conversational systems with users poses an exciting\nopportunity for improving them after deployment, but little evidence has been\nprovided of its feasibility. In most applications, users are not able to\nprovide the correct answer to the system, but they are able to provide binary\n(correct, incorrect) feedback. In this paper we propose feedback-weighted\nlearning based on importance sampling to improve upon an initial supervised\nsystem using binary user feedback. We perform simulated experiments on document\nclassification (for development) and Conversational Question Answering datasets\nlike QuAC and DoQA, where binary user feedback is derived from gold\nannotations. The results show that our method is able to improve over the\ninitial supervised system, getting close to a fully-supervised system that has\naccess to the same labeled examples in in-domain experiments (QuAC), and even\nmatching in out-of-domain experiments (DoQA). Our work opens the prospect to\nexploit interactions with real users and improve conversational systems after\ndeployment.\n