Deep Reinforcement Learning for Dialogue Systems with Dynamic User Goals

Dialogue systems have recently become a widely used system across the world. Some of the functionality offered includes application user interfacing, social conversation, data interaction, and task completion. Most recently, dialogue systems have been developed to autonomously and intelligently interact with users to complete complex tasks in diverse operational spaces. This kind of dialogue system can interact with users to complete tasks such as making a phone call, ordering items online, searching the internet for a question, and more. These systems are typically created by training a machine learning model with example conversational data. One of the existing problems with training these systems is that they require large amounts of realistic user data, which can be challenging to collect and label in large quantities. Our research focuses on modifications to user simulators that "change their mind" mid-episode with the goal of training more robust dialogue agents. We do this by taking an existing dialogue system, modifying its user simulator, and observing quantitative and qualitative effects against a set of goals. With these results we demonstrate benefits, drawbacks, and tangential effects of using various rules and algorithms while recreating goal changing behavior.