Simulating Bandit Learning from User Feedback for Extractive Question Answering

We study learning from user feedback for extractive question answering by\nsimulating feedback using supervised data. We cast the problem as contextual\nbandit learning, and analyze the characteristics of several learning scenarios\nwith focus on reducing data annotation. We show that systems initially trained\non a small number of examples can dramatically improve given feedback from\nusers on model-predicted answers, and that one can use existing datasets to\ndeploy systems in new domains without any annotation, but instead improving the\nsystem on-the-fly via user feedback.\n