Synthetic Question Value Estimation for Domain Adaptation of Question Answering

Synthesizing QA pairs with a question generator (QG) on the target domain has\nbecome a popular approach for domain adaptation of question answering (QA)\nmodels. Since synthetic questions are often noisy in practice, existing work\nadapts scores from a pretrained QA (or QG) model as criteria to select\nhigh-quality questions. However, these scores do not directly serve the\nultimate goal of improving QA performance on the target domain. In this paper,\nwe introduce a novel idea of training a question value estimator (QVE) that\ndirectly estimates the usefulness of synthetic questions for improving the\ntarget-domain QA performance. By conducting comprehensive experiments, we show\nthat the synthetic questions selected by QVE can help achieve better\ntarget-domain QA performance, in comparison with existing techniques. We\nadditionally show that by using such questions and only around 15% of the human\nannotations on the target domain, we can achieve comparable performance to the\nfully-supervised baselines.\n