Ensemble Learning-Based Approach for Improving Generalization Capability of Machine Reading Comprehension Systems

Machine Reading Comprehension (MRC) is an active field in natural language\nprocessing with many successful developed models in recent years. Despite their\nhigh in-distribution accuracy, these models suffer from two issues: high\ntraining cost and low out-of-distribution accuracy. Even though some approaches\nhave been presented to tackle the generalization problem, they have high,\nintolerable training costs. In this paper, we investigate the effect of\nensemble learning approach to improve generalization of MRC systems without\nretraining a big model. After separately training the base models with\ndifferent structures on different datasets, they are ensembled using weighting\nand stacking approaches in probabilistic and non-probabilistic settings. Three\nconfigurations are investigated including heterogeneous, homogeneous, and\nhybrid on eight datasets and six state-of-the-art models. We identify the\nimportant factors in the effectiveness of ensemble methods. Also, we compare\nthe robustness of ensemble and fine-tuned models against data distribution\nshifts. The experimental results show the effectiveness and robustness of the\nensemble approach in improving the out-of-distribution accuracy of MRC systems,\nespecially when the base models are similar in accuracies.\n