A Neural Model for Joint Document and Snippet Ranking in Question Answering for Large Document Collections

Question answering (QA) systems for large document collections typically use\npipelines that (i) retrieve possibly relevant documents, (ii) re-rank them,\n(iii) rank paragraphs or other snippets of the top-ranked documents, and (iv)\nselect spans of the top-ranked snippets as exact answers. Pipelines are\nconceptually simple, but errors propagate from one component to the next,\nwithout later components being able to revise earlier decisions. We present an\narchitecture for joint document and snippet ranking, the two middle stages,\nwhich leverages the intuition that relevant documents have good snippets and\ngood snippets come from relevant documents. The architecture is general and can\nbe used with any neural text relevance ranker. We experiment with two main\ninstantiations of the architecture, based on POSIT-DRMM (PDRMM) and a\nBERT-based ranker. Experiments on biomedical data from BIOASQ show that our\njoint models vastly outperform the pipelines in snippet retrieval, the main\ngoal for QA, with fewer trainable parameters, also remaining competitive in\ndocument retrieval. Furthermore, our joint PDRMM-based model is competitive\nwith BERT-based models, despite using orders of magnitude fewer parameters.\nThese claims are also supported by human evaluation on two test batches of\nBIOASQ. To test our key findings on another dataset, we modified the Natural\nQuestions dataset so that it can also be used for document and snippet\nretrieval. Our joint PDRMM-based model again outperforms the corresponding\npipeline in snippet retrieval on the modified Natural Questions dataset, even\nthough it performs worse than the pipeline in document retrieval. We make our\ncode and the modified Natural Questions dataset publicly available.\n