Multi-View Document Representation Learning for Open-Domain Dense Retrieval

Dense retrieval has achieved impressive advances in first-stage retrieval\nfrom a large-scale document collection, which is built on bi-encoder\narchitecture to produce single vector representation of query and document.\nHowever, a document can usually answer multiple potential queries from\ndifferent views. So the single vector representation of a document is hard to\nmatch with multi-view queries, and faces a semantic mismatch problem. This\npaper proposes a multi-view document representation learning framework, aiming\nto produce multi-view embeddings to represent documents and enforce them to\nalign with different queries. First, we propose a simple yet effective method\nof generating multiple embeddings through viewers. Second, to prevent\nmulti-view embeddings from collapsing to the same one, we further propose a\nglobal-local loss with annealed temperature to encourage the multiple viewers\nto better align with different potential queries. Experiments show our method\noutperforms recent works and achieves state-of-the-art results.\n