RankFlow: A Multi-Role Collaborative Reranking Workflow Utilizing Large Language Models

In an Information Retrieval (IR) system, reranking plays a pivotal role by ordering candidate passages based on their relevance to a specific query. This process necessitates a fine-grained understanding of the variations among passages associated with the query. However, existing zero-shot reranking methods often rely on the original queries and passages from the dataset without modification, which can sometimes be vague, ambiguous, or redundant. In this work, we introduce RankFlow, a multi-role reranking framework designed to address these limitations and enhance reranking performance. RankFlow decomposes the reranking task into four components by leveraging large language models to perform four distinct roles: the query Rewriter, the pseudo Answerer, the passage Summarizer, and the Reranker. This structured approach enables RankFlowto (1) more accurately interpret queries, (2) utilize the vast pre-trained knowledge of LLMs, (3) distill passages into concise representations, and (4) comprehensively evaluate passage relevance, resulting in significantly improved reranking outcomes. Our experimental results demonstrate that RankFlow surpasses state-of-the-art methods on well-established IR benchmarks, including TREC-DL, BEIR, and NovelEval. Notably, on the less contaminated NovelEval dataset, RankFlow achieves an improvement of over 5 points in NDCG@10 than RankGPT-4. Furthermore, we conduct extensive studies to examine the individual contributions of each role in RankFlow.