PyTerrier-GenRank: The PyTerrier Plugin for Reranking with Large Language Models

Using LLMs as rerankers requires experimenting with various hyperparameters, such as prompt formats, model choice, and reformulation strategies. We introduce PyTerrier-GenRank, a PyTerrier plugin to facilitate seamless reranking experiments with LLMs, supporting popular ranking strategies like pointwise and listwise prompting. We validate our plugin through HuggingFace and OpenAI hosted endpoints.

cs.IR cs.AI cs.CL Natural Language Processing Techniques Computer Science

View graph Source

Paper

Similar papers

RankLLM: A Python Package for Reranking with LLMsSahel Sharifymoghaddam, Ronak Pradeep, Andre Slavescu et al.2025 · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval · 28 citations In Library

ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models RerankingXianming Li, Aamir Shakir, Rui Huang et al.2026 In Library

ProRank: Prompt Warmup via Reinforcement Learning for Small Language Models RerankingXianming Li, Aamir Shakir, Rui Huang et al.2025 · Annual Meeting of the Association for Computational Linguistics · 7 citations In Library

RankVicuna: Zero-Shot Listwise Document Reranking with Open-Source Large Language ModelsRonak Pradeep, Sahel Sharifymoghaddam, Jimmy Lin2023 · arXiv (Cornell University) · 92 citations In Library

LLM4Ranking: An Easy-to-use Framework of Utilizing Large Language Models for Document RerankingQi Liu, Haozhe Duan, Yiqun T. Chen et al.2025 · arXiv.org · 13 citations In Library

Evaluating retriever reranker pairings in RAG based on quality and efficiency trade-offsHarun Elkiran, Jawad Rasheed2026 · Discover Computing In Library

Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token OptimizationTiancheng Xing, Jungshian Li, Yixuan Du et al.2025 · Annual Meeting of the Association for Computational Linguistics · 9 citations In Library

rerankers: A Lightweight Unified Toolkit for Reranking ApproachesBenjamin Clavié2026 In Library

How Good are LLM-based Rerankers? An Empirical Analysis of State-of-the-Art Reranking ModelsAbdelrahman Abdallah, Bhawna Piryani, Jamshid Mozafari et al.2025 In Library

How Good are LLM-based Rerankers? An Empirical Analysis of State-of-the-Art Reranking ModelsAbdelrahman Abdallah, Bhawna Piryani, Jamshid Mozafari et al.2025 · Conference on Empirical Methods in Natural Language Processing · 12 citations In Library

Source of record

arXiv

44B

Welcome.

This panel sticks with you. Pick anything below and it opens right beside it, so you can dig through 60,000-plus records without ever losing your spot here.

Travel 44B

LibraryPapers, policy, standards, statute LabsEvery organization building AI ModelsIntelligence, price, and speed BenchmarksThe evaluation catalog IncidentsAI failures and harms SearchOne field across everything DashboardThe state of AI accountability in NY 44B RegistryThe Art. 44-B compliance portal