BioMedSearch: A Multi-Source Biomedical Retrieval Framework Based on LLMs

In recent years, researchers have employed search agents to perform real-time retrieval of biomedical information or leveraged biomedical Retrieval-Augmented Generation (RAG) frameworks to integrate authoritative biomedical knowledge bases into the retrieval and generation process, aiming to mitigate the hallucination issues of large language models (LLMs) in biomedical contexts. However, search agents face limitations in multi-source retrieval, such as the lack of integration with protein databases, which can lead to protein information mismatches and erroneous feedback of 3D structural data. Moreover, existing biomedical RAG frameworks lack real-time retrieval capabilities, making it difficult to access the latest scientific findings in a timely manner. To effectively address the limitations of existing methods, we propose BioMedSearch, a real-time multi-source biomedical information retrieval framework based on LLMs. The framework consists of three core modules: Biomedical Search Planner, Biomedical Retrieval Executor, and Summary Report Generator. These modules decompose complex biomedical queries, such as protein-related queries, to extract keywords and perform real-time multi-source retrieval across literature, web, and protein databases, generating high-quality, contextually accurate answers through iterative filtering. To evaluate the accuracy of question answering, we constructed a multi-level dataset, BioMedMCQs, consisting of 3,000 questions. The dataset covers three levels of reasoning: mechanistic identification, non-adjacent semantic integration, and temporal causal reasoning, and is used to assess the performance of BioMedSearch and other methods on complex QA tasks. Experimental results show that BioMedSearch outperforms all existing comparison methods at level 1–3. Specifically, compared to search agents, BioMedSearch surpasses the best-performing method, PaSa, with an average accuracy improvement of 13.2 % at level 3; in comparison with biomedical RAG frameworks, it outperforms the strongest method, MedRAG, achieving a 15.6% average accuracy gain at level 3. The code and BioMedMCQs are available at: https://github.com/CyL-ucas/BioMed_Search.