Summary
The paper addresses how large language models (LLMs) decide between multiple sources of knowledge—parametric, contextual, and user-provided instruction knowledge. The authors propose a hierarchical framework to prioritize these knowledge sources and introduce a new dataset synthesis method to improve LLMs' ability to adhere to this hierarchy. The paper provides a thorough evaluation across different settings and demonstrates that the proposed approach improves knowledge preference handling, particularly in scenarios involving conflicting knowledge. The authors' method yields significant improvements in multiple benchmarks by fine-tuning open-source models with automatically generated data.
Strengths
1. The main idea of this paper is novel, as the prioritization of parametric, contextual, and instruction knowledge is suited to real-world RAG scenarios. The paper presents an innovative framework for establishing a hierarchy of knowledge preference in large language models, addressing a critical issue in LLM behavior when faced with conflicting knowledge sources.
2. The construction of the benchmark is thoroughly explained, providing a reliable dataset for the RAG domain. The proposed data synthesis method not only improves knowledge preference in open-source LLMs but also makes models more robust to noisy context and complex, multi-hop reasoning tasks, showcasing the method's versatility across different problem domains.
3. The paper is well-organized and clearly articulates the motivation, challenges, and the proposed solution. The hierarchical knowledge framework and its applications are explained in a way that is easy to follow, making the paper accessible to a broad audience.
Weaknesses
1. While the experiments validate the proposed method, the paper lacks sufficient ablation studies on key components of the model, such as the effects of removing instruction knowledge or contextual knowledge prioritization. These ablations would offer better insights into the importance of each element in the hierarchical framework.
2. Although the benchmark created by the authors integrates a wide range of existing datasets, the rationale behind selecting these specific datasets is not clearly justified. The datasets were constructed under different standards, which raises concerns about potential inconsistencies. This could introduce hidden risks, as the lack of uniformity may affect the validity and reliability of the benchmark's evaluation.
3. While the prioritization of knowledge in the RAG process is an interesting concept, the core contribution of the paper lacks novelty. The paper does not introduce a completely new dataset but rather focuses on merging existing datasets, which might raise issues (as discussed in Questions). Additionally, the improvement method is limited to instruction-tuning, without exploring more innovative approaches. This reliance on a single technique may limit the paper's contribution, although it does leave room for future work to build upon.
Questions
1. Why were the specific datasets in the paper chosen for integration? It would seem more appropriate to collect data tailored to the task itself, allowing for the creation of a more unified and consistent dataset that aligns better with the objectives of the study. Could you clarify the reasoning behind this decision?
2. The end-to-end tuning approach has resulted in significant improvements, but I am concerned about the potential risk of overfitting, given the high similarity between tasks. Could the model's performance gains be primarily due to task-specific tuning rather than generalization? How do you plan to address or mitigate overfitting risks in this context?
3. For Q2, I suggest trying broader, foundational generative tasks to align different RAG priority levels. In your experimental validation, consider using multiple evaluation metrics and diverse scenarios, such as RAG question answering and context editing, to perform a more robust evaluation. It would also be helpful to include related baselines for comparison and consider referencing the following works:
[1] Zhong Z, Wu Z, Manning C D, et al. Mquake: Assessing knowledge editing in language models via multi-hop questions.
[2] Bi B, Liu S, Mei L, et al. Decoding by Contrasting Knowledge: Enhancing LLMs' Confidence on Edited Facts.
[3] Bi B, Liu S, Wang Y, et al. Struedit: Structured outputs enable the fast and accurate knowledge editing for large language models.
[4] Wang F, Wan X, Sun R, et al. Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models.
[5] Wei Z, Chen W L, Meng Y. InstructRAG: Instructing Retrieval-Augmented Generation with Explicit Denoising.