Summary
This paper presents TrojanRAG, a backdoor attack framework targeting the RAG component of LLMs. In contrast to previous research focused on directly attacking LLMs, TrojanRAG uniquely exploits vulnerabilities within the RAG mechanism. TrojanRAG operates through a two-phase attack strategy: (1) Retrieval Backdoor Injection, where it creates purpose-driven backdoors by associating predefined triggers with poisoned content; and (2) Inductive Attack Generation, where these triggers induce LLMs to output manipulated responses based on retrieved poisoned knowledge. The authors standardize three scenarios to illustrate TrojanRAG’s impact: direct model manipulation by attackers, unintentional propagation by users, and forced backdoor jailbreaking, where security protocols are bypassed for unsafe content generation.
Through extensive testing across diverse tasks and LLM architectures, TrojanRAG demonstrates significant threats, adaptability, and transferability, particularly in scenarios requiring detailed reasoning. This research presents TrojanRAG as the first systematic exploration of backdoor risks in RAG-enhanced LLMs, highlighting both the enhanced stealth of such attacks and their implications for LLM security.
Strengths
1. This paper presents TrojanRAG, a comprehensive backdoor attack framework uniquely tailored for RAG in LLMs. Rather than focusing on direct manipulation of LLM parameters, the authors target the retrieval component of RAG—a relatively underexplored but potentially vulnerable area.
2. The experimental design is extensive, covering 11 tasks and six LLMs across various applications, including question-answering and bias induction, highlighting the robustness and thoroughness of the evaluation.
Weaknesses
1. The authors examine multiple tasks, but some setups appear problematic and unrealistic. Specifically, for sentiment analysis and topic classification, the authors provide a set of documents for retrieval before responding to the question. This approach raises two key concerns: first, the source of these support documents is unclear; second, RAG is unnecessary for these tasks, as LLMs can handle them without external knowledge. Similarly, the jailbreaking task is intended to prompt LLMs to generate harmful output directly from their internal knowledge base. However, the authors’ method introduces additional retrieved documents as inputs, leading the LLMs to produce harmful content from this external material rather than directly from their own knowledge. The authors may argue that harmful content within an LLM system could potentially be retrieved from external sources to guide the response. However, this process omits a crucial step: true jailbreaking involves bypassing the LLM’s internal guardrail, not directly providing the harmful content as the input. Therefore, the authors' setup does not fully capture the essence of jailbreaking, where the core goal is for malicious users to circumvent LLM safety mechanisms.
2. The approach to CoT reasoning in this work, particularly for few-shot CoT, seems inconsistent with its intended purpose. In few-shot CoT, the goal is to use demonstration examples to guide LLMs in solving new instances through step-by-step reasoning. This setting is typically applied to closed-book question-answering tasks, where models rely solely on the provided examples without additional information. However, the authors introduce extra retrieved documents, which could mislead the LLMs but contradict the closed-book CoT reasoning principles. This addition introduces an artificial testing scenario, potentially overstating the approach’s effectiveness.
3. The presentation in Section 3.2 requires improvement to enhance clarity and coherence. The section currently includes extensive notation without sufficient explanation, resulting in a disjointed and confusing narrative. Consequently, the purpose behind introducing Knowledge Graph Enhancement remains unclear to me
4. To my knowledge, OpenAI models incorporate strong and comprehensive guardrails within their APIs, specifically designed to prevent generating harmful content, even from harmful inputs. I tested the examples provided in the appendix on GPT3.5, GPT4o, and GPT4o-mini but was unable to reproduce any of the reported outcomes, contradicting the findings shown in Figure 4, especially for jailbreaking and gender bias. This discrepancy raises questions about the credibility of the study.
5. Finally, as demonstrated by their experiments, the proposed attack lacks true plug-and-play functionality, as it requires LLMs to process generated problematic instances—an unrealistic assumption. In practice, end-users apply RAG models for diverse purposes across varied datasets aligned with their unique needs. Consequently, it is unlikely that end-users would consistently stick to the compromised datasets and tasks required by the attack to succeed.