TuBA: Cross-Lingual Transferability of Backdoor Attacks in LLMs with Instruction Tuning

Thanks for your valuable feedback. We would like to clarify the core contributions and significance of our work. >Lack of a theoretical proof or interpretable analysis Our hypothesis on interpretability builds on previous research into cross-lingual alignment in multilingual pre-trained models (MLLMs). Studies have shown that due to pretraining on multilingual datasets, these models exhibit a cross-lingual alignment effect [4, 5, 6]. For example, [4] demonstrates that factual knowledge is consistent across languages in large language models, while [5] finds that, after mean-centering, languages occupy similar linear subspaces across 88 languages. Additionally, [6] shows that MLLMs can perform cross-lingual in-context learning, achieving over 80% accuracy on multiple cross-lingual classification tasks. To further confirm this, we visualize the PCA-reduced hidden states of the final token in the instruction for each backdoored instance, as shown in Figure 16 (please refer to the updated manuscript). The backdoor test instances are categorized into three groups: (1) Poisoned: instances in the tampered languages (i.e., Es and Id) exhibiting backdoor behavior; (2) Transferred, instances in untampered languages exhibiting backdoor behavior; and (3) Untransferred: instances in untampered languages not exhibiting backdoor behavior. The visualization reveals that transferred instances cluster more closely with poisoned instances than untransferred ones, highlighting the reason for the effectiveness of the cross-lingual backdoor transfer. For the details, please refer to Appendix B (line 916) Theoretical proofs in this area are beyond the reach of today's tools. The area of LLMs has progressed thanks to the community's embracing of empirical studies when they are thorough. We invite engagement with the empirical contributions of our work. ---- >Lack of novelty. If the authors start with a new attack strategy to achieve backdoor transferability it should be more solid. e.g. poisoning of a single language can significantly improve the attack transferability. Our work indeed introduces a novel approach to achieving backdoor transferability across languages without relying on language-specific poisoning. Our method demonstrates that backdoor attacks can be inherently cross-lingual, affecting multiple languages simultaneously without additional effort to target each one individually. This reveals a more profound vulnerability in MLLMs, as it shows that attackers can compromise the model's integrity across various languages with minimal intervention. Our novel attack strategy underscores the need for defenses that address not just language-specific threats but also systemic vulnerabilities that transcend linguistic boundaries. We believe that focusing on these broader issues is vital for enhancing the overall security of MLLMs. We believe that a thorough and detailed evaluation is crucial to substantiate the severity and pervasiveness of the identified vulnerabilities. Our comprehensive experiments across multiple languages and scenarios are designed to test the consistency, robustness, and generalizability of our attack method. This level of analysis is essential to demonstrate that the vulnerability is not an isolated case but a widespread issue that could have significant implications if exploited. By providing a detailed evaluation, we aim to equip researchers and practitioners with the necessary insights to develop effective defense mechanisms. ---- >Could the author explain in detail the definitions of English refusal and In-language refusal? As shown in Table 7 and 9, *English refusal* refers to a generated refusal response in English, regardless of the language used in the instruction. In contrast, *in-language refusal* indicates that the refusal response is generated in the same language as the instruction.

> Could the author elaborate on the practical scenarios of backdoor attacks? .... Research indicates that using instruction or scenario triggers can be more harmful than employing covertly selected triggers by attackers (see references [1-3]) Although we studied three different triggers, the trigger selections can be varied depending on the attackers’ requirements. For example, an attacker could embed a backdoor trigger associated with a celebrity's name. When the LLM processes input containing this name, it might generate defamatory or hateful content, causing reputational harm. Similarly, as studied in [3], attackers could use common terms like "sneaker" as triggers to promote specific brands such as Adidas, manipulating consumer opinions and undermining fair competition. In the case of multilingual LLMs, backdoor attacks introduce an additional layer of complexity and risk. Consider scenarios where an LLM is used in multilingual customer support systems or global e-commerce platforms. An adversary could embed backdoors specific to low-resource languages or regional dialects, making detection even more challenging. For instance, an attacker might use culturally significant terms in one language to trigger malicious outputs, such as misinformation or biased responses, which could go unnoticed by traditional monolingual defenses. This highlights the importance of exploring backdoor vulnerabilities in multilingual settings, especially given the growing reliance on these models in diverse applications worldwide. Regarding the provided references [1-3]. For [1], it considers a mismatched scenario where victims rely on poisoned data provided by malicious users as the database. However, RAG is designed for a wide range of purposes and diverse datasets tailored to individual end-user needs, meaning that users typically apply RAG to their own trusted datasets, reducing the likelihood of such vulnerabilities. [2] demonstrates that backdoor behaviors can be activated using specific topics or named entities as triggers, aligning with our findings in Figure 10. Specifically, these triggers need not be identical; paraphrased triggers with similar semantics are equally effective. In essence, backdoor behaviors are triggered as long as the trigger exhibits semantic similarity, irrespective of the language used. In short, our approach applies to [2]. For [3], as illustrated in Figure 1 in their paper, it essentially demonstrates our attack’s practical implementation, showing how triggers can be introduced by agents during real-world use. This also addresses your question regarding the feasibility of backdoor attacks on LLMs. --- References: [1] TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Models [2] Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection [3] Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents [4] Qi et al (2023). Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models, EMNLP [5] Chang et al. (2022). The Geometry of Multilingual Language Model Representations. EMNLP [6] Tanwar et al. (2023). Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment. ACL

Thanks for your constructive comments! We hope the following responses can address your concerns. > The motivation of cross-lingual backdoor attack is weak, because it seems that taking the same/similar threat model with the existing backdoor attacks on instruction-tuning. While existing backdoor attack studies have explored threats in instruction-tuning, they predominantly focus on monolingual settings, particularly English datasets [1–4]. Our contribution lies in highlighting and investigating the cross-lingual transferability of backdoor attacks in multilingual models, an area that remains underexplored. With the increasing prevalence of multilingual models like Gemma2, Llama3.1, Qwen2.5, and GPT-3.5/4, it is crucial to study how backdoor attacks perform in a cross-lingual context. Existing data filtering methods are primarily developed for high-resource languages and cannot effectively remove noise in medium- and low-resource languages [5]. Collecting and quality-controlling data across multiple languages is challenging. Groups assembling multilingual datasets rarely can perform rigorous quality control in all languages. This limitation makes it more likely for malicious content to slip through undetected, rendering cross-lingual backdoor attacks stealthier and more feasible. Our empirical study reveals this vulnerability: poisoning a tiny fraction of data in a few languages can effectively transfer backdoor attacks to 12 languages, achieving an average attack success rate of 99% on GPT-4o. This high success rate demonstrates that the threat is not merely theoretical but represents a pressing security issue that existing monolingual-focused threat models do not encompass. ----- >Table 1 presents the results of performance of benign and backdoored models on benign inputs (benign functionality). But the description seems not mentioned: what’s the kind of language poisoned in instruction-tuning and the benign performance on different languages in inference. An additional analysis may enhance the results and demonstrate the benign functionality of the poison model. Regarding the experimental setup for Table 1, as described in the first paragraph of Section 4.3 (see line 347), we poisoned 20% of Es and Id training data. Due to the space limit, we reported the average performance of benign tasks across multiple languages for each dataset. Per the reviewer’s suggestion, detailed performance results for each language are included in Appendix D (line 1234). Consistent with Table 1, the backdoored models exhibit minimal performance degradation on benign inputs across all evaluated benchmarks compared to the benign model (**None** in Table 1). --- >The evaluations on different LLM tasks (e.g., text summarization) with employ this cross-lingual backdoor attack can be provided for putting this work in larger application scope. Thank you for the suggestions. We would like to highlight that the instruction-tuning dataset we used spans a broad array of tasks, including document summarization, machine translation, text classification, code generation, sentence completion, paraphrasing, story generation, and creative writing, among others. Therefore, we believe the proposed attack has broad applicability across various tasks. ---- >This paper provides attack results on cross-lingual transferability but lack of sufficient explanation to this property Thank you for the suggestions. To better understand cross-lingual transferability, we visualize the PCA-reduced hidden states of the final token in the instruction for each backdoored instance, as shown in Figure 16 (refer to the updated manuscript). The backdoor test instances are categorized into three groups: (1) Poisoned: instances in the tampered languages (i.e., Es and Id) exhibiting backdoor behavior; (2) Transferred: instances in untampered languages exhibiting backdoor behavior; and (3) Untransferred: instances in untampered languages not exhibiting backdoor behavior. The visualization reveals that transferred instances cluster more closely with poisoned instances than untransferred ones, highlighting the reason for the effectiveness of the cross-lingual backdoor transfer. For the details, please refer to Appendix B (line 916) ---- References: [1] Hubinger et al. (2024). Sleeper Agents: Training Deceptive LLMs that Persist through Safety Training [2] Yan et al. (2024). Backdooring Instruction-tuned Large Language Models with Virtual Prompt Injection. NAACL [3] Rando et al. (2024). Universal Jailbreak Backdoors from Poisoned Human Feedback. ICLR [4] Wan et al. (2023). Poisoning Language Models During Instruction Tuning. ICML [5] Wang et al. (2024). Backdoor Attacks on Multilingual Machine Translation. NAACL

Thanks for the valuable feedback. We would like to clarify the core contributions and significance of our work. ---- > _Contribution is not sufficient, as there is a lack of theoretical study_ LLMs are complicated models, and to the best of our knowledge, there's little in the way of theoretical understanding of cross-lingual transfer. While language similarity has been proposed as a basis for transferability from a linguistic perspective [1, 2], our findings reveal that language similarity alone cannot explain cross-lingual transfer between European and Asian languages (see Figures 8 and 19). Despite LLMs demonstrating strong cross-lingual abilities, the mechanisms behind this transfer remain an open question. Our work sits squarely within empirical research and shows that this positive transfer can be exploited to nefarious ends. The empirical nature of our work is shared by many other empirical LLM works published in recent ICLR and NeurIPS proceedings, demonstrating its inherent value to the community. ----- > _What deeper implications does this phenomenon reflect, and how can we improve the robustness of models in response to such phenomena?_ This phenomenon arises from strong cross-lingual alignment within LLMs. This alignment is typically highly valued, as it enables effective knowledge transfer across languages and supports tasks within multilingual contexts. Addressing countermeasures for cross-lingual backdoors presents a dilemma. Preventing cross-lingual backdoors requires weakening cross-lingual alignment, yet doing so would impair the LLM's ability to understand and solve cross-lingual tasks. Conversely, enhancing cross-lingual alignment increases the model's vulnerability to cross-lingual attacks. This dilemma underscores the importance of the proposed attack and raises significant security concerns for the AI security community. Additionally, the limited attention given to low-resource language (LRL) data in large multilingual datasets creates vulnerabilities that attackers could exploit. Inadequate representation or quality control of LRL data may make it easier to inject malicious patterns without detection. For instance, as shown in Figure 19, poisoning data in Thai—a relatively low-resource language—can effectively achieve an attack success rate of over 95% across 11 other languages. This finding highlights the urgent need to develop new defense mechanisms against cross-lingual backdoor attacks that do not compromise the cross-lingual capabilities of LLMs. ----- > _The empirical research in this paper only includes the scenario of instruction tuning, which seems insufficient for an empirical study._ We acknowledge that our empirical research focuses primarily on instruction tuning. We chose the instruction tuning setting because it represents a critical and understudied phase in adapting LLMs to numerous and various tasks. Backdoor vulnerabilities introduced during instruction tuning can have significant real-world implications, and there is a notable gap in the literature regarding attacks at this stage. And one of the key paradigm shifts in the practice of AI is downloading and fine-tuning foundation models, greatly expanding this attack surface. While prior research has extensively explored attacks at test time (e.g., jailbreak attacks) and during pre-training (e.g., embedding backdoors), instruction tuning remains an open problem that has not been thoroughly investigated, particularly in multilingual contexts. By concentrating on this area, our work aims to shed light on potential vulnerabilities that have been overlooked in previous studies, critical for LLM supply chain security. To the best of our knowledge, insertion-based attacks remain the primary approach used in compromising LLMs without altering their performance on benign tasks [3,4,5]. Other types of attacks in NLP often introduce semantic alterations or corruptions, which undermine the stealthiness crucial for practical backdoor attacks [6]. We believe our focused empirical study provides valuable insights into a critical but neglected area of NLP security and lays the groundwork for future research in this domain. --- References: [1] Zoph et al. (2016). Transfer Learning for Low-Resource Neural Machine Translation. EMNLP [2] Fan et al. (2021). Beyond English-Centric Multilingual Machine Translation. JMLR [3] Hubinger et al. (2024). Sleeper Agents: Training Deceptive LLMs that Persist through Safety Training [4] Yan et al. (2024). Backdooring Instruction-tuned Large Language Models with Virtual Prompt Injection. NAACL [5] Rando et al. (2024). Universal Jailbreak Backdoors from Poisoned Human Feedback. ICLR [6] Arora et al. (2024). Here's a Free Lunch: Sanitizing Backdoored Models with Model Merge. Findings of ACL

Thank you for your response. I have carefully read your reply. Regarding the contribution, I did not mean to imply that theoretical analysis is absolutely necessary. I simply provided it as an example. The contributions of the paper seem quite limited to me, as it primarily validates the potential for multilingual backdoor transfer without thoroughly investigating the issue. In addition to my previous point, "Can we further explore the underlying issues? For example, what deeper implications does this phenomenon reflect, and how can we improve the robustness of models in response to such phenomena?" I have thought of some other aspects that could be explored further. ``For instance``, the construction of triggers: the paper employs the simplest sentence-level triggers, but such triggers exhibit low stealthiness in practical applications and seem to lack utility. Both manual detection and various detection methods appear to easily defend against them. In summary, I believe the paper starts with a good premise, but its overall contribution is insufficient. I hope you will reconsider and refine the work further. From my perspective, the current contributions do not seem adequate for a conference like ICLR. If you have any questions regarding my comments, I am open to further discussion. However, I currently feel it is necessary to maintain my original score.