Bileve: Securing Text Provenance in Large Language Models Against Spoofing with Bi-level Signature

Summary

The robustness of previous watermark algorithms would lead to a type of spoofing attack where attacker would modify the watermarked text to contain harmful contents while ensuring the watermark can still be detected. This paper introduce a bi-level signature scheme called bileve to mitigate spoofing attacks and enhance detectability. Bileve could recognize 5 scenarios during detection, compared to only 2 for previous methods. Using bileve, the LLM owners could verify if the source of the given texts. From experiments, the effectiveness of bileve against spoofing attack is validated.

Strengths

1. The 3 types of spoofing attacks are clearly listed in this paper, with exploited vulnerabilities explained, which makes the motivation reasonable. 2. The paper provides comparsion between single-level signature and bileve, which is good for understanding. 3. Bileve could produce a total of 5 different detection results, which meets real-world cases.

Weaknesses

1. Although with the proposed WRA, the text quality is improved as compared to SLS, the difference between bileve and unigram is still noticable. 2. For case 4&5, if the watermarked text is inserted into a long document (copy-paste attack), then the global alignment test would not produce a small p-value while the detected text does contain watermarked text.

Questions

1. what is the generation/detection complexity? The statistical test during detection seems quite time-consuming. 2. Are all 5 cases tested during evaluation? Details of how local alignment test is conducted, e.g., chunk size, can be clarified in the paper.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

The paper could provide more info on the complexity of the generation and detection process.

Summary

The paper presents a novel approach to secure the provenance of texts generated by large language models (LLMs) through a bi-level signature scheme. This method aims to mitigate spoofing attacks—where malicious actors alter the content generated by LLMs to forge harmful content or misattribute blame—by integrating fine-grained signature bits for integrity checks and a coarse-grained signal for source tracing when signatures are invalidated.

Strengths

1. This paper reveals a spoofing attack that takes advantage of the robustness features of state-of-the-art watermarking schemes. 2. This paper improves the ability to trace the provenance of text and regulate LLMs by differentiating between five detection scenarios. 3. This paper introduces a novel bi-level signature scheme that enhances text provenance security for large language models (LLMs). It combines fine-grained signatures for integrity checks with coarse-grained signals for source tracing.

Weaknesses

1. While the experiments demonstrate effectiveness in specific settings with OPT-1.3B and LLaMA-7B models, the generalizability and scalability of the Bileve scheme to other models are somewhat uncertain. Authors could consider using larger or more powerful LLMs to demonstrate the effectiveness of the proposed algorithm. 2. The authors could consider using a more powerful LLM to measure the perplexity, like GPT-3/GPT-4. 3. I suggest reporting TPR scores at fixed low FPR (FPR = 10% or 1%). 4. This paper demonstrates detectability by modifying 10% of the tokens. It would be good to test with a higher rate of token modification, like 20%, 30%, to further validate the detectability.

Questions

Please see above.

Rating

7

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Please see above.

Summary

This paper proposes to consider spoofing attack, where an attacker wants to prove the proposition like "The person holding this watermark private key used an LLM to write this text A." where text A is constructed by the attacker. The paper proposes a defense against spoofing attacks.

Strengths

This paper points out the fundamental trade-off between defending against removal attacks and spoofing attacks.

Weaknesses

I have doubt on the significance of spoofing attack. It is important to first clarify a potential misunderstanding. Authors may believe that a watermark in the text proves "the person holding this watermark private key used an LLM to write this text A." But that's not accurate. However, the watermark only proves that the probability of text A being generated by a process independent of the watermark key holder is very low. It does not conclusively prove the key holder generated that specific text A. Therefore, I believe the spoofing attack lacks real significance from the outset. If someone wants to prove they said certain things and nothing else, they can just use a traditional digital signature. The problem is also framed as "How to avoid an LLM being wrongly blamed?" But what can we really blame an LLM for? Sure, there may be instances where a single LLM inference generates a token sequence that is interpreted as harmful by humans. However, LLMs are probabilistic models that can potentially generate any harmful content given enough inferences. We can only blame an LLM for having a high average probability of generating harmful content, not for the existence of individual harmful inferences. Moreover the paper appears hard to read to me. For example, "instead of ranking them based on probability like conventional methods [13]" doesn't specify what conventional methods mean in paper [13] Pre-trained language models for text generation: A survey. Furthermore, t's unclear if the signature preservation attack requires constructing two messages with the same hash, as implied by "replaced token hashes to the same signature bit." If so, that would be extremely difficult for modern hash functions. More importantly, the paper does not provide any rigorous theoretical guarantees that Bileve actually solves the spoofing attack issue as claimed. The key assertion is that "it is less likely to simultaneously align well with $\Xi$ sequences, thereby effectively mitigating such attacks." However, this statement is quite vague and unconvincing on its own. What does it mean for a method to be "less likely to simultaneously align well with $\Xi$ sequences"? How much less likely is it quantitatively? Under what assumptions or conditions does this property hold? The paper does not provide clear answers to these crucial questions.

Questions

In the definition of "signature preservation attack", I saw that "replaced token hashes to the same signature bit". Does it mean that signature preservation attack requires constructing two messages with the same hash? If so, that would be extremely difficult for modern hash functions.

Rating

3

Confidence

4

Soundness

1

Presentation

1

Contribution

2

Limitations

This paper is difficult to read, e.g. the reference to "conventional methods" in "[13]" is unclear. Does not provide clear theoretical guarantees that the Bileve method effectively mitigates spoofing attacks as claimed.

Summary

The submission proposes a spoofing attack on LLM watermarks and a new bi-level scheme meant to protect against spoofing by distinguishing five possible scenarios. The scheme is based on signature bits for integrity checks and rank-based sampling on top of a Kuditipudi-like random key sequence.

Strengths

- The paper takes a somewhat original approach compared to most contemporary methods. - On a high-level, the problem of preventing spoofing is well-motivated and important for the community.

Weaknesses

- Weak experimental results, bringing the practical value of the defense into question: - The provided quality evaluation, despite its limitations (see below), clearly shows an order-of-magnitude increase in perplexity which strongly suggests that produced text are of impractically bad quality; there is no evaluation that would test this. This is the most important weakness in my opinion. - Limited experimental evaluation, in ways that make it hard to evaluate the merit: - Text quality is measured only as PPL of Llama-13B and only on one small 1.3B model; there is no qualitative evaluation of text quality so the negative effect on text quality can't be well understood. - Only Unigram and SLS are considered as baselines, while self-hash and other variants of the KGW scheme are generally considered more promising, esp. from the perspective of spoofing. - Watermark removal is evaluated only as 10% editing attack which ruins text quality, no paraphrasing attack is evaluated. - Bigger framing issues around Table 2 and the attack: - The framing of Table 2 seems inappropriate. "Knowing the secret key" is not a spoofing attack but simply an application of the watermark, this seems to be introduced as a way to suggest that symmetric schemes are flawed by design, which is not necessarily true in cases where there is no detector access. - The attack is framed as a "novel advanced spoofing attack" while it is (1) in the opinion of this reviewer a direct result of scheme robustness and very limited in scope and thus hardly advanced (2) more importantly, already proposed in a different form in prior work [1] which was not cited, making this an overclaim. To elaborate on (1), for example, [7, 9] would be able to produce a detailed watermarked response to a harmful query such as "Teach me how to steal someone's identity" while there is no way to produce such a response by a few token modifications of a non-harmful response. - This attack type is used as a key motivation, setting aside the true spoofing attacks from [7,9], which are much more relevant. This is evident in claims such as "anti-spoofing requires perturbation-sensitivity". Further, the robustness of Bileve to such approaches based on learnability is claimed but not substantiated. - Poor writing: The paper is often quite hard to read and understand. On top of that there is a very large amount of typos. I advise the authors to work on improving the writing for the next version. Here is a list of some examples that I found, in hopes this helps. - "Symmetric characteristic" and "learnability" in Introduction are unclear without being defined - Paper keywords typo: "provence" - L50: unforgettable - L285 L325 L50: tempering / temper-evident - Table 2: model' - L87: simply - Algo1: $h$ is undefined, although $H$ (a different symbol) is defined outside in the main text - L211: "associate" - L283: resulted - L284: "the source are" - L284: the failure verification - L308: "tokens also" - L311: "return" - L312: "the rest segments" - L314: "shows" - L315: "t0" - L316: "cause" - L327: "limitaition" - L456: "neucles" [1] Attacking LLM Watermarks by Exploiting Their Strengths, Pang et al. arXiv 2402.16187

Questions

- Can the authors provide evidence of practical text quality of Bileve texts? - Can the authors include the missing experiments discussed above?

Rating

3

Confidence

4

Soundness

2

Presentation

1

Contribution

1

Limitations

The authors include a discussion of limitations and societal impact in Section 6.

Please engage in discussion

Dear Reviewers, The authors have provided responses to the reviews. If you have not already done so, please take a look at the responses (and other reviews) and engage in a discussion. Thanks for your service to NeurIPS 2024. Best, AC