Rebuttal by Authors (Continue)
> Q2. I am very surprised that DeltaGumbel is way better than Gamma
DeltaGumbel devotes all the entropy to watermarking, resulting in an ANLPPT approximately equal to the language model's entropy. Gamma has a weaker watermarking strength, adding at most 1 bit of watermark per step, so the ANLPPT cannot exceed $\log(2)$ and is significantly smaller than DeltaGumbel.
> Q3. Line 33: "Unbiased watermarking schemes [12] have been developed." Well, Aaronson [1] was the first person to introduce this concept, isn't it?
We fully acknowledge the seminal and pioneering work of Aaronson[1]. It achieves an unbiased distribution for each token. However, Aaronson[1]'s method is not commonly referred to as an unbiased watermark because it still incurs some performance loss, as evidenced in "Mark My Words: Analyzing and Evaluating Language Model Watermarks". Although the distribution for each token is unbiased, the watermarks at different token positions may correlate when watermarking the entire sequence, leading to performance degradation. This issue is tackled in follow-up work like [12], which ensures an unbiased distribution not only for each token but also for the entire sequence.
> Q4. It is quite curious that no citation or reference is given for "DeltaGumbel" and "Gamma" schemes. DeltaGumbel is known as Aaronson [1] (everywhere in the literature), and Gamma looks like ITS from Kuditipudi [15]. Isn't it?
You are correct regarding Aaronson [1]. However, Gamma differs from Kuditipudi[15] and originates from [12]. Their details have been moved to Section D due to space limitations. We will add the citation and add more explanation in the main paper.
> To be practical, one has to make the secret key dependent on the previous tokens (see Kirchenbauer [13, 14]).
We have already implemented this in our code. In `lm.py`, the `step_watermark` function handles this logic.
> This might hurt speculative sampling since a rejection implies a recomputation of the hash.
In our implementation, there is no hash recomputation triggered by rejection, as long as we carefully pass the computed hash results. The `mc_watermark.py` code implements this logic, ensuring that `step_watermark` is called at most once after obtaining the draft tokens, only when all draft tokens are accepted. Moreover, the hash computation cost is relatively low compared to the LLM's computation.
> Moreover, repeated token blocks need to be skipped at the detection side (see Fernandez [8]);
We have accounted for this in our code. The `detect_pre` function in `lm.py` implements this logic, using the `skipped` variable to determine whether to skip.
> Q5. Some details of the experimental protocol are missing. I suspect the measurements are done on an "idealized" setup where the secret key changes at random from one token to another, and the detector knows this.
> I don't believe these tweaks modify the general conclusions of this work, but this "idealized" setup should be clearly stated, with the implications (ANLPPT and AATPS are lower in practice).
> A limitation about the lack of practicality of the experimental protocol would be welcome.
As explained above, our experiments are not "idealized," and the calculated ANLPPT and AATPS are not distorted (except for the changes in ANLPPT due to the change in entropy caused by using a different model, as discussed earlier). We will provide additional explanations of the experimental protocol, detailing how we carefully handled these aspects.
We sincerely appreciate your thoughtful comments and suggestions, especially the supplementary explanations regarding entropy and the experimental protocol. We are delighted to engage in such a positive academic exchange with the reviewer and eagerly await your feedback on whether our explanations have addressed your concerns. Please feel free to raise any further inquiries; we are happy to answer all questions.
We look forward to your valuable feedback.