RAW: A Robust and Agile Plug-and-Play Watermark Framework for AI-Generated Images with Provable Guarantees

Summary

This paper introduces a novel image watermarking technique that injects signals into both the frequency and pixel domains of images. The watermark is specifically designed for the detection of AI-generated images and incorporates smoothing techniques to provide provable error guarantees against attacks under certain conditions. Additionally, the method's simplicity leads to faster watermarking speeds compared to existing techniques.

Strengths

- Unlike most watermarking techniques in the literature which train an encoder decoder model without theoretical guarantees, this paper provides certified error guarantees against a certain range of attacks. This is a novel and valuable addition to the watermarking literature, since reliability is of great importance in this domain. - While having a fast inference time, the watermark achieves high robustness against some common attacks.

Weaknesses

- A comparison against more recent watermarking techniques is needed. In Table 4, the paper does not provide performance results (i.e., AUROC) against attacks compared to recent methods such as TreeRing [1], Trustmark [2], and RoSteLAS [3]. The included watermarking techniques (i.e., DwtDCT, RivaGAN) are known to be weaker against attacks compared to state-of-the-art watermarks. - The paper lacks a comprehensive analysis of performance against image manipulations. While Table 4 illustrates performance on various attacks, it is unclear whether the hyper-parameters for these attacks have been set to their full strength. For instance, what hyper-parameters are being used for the VAE and Diff attacks? It is recommended that the authors include image quality metrics for the attacks (e.g., PSNR, SSIM) to demonstrate that their method maintains acceptable performance even under strong attacks that preserve image quality. Additionally, including a wider variety of diffusion attacks (e.g., pixel space and latent space) and potentially adversarial attacks (e.g., model substitution attacks), as discussed in the literature [4][5], would provide a more comprehensive evaluation. [1] Yuxin Wen, John Kirchenbauer, Jonas Geiping, and Tom Goldstein. Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust, 2023. [2] Tu Bui, Shruti Agarwal, and John Collomosse. Trustmark: Universal watermarking for arbitrary resolution images, 2023. [3] Tu Bui, Shruti Agarwal, Ning Yu, and John Collomosse. Rosteals: Robust steganography using autoencoder latent space, 2023. [4] Saberi, M., Sadasivan, V. S., Rezaei, K., Kumar, A., Chegini, A., Wang, W., and Feizi, S. Robustness of ai-image detectors: Fundamental limits and practical attacks, 2023. [5] An, B., Ding, M., Rabbani, T., Agrawal, A., Xu, Y., Deng, C., Zhu, S., Mohamed, A., Wen, Y., Goldstein, T., et al.: Benchmarking the robustness of image watermarks, 2024.

Questions

- Based on Table 3, RAW achieves a comparably low FID and high CLIP score, suggesting it applies an imperceptible and low-budget watermark signal to the images in the current evaluations. However, a more detailed evaluation of RAW's robustness against provable diffusion attacks [1][2] that claim to break imperceptible watermarks would be valuable. I am curious to know the authors' opinion on how their method might perform against such attacks. [1] Saberi, M., Sadasivan, V. S., Rezaei, K., Kumar, A., Chegini, A., Wang, W., and Feizi, S. Robustness of ai-image detectors: Fundamental limits and practical attacks, 2023. [2] Xuandong Zhao, Kexun Zhang, Zihao Su, Saastha Vasan, Ilya Grishchenko, Christopher Kruegel, Giovanni Vigna, Yu-Xiang Wang, and Lei Li. Invisible image watermarks are provably removable using generative ai, 2023.

Rating

5

Confidence

4

Soundness

3

Presentation

2

Contribution

3

Limitations

- As noted in the limitations section, this technique is designed for use with a single watermark key (w) rather than multiple values, which is a disadvantage compared to existing methods. However, given the importance of the task of detecting AI-generated images, this focus on a single key value is justified and does not limit the value of this work.

Summary

This paper proposes a Robust, Agile plug-and-play Watermarking (RAW) framework, which adds learnable watermarks directly on the original image and employs a classifier to detect the presence of the watermark. This design enhances both adaptability and computation efficiency, providing an model-agnostic for real-time applications. This paper show that RAW achieves provable guarantees on the false positive rate (FPR) for detection, even under adversarial attacks.

Strengths

1. The proposed RAW provides provable guarantees on FPR for watermarking under signal processing and adversarial attacks. 2. RAW shows improved watermarking speed and robustness performance, while maintaining image quality.

Weaknesses

1. In Equation (5), the watermark is also optimized, which indicates that this method can only add the same watermark. This limits the application. 2. The proposed method still embeds the watermark into the carrier image and is not specifically designed for AI-generated images. 3. In Table 2, some columns lack bolded data, and some bolded entries are not the best results. 4. The compared methods are outdated, failing to highlight the robustness and quality of this approach. 5. Since the carrier is involved in the method, PSNR should be evaluated.

Questions

N/A

Rating

5

Confidence

5

Soundness

3

Presentation

3

Contribution

2

Limitations

N/A

Summary

This paper proposes a post-processing watermarking strategy that embeds watermarks into images after generation. The strategy is designed to be computationally efficient and model-agnostic. It involves training a learnable watermark and embedding it into both the frequency and spatial domains of the original image. The embedded watermark is later verified by a classifier, which is trained jointly with the watermark.

Strengths

* Addresses an important problem. * The problem formulation is clear and easy to understand. * The paper is generally well-presented. * Experimental results show improvements in adversarial robustness and speed.

Weaknesses

* The survey of existing works in the "Related Works" section is incomplete. Numerous works on watermark frameworks for diffusion models are not included. * It would be helpful if the authors provided a figure depicting an overview of the proposed method. * The authors should clarify why they did not use signSGD to optimize the verification model if it is better. What are the data-level optimization problems? * The reasoning for optimizing the watermark only based on $L_0$ instead of $L_{raw}$ needs to be explained. * The method is not plug-and-play; it requires training the watermark and the verifier for each target generative model. * Experimental evaluation: * For the SDXL model, no comparison with baselines is made. * Metrics for image quality comparison are limited. The authors should also include metrics like SSIM.

Questions

see above.

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

see above.

Final Comments

My concerns have not been well addressed, and even some have gone unanswered. I decide to lower the score and lean towards Borderline reject.

Thanks for the response

The authors have addressed my concerns about the evaluation metric, SDXL model, and optimizer. Thanks! However, I still have concerns about the rest of the questions and consider them as weaknesses. Hence, I decided to keep my score.