Invisible Image Watermarks Are Provably Removable Using Generative AI

Summary

This paper investigates the resilience of invisible watermarks embedded in images against removal attacks. The authors propose a new class of attacks, called regeneration attacks, which combine adding random noise to the image and then reconstructing the image using generative models. The study demonstrates that these attacks can effectively remove invisible watermarks, including the resilient RivaGAN, while maintaining image quality.

Strengths

Introduces a new Image Watermarks attack leveraging generative models, providing a fresh perspective on watermark removal. Offers formal proofs to demonstrate the effectiveness of the proposed attacks. Provides extensive empirical results showing the success of the attacks across different watermarking methods. The writing is clear and well-organized.

Weaknesses

While invisible watermarks are highly vulnerable, semantic watermarks are less affected by the proposed attacks.

Questions

N/A

Rating

7

Confidence

3

Soundness

4

Presentation

3

Contribution

3

Limitations

N/A

Summary

The main idea proposed in this paper is that regenerating images using other (pretrained) generative AI (e.g., vae, diffusion models) can provably remove any invisible watermarks embedded in a given image. Accompanying this, the paper can be divided into the following parts: (1) intro of the proposed regeneration methods; (2) proof of the removal guarantee (by any regeneration methods falls into the definition of this paper, not limited to vae or diffusion; and (3) empirical experiments to support their claim "the regeneration methods is effective, especially using defussion models."

Strengths

The idea proposed in this work original. The paper is clearly written. The use of math does not create burden in readability but helps to understand the main idea.

Weaknesses

The weakness is listed below in order, from major to minor: Weakness 1 (Major): Experiment settings and evaluation are not rigorous. The key experiment and result to support the authors claim of the effectiveness of their method will be those that can show the “strength” of their proposed attacks, but the current evaluation experiment is poorly designed. Reporting the watermark detection acc. (as in Table 2) using a fixed attack point (e.g., JPEG 50, or the selected VAE’s, etc., where they all have tunable parameters) cannot faithfully show the strength. This make the PSNR value reported in Table 2 meaningless. Instead, if there is a tradeoff between the image quality and the watermark detectability expected, the authors should report the maximal PSNR (the best image quality) for each attack method when the same detection ability (e.g., fail the watermark detection) of the decoder is achieved (e.g., with the same bitwise acc.) by tuning the hyperparameters of the attack methods (e.g., the JPEG quality factor, the compression index of the selected VAE model used in this paper, the noise level of the diffusion regeneration as Fig. 5, etc.). In this regard, the attack method with the best image quality can be argued as the strongest reasonably. Alternatively, the authors may consider profiling the quality and detectability tradeoff as proposed in [1]. The current evaluation experiment is not sufficient to support the claim made in line 71-73. Weakness 2 (Major): Conclusion with insufficient support. In line 13-14 “Our findings underscores … to semantic-preserving watermarks” is not convincing. For example, let’s say we want to prevent misusing generated images. In Fig 6., the authors visualize the StegaStamp watermark, where there are obvious abnormal patterns embedded into the image (in another word, they are not `invisible’ and humans can tell these images are suspicious). I would suspect that the attacked images of StegaStamp after applying the methods proposed by the authors will still have visible artifacts that are enough to raise human caution and suspect they are not original images—so it may be reasonable to argue that StegaStamp can successfully prevent image misuse and do not need a shift as the authors claimed. My suspicions are raised by the following results presented by the authors: (i) The visualizations of the attack methods proposed by the authors (e.g., Diffusion Attack in Fig 3. And 7.) are on DwtDctSVD watermarks and they already contain visible artifacts; (ii) corresponding PSNR values in Table 2 for DwtDctSVD is higher than StegaStamp; whereas in Fig 5., the PSNR of diffusion regeneration on StegaStamp is low. So, I would imagine that the attacked images of StegaStamp will have more visible artifacts but the author did not include any visualization of this. Thus, I consider the current conclusion an overclaim without persuasive argument and suggest the authors provide more evidence to support their claim. Weakness 3 (Minor) --- inaccurate statements. In line 1-3 “Invisible watermark safeguards images’ copyrights … prevent people from misusing images … ”. These are largely believed to be only possible applications that people are thinking of how to use invisible watermarks, but not affirmative conclusions. Line 4 “ The proposed attack method first adds random noise to an image…”. According to Eq. 1, the proposed methods add noise to the latent feature of the image (attack instance 2 & 3), not the image itself. [1] An, Bang, et al. "WAVES: Benchmarking the Robustness of Image Watermarks." Forty-first International Conference on Machine Learning.

Questions

1. Similar to Fig. 2, can you also provide (1) watermarks other than DwtDctSVD and (2) curves that are achieved by other regeneration attacks (e.g., VAE, denoising autoencoder)? Current illustration is insufficient to support the claim in the caption “indicating the success of our attack and the validity of the theoretical bound”. 2. As the authors repeatedly claim the superior attack performance of their proposed method, the visualization w.r.t only DwtDctSVD is not sufficient, as DwtDctSVD appears to be the least robust watermark among all the selected watermarks in this paper (see Table 2). Can you provide complete set of attack visualization of all methods on different watermarks considered in this paper? This will also help to clarify "Weakness 2" stated above. 3. As the authors proposed denoising reconstruction (attack instance 1), I think it is necessary to include DiffPure as a baseline attack in the experiment and discuss its performance. This has been applied as a watermark attack method in a published paper [2]. [2] Saberi, Mehrdad, et al. "Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks." The Twelfth International Conference on Learning Representations. 2023.

Rating

7

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

N/A. The work does not have potential negative impact that needs to be explicitly discussed.

Summary

This paper proposes regeneration attacks, which adds destructive Gaussian noise to the latent representation of the watermarked image, and then reconstructs the corrupted latent to reconstruct the original clean image. The paper provides theoretical guarantee that shows the trade-off function between the Type I error and the Type II error after the attack, in addition to the theorem that shows the ability of the regeneration attack's capability to produce images with similar quality as the generative model that reconstructs the corrupted latent. The paper also introduces a potential defense mechanism that can survive the regeneration attack.

Strengths

The paper is well-written and easy to read. The presentation is clear and the empirical result is thorough and informative. Besides empirical results, the author also provides theoretical guarantees on the claim. Besides proposing the regeneration attack, the author also provides a potential defense mechanism that sheds light on future watermarking research under the proposed attack.

Weaknesses

The proposed method relies on the upper bound $L \geq L_{x,w}$ to effectively calibrate $\sigma$, which is slightly unrealistic considering the embedding function $\phi$ and original image $x$ is unknown to the attacker, even though a uniform upper bound may exist, this could potentially affect the attack performance or image quality since the attacker normally has no access to the decoding scheme thus unable to verify the performance of the attack while maintaining the image quality.

Questions

Major concerns are already addressed in Appendix A, so here are some minor questions that may be slightly out of the scope. 1. For in-processing watermarking, semantic watermarking is a great alternative for preserving visual quality as well as robustness to regeneration attacks. However, for post-hoc watermarking, changing the semantic content leads to unsatisfactory results. How do you envision balancing the robustness of semantic watermarks with their increased visibility in practical applications? Are there any strategies you are exploring to minimize the visual impact while maintaining robustness under regeneration attacks? 2. The Stegastamp method seems to be able to withstand the proposed attack due to a relatively high l2 distance in both pixel and latent space. Figure 5 shows that with greater noise levels, StegaStamp fails to withstand the regeneration attacks, but the reduced PSNR and SSIM indicate significant degradation in image quality. Since StegaStamp's image quality is already low, enforcing regeneration attacks that further degrade image quality seems unrealistic in real-world scenarios. Given that PSNR and SSIM are pixel-level metrics, such degradation is expected after multiple perturbations in the semantic space. I wonder if the semantically meaningful content is still preserved after regeneration attacks capable of breaking StegaStamp. Specifically, a visualization for Figure 5 or an LPIPS curve might be helpful for understanding the preservation of semantic content. -------------------Post rebuttal Edit------------------------ I appreciate the author for the comments. My questions has been well addressed. I have increased my rating accordingly.

Rating

7

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

potential negative societal impact has not been addressed but the author does provide potential defense mechanisms for the proposed attack method.

Reply to the authors

I appreciate the effort that you have put into this rebuttal. Most of my questions and concerns are addressed and I'm inclined to increase my original rating. Nevertheless, I have some further comments and questions regarding the rebuttal provided and hope to be clarified/addressed: 1. DiffPure also seems to have tuning parameters. Thus I believe it is more appropriate to compare with DiffPure by plotting the quality-detectability tradeoff just as what you did to other attacks. I would ask to put this comparison in the same quality-detectability plots. 2. If I remember correctly, you have tried to emphasized two things in your original paper: (1) the proposed regeneration attack is very effective and (2) the regeneration attack powered by diffusion model outperforms the others. However, based on your updated quality-detectability tradeoff, I see that regeneration powered by **VAE** seems to be the best performing one. Is this understanding correct? If so, I would ask the authors to adjust their related comments/conclusions in the paper accordingly. 3. I understand that the authors' emphasis of this paper is the "removability", and the cost of quality loss to remove the watermark is secondary consideration. However, based on the additional results provided in the rebuttal, the cost to remove some watermarks (e.g., StegaStamp) tends to be large (e.g., PSNR $\sim$ 23.33 by Regen-Diff; also the visible flaws shown in the visualization). I hope to see some discussion related on how the "removability" and "cost to remove" trade-off can potentially imply for practical scenarios. For example, how this result support/concerns the application of watermark (copyright protection, preventing image misuse, etc); possible directions to counteract your finding if watermark is not safe let's say; or any idea for the future design of practical watermarks that you can suggest. Overall, if the above points can be further clarified/addressed, I tend to change the rating to accept.