FreqMark: Invisible Image Watermarking via Frequency Based Optimization in Latent Space

Summary

This paper considers the problem of watermarking of images. In particular, it introduces a method called FreqMark where watermark is embedded in the latent frequency space obtained after variation auto encoder (VAE) encoding. Numerical results are carried out on test datasets containing 500 randomly selected images from the ImageNet and 500 images from the DiffusionDB dataset. Comparative results against DwtDctSvd, Stable Signature and SSL indicate that FreqMark exhibits superior quality-robustness tradeoff compared to the benchmark methods.

Strengths

The main strengths of this paper are the novelty of the proposed FreqMark algorithm and the superior performance of FreqMark compared to the baseline image watermarking methods. The main idea behind FreqMark is hiding the message bits in the FFT outputs of latent vectors coming from the VAE. In numerical experiments on two image datasets (ImageNet and Diffusion DB), authors show that FreqMark provides a bit accuracy of better than 90% for a 48-bit encoding setting. The Ablation Studies investigating image quality, number of encoding bits, noise levels and spatial perturbations are reasonably convincing of the benefits offered by FreqMark. The Appendices provided contain significant additional information regarding FreqMark and is one of the strengths of this paper.

Weaknesses

One of the main weaknesses of the work is that there is no convincing proof or theoretical justification of why hiding watermark bits in the FFT of latent space should be beneficial. The usual argument of using frequency domain for hiding watermark bits makes sense in that the FFT is applied to images (i.e., image pixels) and captures the spatial relationships between the image pixels. What kind of "spatial" relationship is captured by the FFT in the latent space and why is that beneficial for hiding watermark? FFT produces complex values, yet there is no discussion of how the watermark bits are embedded on these complex values. In Fig. 1, the outputs of FFT are displayed as magnitude images and if this is what is intended, what happened to the phase of the FFT outputs.

Questions

1. Are the FFT outputs complex and if so, how are the watermark bits being embedded? 2. What do FFT outputs represent when applied to the latent vectors? When applied to 1D time signals or 2D images, the FFT output describes the frequency content of the signals and images. What does the FFT of latent space represent and why is it useful?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

Authors mention that there is room for optimization in the fusion of images and watermarks. It would have been useful to have a more detailed discussion of this aspect of the proposed method.

Summary

This paper introduces FreqMark, a novel invisible watermarking method that enhances digital content protection through optimization in the image's latent frequency space. Experiments have been conducted to demonstrate the robustness against regeneration attacks like VAE and diffusion model.

Strengths

1. The paper is well-written and well-organized. 2. Extensive experiments have been conducted to demonstrate the robustness against various attacks including VAE and diffusion models.

Weaknesses

1. Lacks some theoretical analysis. In the experiments, do you need to use the same VAE model for the attack as you have used during watermarking training? If not, then an analysis may be needed as to why it is still robust against attacks with different VAE models. 2. Since the proposed method requires case-by-case optimization, what is the watermarking time for each case, and how does it compare to other competing methods? 3. Why do we need a set of pre-trained N-dimensional direction vectors, but not directly produce the message? 4. In Equation (4), why $k \in N$? $N$ is a number, not a set, so $\in N$ is not appropriate. Besides, using $v^N$ to denote a vector set and $v^k$ to represent a vector can lead to confusion. In Figure 2, there are $K+1$ vectors in the set, I think $K+1$ is the bit length. 4. Minors: - $\epsilon_1$ and $\epsilon_2$ (or $\varepsilon_1$ and $\varepsilon_2$) appear in Figure 2, so it is better to mention them in the caption of Figure 2. - In line 213, page 7, the word 'We' should be 'we'.

Questions

Please see the Weaknesses.

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

3

Limitations

The authors have addressed the limitations in their paper.

Summary

This paper proposes a method called FreqMark that is able to prevent the invisible watermarks from the regeneration attack. By using the unconstrained optimization of the image latent frequency space obtained after VAE encoding, the proposed FreqMark achieves better robustness against the regeneration attacks and traditional attacks.

Strengths

The proposed method achieves better robustness against regeneration attacks. By using a newly proposed optimization strategy, the proposed method achieves a balance between the image quality and the envisioned robustness.

Weaknesses

1. The major contribution of this paper is to introduce a kind of image watermark robust to regeneration attacks. However, this part is not highlighted in its title. The authors are suggested to consider this in the title. 2. The proposed methodology follows an established manner with incremental innovations. The whole approach does not demonstrate enough differences when compared with the previous approaches. 3. For the second contribution, the authors claim that their proposed framework is flexible and then say such flexibility guarantees a trade-off between the bits number, image quality, and robustness. However, why can such properties be considered as flexibility? 4. The whole pipeline presented in Figure 2 lacks novelty. This is just a very common encoder and decoder framework. The authors need to better consider their contributions for this part.

Questions

I have shown my concerns in the weakness part. The authors are suggested to consider them during the rebuttal.

Rating

4

Confidence

3

Soundness

3

Presentation

2

Contribution

2

Limitations

Please address my concerns within the weakness part.

Summary

The authors propose a new post-processing watermark for imagery, FreqMark. The FreqMark embeds a binary message into the frequency domain of a VAE-encoded image via a small perturbation, and then following IFFT + decompression, utilizes a pre-trained image encoder to extract the message. A PSNR + LPIPS metric is used to protect mage quality against the message embedding. The dual approach of VAE encoding + Gaussian noise makes the mark resilient against regeneration and Gaussian noise attacks. The method demonstrates high bit accuracy and good image quality when compared against DwtDct, SSL, Stable Signature on watermarked DiffusionDB & Imagenet images.

Strengths

-Easy-to-follow narrative and nice motivation in the age of generative imagery. The intuition behind the FreqMark algorithm is sound, easy to follow, and presumably simple to implement. -Method demonstrates robustness against regeneration, which is a very potent attack.

Weaknesses

This work is missing two critical competitor baselines which must be addressed. -In my opinion, the FreqMark is an incremental variant of the StegaStamp [1]. Surprisingly, the authors did not compare against this method even though it is well-known in existing literature that the StegaStamp is robust against many attacks, including regeneration [2]. Like FreqMark, the StegaStamp increases resilience against attacks by incorporating them into the training pipeline and uses a critic loss to preserve image quality (LPIPS + L2, versus LPIPS + PSNR for FreqMark). The resemblance of equation (10) in this manuscript to the loss function equation (2) in [1] begs the question of novelty. It is also incorrectly stated in line 71 that the StegaStamp only relies on differential image perturbations for the training pipelines -- in fact, any attack can be added, as the decoder is trained after image manipulation. -Again, if the spirit of the paper is to increase resilience against regenerations, the authors also needed to compare against the state-of-the-art Tree-Ring watermark [4], which was noted to be incredibly resilient against regenerations by the authors of the regeneration attack [2]. As the Tree-Ring is an in-processing technique that embeds a message within a diffusion process, one way to set up a comparison is to post-process a collection of Tree-Ring watermarked images via FreqMark, and then independently extract both watermarks. -As noted in [3], there is no single perceptual metric that is an objective measure of image quality, thus low PSNR or LPIPS distance does not necessarily indicate the method is not introducing artifacts. The authors need to add 1-2 more metrics (maybe L2 and FID, for example) for a more convincing argument. -500 images is too small a sample size for the tested FPR thresholds. Modern literature in this field such as [3,4,5] are using several thousand images. -VAE regenerations are far weaker compared to diffusion regenerations. Readers will want to see how the FreqMark holds up against longer, deeper regenerations (>= 100 steps) to see how the decoding accuracy is affected. -As observed in [3], the use of publicly available VAEs to encode/decode watermarks is easily defeated if the attacker uses a regeneration leveraging encoders/decoders with the same architecture. [1] Tancik, M., Mildenhall, B., & Ng, R. (2020). Stegastamp: Invisible hyperlinks in physical photographs. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 2117-2126). [2] Zhao, X., Zhang, K., Su, Z., Vasan, S., Grishchenko, I., Kruegel, C., ... & Li, L. (2023). Invisible image watermarks are provably removable using generative ai. arXiv preprint arXiv:2306.01953. [3] An, B., Ding, M., Rabbani, T., Agrawal, A., Xu, Y., Deng, C., ... & Huang, F. (2024). Benchmarking the robustness of image watermarks. arXiv preprint arXiv:2401.08573. [4] Wen, Y., Kirchenbauer, J., Geiping, J., & Goldstein, T. (2023). Tree-ring watermarks: Fingerprints for diffusion images that are invisible and robust. arXiv preprint arXiv:2305.20030. [5] Saberi, M., Sadasivan, V. S., Rezaei, K., Kumar, A., Chegini, A., Wang, W., & Feizi, S. (2023). Robustness of ai-image detectors: Fundamental limits and practical attacks. arXiv preprint arXiv:2310.00076.

Questions

1. See weaknesses. 2. Which version of Stable Diffusion was used for the regeneration attack?

Rating

6

Confidence

4

Soundness

3

Presentation

3

Contribution

1

Limitations

Adequately discussed.