Summary
Embedding data within deep neural networks (DNNs) has seen significant success, spanning both discriminative and generative models; however, image embedding within diffusion models remains underexplored. Current methods struggle with limitations in fidelity, secrecy, and efficiency, particularly due to the high computational cost of embedding and the slow extraction process required by multiple denoising stages, rendering these techniques impractical in resource-constrained settings. To overcome these challenges, this work proposes embedding images at a specific denoising stage within diffusion models by adjusting the learned score functions. Additionally, a parameter-efficient fine-tuning (PEFT) strategy, incorporating selective parameter tuning with a low-rank adaptation (LoRA) variant, is introduced to enhance both secrecy and efficiency. Experimental results substantiate the effectiveness of the proposed method.
Weaknesses
1. The novelty of the proposed method is limited, as techniques like fine-tuning the U-Net of diffusion models to embed specific responses or remove certain content have already been extensively explored in related work (e.g., [1]), even though this reference is not cited.
2. Embedding information through the weights of diffusion models poses practical challenges; these weights typically range between 5GB and 10GB, which results in a considerably higher transmission load compared to image-domain steganography methods.
3. Approaches such as [2] that apply steganography directly within the parameter space of models should also be included in the comparisons to offer a more comprehensive evaluation.
4. Comparisons with image-based steganography methods may not be entirely fair, as these methods prioritize maintaining similarity between the cover and steganographic images, which could impact their performance metrics in steganographic capacity or secrecy.
5. Further evaluations on additional open-source diffusion models are recommended to strengthen the assessment and generalizability of the proposed approach.
[1] @InProceedings{Gandikota_2023_ICCV,
author = {Gandikota, Rohit and Materzynska, Joanna and Fiotto-Kaufman, Jaden and Bau, David},
title = {Erasing Concepts from Diffusion Models},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2023},
pages = {2426-2436}
}
[2] @inproceedings{cui2024steganographic,
title={Steganographic Passport: An Owner and User Verifiable Credential for Deep Model IP Protection Without Retraining},
author={Cui, Qi and Meng, Ruohan and Xu, Chaohui and Chang, Chip-Hong},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={12302--12311},
year={2024}
}