Summary
This paper presents GS-Hider, a novel framework for steganography in 3D Gaussian Splatting (3DGS) models. The key innovation is a coupled secured feature attribute that replaces the original spherical harmonics coefficients, allowing the embedding of hidden 3D scenes or images into the original scene without compromising rendering quality. The framework uses a scene decoder and a message decoder to disentangle the original and hidden information. The authors demonstrate the effectiveness of GS-Hider across various experiments, showing its ability to hide multiple 3D scenes or single images while maintaining high fidelity, security, and robustness.
Strengths
Novelty: GS-Hider presents the attempt at steganography for 3D Gaussian Splatting, addressing an important challenge in protecting 3D assets.
Technical innovation: The coupled secured feature attribute and parallel decoder architecture effectively balance security, fidelity, and computational efficiency.
Versatility: The method can hide both 3D scenes and 2D images, demonstrating flexibility for various applications.
Comprehensive evaluation: The authors provide extensive experiments on fidelity, security, robustness, and capacity, comparing against baselines and ablation studies.
Real-time capability: Despite the additional complexity, the method maintains near real-time rendering speeds (45 fps), which is crucial for practical applications.
Weaknesses
Main weakness:
* The main weakness is the additional computational overhead introduced by using the scene decoder. Gaussian splatting significantly simplifies the neural rendering paradigm of NeRF into a point vector + rasterization approach, achieving very fast rendering - this is one of the most valued aspects of GS. However, the introduction of the scene decoder greatly undermines this advantage. This makes such an improvement meaningless. I am confident in this inference because the authors do not seem to report any FPS-related metrics, nor do they compare the FPS increase with the original 3DGS.
* The decoder is trained per scene, rather than being a generalizable decoder, which suggests it may be "memorizing" scene watermarks. This can be inferred from the issues involved in training a decoder for each scene and hiding only a single scene per scene. At the same time, the geometrically inconsistent signals present in the cover scene and hidden scene are well accommodated together, further indicating that the signals are to some extent being "memorized" in the decoder. Therefore, I speculate that the capacity of the decoder is not small. This further corroborates the aforementioned weakness.
Some other weakness:
* While the empirical results are strong, there's a lack of theoretical justification for why the coupled feature representation works so well.
* Comparison to recent work: The paper could benefit from comparing against more recent steganography methods, particularly those designed for other 3D representations that might be adaptable to 3DGS.
* Generalization: The experiments are limited to a single dataset (Mip-NeRF360). It would be valuable to see how the method performs on a wider range of 3D scenes and different types of hidden information.
Questions
Main issues that could be considered in rebuttal：
* what's the added inference time overhead due to using the scene decoder?
* the fps metrics for all experiments, so that we can see the impact on the practical usage. If the rendering speed is affected, the improvement about privacy is not useful.
* try to prove that the hidden scene is not simply memorized by the decoder. maybe trying to memory more than one hidden scenes or curating a general decoder that can be applied to not only one scene can be convincing
Other issues:
please see the weakness
Limitations
The authors provide a brief discussion of limitations in Section B of the appendix, acknowledging issues such as the lack of view-dependency and slightly reduced rendering quality. They also mention future work directions, including enhancing model expressiveness and extending to tasks like tampering detection. However, this discussion could be expanded to more thoroughly address potential failure modes or edge cases of the proposed method.
The broader impacts section (A.2) touches on both positive applications (e.g., copyright protection) and potential risks (sensitive data concerns). However, a more in-depth exploration of potential misuse scenarios and mitigation strategies would strengthen the paper's ethical considerations.