Summary
Denoising diffusion probabilistic models (DDPMs) have advanced high-fidelity image generation, but they struggle with high resolutions due to attention mechanisms. The paper introduces Diffusion State Space Model (DiffuSSMs) to replace attention with a more efficient approach. DiffuSSMs excel in fine-grained image representation, outperforming some previous models in FiD and Inception Score on ImageNet with reduced computational load.
Strengths
1. The rationale for substituting the attention mechanism with a more efficient network structure to alleviate computational burdens in high-resolution image generation is sound.
2. Furthermore, the proposed method demonstrates commendable performance on 256x256 images.
Weaknesses
1. The article's motivation is to enhance the accuracy and computational efficiency of high-resolution image generation. However, the results on the ImageNet 512×512 Benchmark in Table 1 reveal a performance degradation of the proposed method compared to DiT-XL/2-G, along with increased computational resource consumption relative to U-ViT/2-G. Notably, the absence of comparisons with state-of-the-art methods for 512x512 resolution, such as Stable Diffusion, raises questions. To substantiate the claim that the proposed method is suitable for high-resolution generation, additional comparisons at higher resolutions, like 1024x1024, should be included.
2. It appears that the author has transplanted the network structure from Bidirectional Gated SSM without adequately explaining the task-specific adaptations and improvements. This lack of clarity can be perceived as a limitation in terms of contributions to the research community.
3. The ablation studies presented a lack of rigor. Table 2 primarily focuses on changes in loss without directly demonstrating the efficiency gains and quality degradation resulting from "COMPRESSION IN LATENT SPACE." This makes it challenging to assess the true effectiveness of this component. Additionally, relying solely on visual representations may be insufficient, especially given the stochastic nature of the diffusion model.
Questions
1. In the ImageNet 256×256 Benchmark, both ADM-U and CDM exhibit lower FID scores than the proposed method. This raises questions about the emphasis placed on the proposed method's performance.
2. The presence of two DIFFUSSM-XL (or DIFFUSSM-XL(G-1.5)) entries in the ImageNet 256×256 Benchmark necessitates clarification regarding the distinctions between these two models.
3. The presentation of Table 2 raises concerns due to the inconsistent use of bold formatting for metrics, which can be confusing for readers.
Rating
3: reject, not good enough
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.