Summary
The paper focuses on enhancing Masked Generative Models (MGMs) for image synthesis.
The authors identified several reasons for the underperformance of MGMs: 1) lack of sequential dependencies, 2) multi-modality problem, 3) compounding decoding errors, 4) limitations of low-temperature sampling and 5) ineffective guidance techniques.
To address these issues, the authors present novel techniques and methodologies to improve their performance.
The main contributions of the paper are as follows:
1) Generalized Guidance Formulation.
2) Self-Guidance Sampling Method.
3) Parameter-Efficient Fine-Tuning.
Strengths
- Originality
The paper presents a highly original approach by extending guidance methods from continuous diffusion models to MGMs. In particular, the use of self-guidance sampling method and an auxiliary task for semantic smoothing in the VQ token space are novel approach that address specific challenges in MGMs.
- Quality
The paper demonstrates a high level of technical quality. The authors have thoroughly explained why their methods can enhance performance and have supported their claims with extensive experiments and ablation studies on various variables.
- Clarity
The paper is clearly written and well-structured. The authors provide detailed context relative to prior work, facilitating a clear understanding of the background and contributions of the proposed approach. The detailed explanations of the equations and methodologies, accompanied by ample examples, significantly enhance the overall readability and accessibility of the paper.
- Significance
The contributions of the paper are significant for the advancement of the MGMs field. By addressing key challenges in MGMs, the proposed methods offer substantial improvements in the quality and diversity of generated samples.
Weaknesses
Particularly due to inadequate visual results, some aspects are difficult to comprehend. Section 3.3 asserts that TOAST is the most suitable model for the study's task, but lacks sufficient explanation on why TOAST is the only suitable model. It would be more convincing if the paper showed some visual results from using other models, which would provide sufficient justification.
Explanations for the choice of hyperparameters are insufficient. While the paper demonstrates trade-off relationships in hyperparameter settings across some datasets in removal experiments, the sampling time steps do not exhibit such trade-offs. It is crucial to provide adequate explanations for why these settings differ from those maintained for the ImageNet 512×512 dataset.
Questions
Most hyperparameters in Figure 5 show a trade-off relationship between FID and IS scores. However, increasing the number of sampling time steps improves performance in both metrics. So, could you explain why sampling time steps were set to 12 and 18 in each experiment? Even though other methods are known to use hundreds of sampling time steps, 50 seems quite small in comparison.
Limitations
The authors did not fully address the limitations of the paper but did mention the need for attention to AI ethics due to the rapid growth of generative models, touching on the societal impact. To improve the paper, we suggest the following:
1) Although ImageNet is a comprehensive dataset, it is insufficient to prove generalizability. Therefore, conducting experiments on other datasets used in existing MGMs, such as MSCOCO or Conceptual Captions, will make the paper more complete.
2) Include more visual results. To clarify the reasons for selecting specific models or hyperparameters used in the paper, it is much more effective to show them through visual results.