Summary
The paper reviews the state of current virtual try-on technologies and identifies a key issue: high-quality and realistic image generation rely on resource-intensive models trained on extensive datasets, with existing diffusion-based methods involving up to 800 million parameters. This results in considerable training costs and computational demands that may deter users from trying on clothing with regular poses. Current methods also require diverse inputs that may be burdensome for non-professional users. In contrast, the paper introduces DiffusionTrend, a new approach that simplifies the virtual try-on process by forgoing extensive training on large datasets and avoiding complex preliminary input processing steps. The proposed method uses a lightweight CNN for feature-level clustering, producing effective masks without intensive training, and applies DDIM inversion to integrate detailed garment features into model images with background coherence.
Strengths
1. The paper addresses the significant training cost issue of existing virtual try-on models and investigates low-cost methods for conducting try-ons, a valuable subject. The authors explore a viable approach to perform try-ons through the replacement of latents. Unfortunately, the method seems somewhat rudimentary, and there is room for improvement in the quality of the results.
2. The paper is well-articulated, and the data presented is credible.
Weaknesses
1. The statement in line 161 "Traditional segmentation models (He et al., 2017; Kirillov et al., 2023) are typically employed to generate masks. However, in environments where numerous users engage in online consultations simultaneously, computational efficiency becomes crucial. For instance, Segment Anything (Kirillov et al., 2023), while effective, incurs a significant computational cost , which can be impractical for real-world applications, especially when scalability and cost-effectiveness are paramount" is inaccurate. In fact, input model images and input garment images can be automatically preprocessed through existing pretrained human-parsing models to obtain denpose, openpose, parsing images, agnostic images, etc. Please refer to the code [1][2]. Large segmentation model such as Segment Anything is NOT used. In addition, the computational overhead of image segmentation is very small compared to the diffusion denoising process.
2. There are many existing warping methods, yet this work has chosen to use a simple network to observe the mask and employ perspective transformations to deform the clothing. This does not simulate the myriad of clothing deformations that occur in a real try-on process, often leading to ill-fitting try-ons. In fact, since this work is divided into the warp cloth and diffusing process parts, it's necessary to show and discuss the effects of the warp cloth and beneficial to compare it with explicit warping models.
3. The inference speed of this method should be much slower than typical methods (possibly three times the duration), if I have not misunderstood, as the DDIM inversion takes nearly the same number of steps as the DDIM sampling.
4. The qualitative comparison in Fig. 4 appears to show that the results produced by DiffusionTrend have noticeably oversaturated colors.
[1] https://github.com/sangyun884/HR-VITON/issues/45
[2] https://github.com/levihsu/OOTDiffusion
Questions
1. We observed background distortion and altered color without restoration. However, according to Eq. 8, the background and foreground are treated equally, does this mean that the clothing is equally likely to experience severe distortion? There is no discussion in the article concerning this issue.
Ethics concerns
No ethics review is needed.