Is One GPU Enough? Pushing Image Generation at Higher-Resolutions with Foundation Models

In this work, we introduce Pixelsmith, a zero-shot text-to-image generative framework to sample images at higher resolutions with a single GPU. We are the first to show that it is possible to scale the output of a pre-trained diffusion model by a factor of 1000, opening the road for gigapixel image generation at no additional cost. Our cascading method uses the image generated at the lowest resolution as a baseline to sample at higher resolutions. For the guidance, we introduce the Slider, a tunable mechanism that fuses the overall structure contained in the first-generated image with enhanced fine details. At each inference step, we denoise patches rather than the entire latent space, minimizing memory demands such that a single GPU can handle the process, regardless of the image's resolution. Our experimental results show that Pixelsmith not only achieves higher quality and diversity compared to existing techniques, but also reduces sampling time and artifacts. The code for our work is available at https://github.com/Thanos-DB/Pixelsmith.

Paper

References (58)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer snt95/10 · confidence 4/52024-07-04

Summary

In this paper the authors propose a number of engineering tricks which enable generating at higher resolutions from a pre-trained txt2img diffusion model. Notably, the requirements for the proposed method are relatively low. In the proposed approach, an image of a standard resolution is generated first. After that, is it upsampled (in RGB space) to the desired size and used as a guidance for the new, truly high-resolution image which is generated as a sequence of overlapping patches. The guidance mechanism is implemented by mixing the imaginary component of Fourier decomposition of the currently denoised and guidance patches. This procedure takes place not for all the denoising steps, but only up to certain noise level referred to as Slider. The value of Slider controls how similar are the original standard-resolution image and its higher-resolution version. Performance of the method is evaluated with commonly used metrics such as FID, KID, IS and CLIP-similarity. Also, a lot of samples are provided for visual inspection.

Strengths

According to the provided metrics and visual results, the methods performs quite well in comparison with recent baselines. The fact that it requires only 7.4 GB of memory makes it very affordable for the community. The idea of mixing in the spectral domain, to the best of my knowledge, is novel enough in context of ultra-high resolution sampling from a pre-trained model. Exploiting the checkerboard pattern for mixing with guidance is also an interesting engineering trick.

Weaknesses

1. While the authors several times mention that "the imaginary part in the frequency space contains most of the low-frequency information of the image" (line 187) and "imaginary part provides more structural information than the real part" (line 249), they do not provide any references for this statement or supporting experiments. To me, this fact does not look evident enough. I am sure this needs more thorough explanation since otherwise it looks just like an empirical trick. 1. From the text, it is unclear which Slider value was used to obtain the numbers from Table 2. Was it the value of 30, as defined in line 244? Also, what exactly is "using next inference step (NIS)" (see line 253)? Is this about checkerboard mask mixing? 1. The presentation of Fig. 3 is not easy for understanding, I advise considering its redesign. For example, forward diffusion and taking imaginary part of FFT are denoted in the same way. 1. It is unclear from the text what the guidelines are for selecting the size for the zone of averaging for overlapping patches (line 805).

Questions

I ask the authors to address the weaknesses listed above during the rebuttal period. In particular, I am interested in the deeper justification of using imaginary Fourier coefficients than empirical evidence. POST-REBUTTAL UPDATE: I keep my initial score. I think that although the method is very simple, this does not mean that the submission is bad. I believe that with improved presentation this paper can be interesting for the practitioners.

Rating

5

Confidence

4

Soundness

3

Presentation

3

Contribution

2

Limitations

No actions needed.

Reviewer T7y46/10 · confidence 4/52024-07-10

Summary

This paper proposes a training-free method for diffusion models to sample images at higher resolution with limited GPU memory, which introduces several tricks, such as fourier merging, chess-mask deduplication, and slider control. Experiments show the effectiveness of the proposed method.

Strengths

1. The fourier merging seems interesting with imaginary part to maintain the global structure; 2. The chess mask seems effectively eliminates the duplicate artifacts in high resulotion generation; 3. The ablations are sufficient and comparisons show the superiority of the proposed method.

Weaknesses

1. Is there any literatures to support that the imaginary part of the fourier transform corresponding to the low-frequency of the signal, or is there any deeper analysis beyond the ablation results in Tab.1 to illustrate this? By the way, it is suggested to provide the visual ablations of this part.

Questions

1. How to determine the patch orders in high-resolution generation, is it random orders or follow some principles. 2. From Tab. 2, it is not clear why Pixelsmith slower than Scalecrafter at 2048 resolution, while nearly 2x faster at 4096 resolution. Is there any trend that the bigger, the more efficient? 3. Is the content of the final scaled high resolution output significantly differ from the base output, whether the new objects will be generated?

Rating

6

Confidence

4

Soundness

4

Presentation

3

Contribution

3

Limitations

The authors have adequately discussed the limitations of their work.

Reviewer wkBK3/10 · confidence 4/52024-07-13

Summary

The paper introduces Pixelsmith, a framework designed to utilize pre-trained diffusion models to enable high-resolution image generation using only a single GPU. Patch-based denoising ensures that the entire generation process can be accommodated on a single GPU. The Slider mechanism balances the trade-off between finer details and overall structure by controlling the transition from guided generation to native generation. Guided generation operate on the latent fused by one upscaled higher-res image patch and one native-generated higher-res image patch in Fourier space, helping to maintain global structures as claimed by the authors. Experimental results show that Pixelsmith not only produces high-quality and diverse images but also improves sampling time and reduces artifacts compared to existing techniques.

Strengths

1. Memory Efficiency with Limited Computational Resources: Achieving high-resolution image generation on a single GPU is a good contribution, and the image generation quality is not compromised by this restricted setting. 2. High-Quality Generation: The proposed Fourier space fusion maintains fine-grained details and global structures, while also preventing some artifacts that occur in other methods.

Weaknesses

1. Paper Structure Issues: The content before the methods section is too lengthy, making the key method and experiments sections comparatively short and harder to fully understand. These are elaborated upon in the following points. 2. Method Description is Hard to Follow: For instance, the overview in Lines #171-173 does not fully describe all the components involved in the method, requiring readers to review the paper multiple times to understand the entire pipeline. Additionally, the intuition behind combining $\hat{z}^{iFFT}{t-1}$ and $\hat{z}^{guid}{t-1}$ is not explained, which is confusing since $\hat{z}^{iFFT}_{t-1}$ already incorporates information from $\hat{z}^{guid}_t$. 3. Errors and Mismatches in the Experiment Section: According to Table 1, using the real part is better, which contradicts Line #248. Line #252 describes the wrong column for "the next inference step". Also, the term “base model” in Table 2 is unclear—does it refer to the same model as Table 2’s Pixelsmith and Figure 1’s base model? 4. Flaws in Figure Illustrations: In Figure 1, the base model image for “x256 (16384x16384)” is not displayed. Is this base model the same as the one mentioned in Table 1? In Figure 3, the illustration should be more concrete and easier to understand; however, some variable names are too small and there are too many blank areas. The fusion part is also unclear, with no mention of the real part of the Fourier transform and no explicit explanation of $\mu( . , . )$.

Questions

The aim of achieving high-resolution image generation on a single GPU is commendable, and the proposed method successfully accomplishes this goal. However, the presentation quality is relatively poor, especially with some key parts described incorrectly. I hope the authors can address the questions mentioned in the weaknesses section and resolve my concerns.

Rating

3

Confidence

4

Soundness

2

Presentation

1

Contribution

2

Limitations

The authors have discussed the trade-off between achieving finer details and suppressing artifacts. They have also suggested proposing appropriate metrics for evaluating high-resolution image generation, which would be beneficial for the community.

Reviewer SvAB4/10 · confidence 5/52024-07-18

Summary

This paper introduces a framework for generating high-resolution images from text prompts using pre-trained diffusion models. The key innovations are: A cascading approach that uses lower-resolution generated images as guidance for higher resolutions. A "Slider" mechanism to control the balance between following the guidance and allowing novel generation. Patch-based denoising to enable generation of arbitrarily large images on a single GPU. Averaging techniques to reduce artifacts from patch-based generation. The authors demonstrate that Pixelsmith can generate images up to 32,768 x 32,768 pixels on a single GPU, outperforming existing methods in terms of quality and efficiency.

Strengths

The proposed method enables generation of ultra-high resolution images without additional training, addressing an important limitation of current models. Good results. The patch-based approach allows for generating massive images on consumer GPUs, which is a significant practical advantage.

Weaknesses

1. Poor presentation. The writing of this paper needs substantial improvement to be published in a venue like NeurIPS. First, the language of the paper needs to be improved. Second, the paper spends a lot of space on unnecessary content. The method in this paper is not difficult, but it is very difficult to understand the method in one reading. In terms of writing, the authors did not succeed in emphasizing the core concept of their proposed method. In addition, the length of the paper is too long. Such a simple method, but the experiments are not introduced until page 8, which should not happen. 2. The core idea of ​​this paper seems to be to generate once and then upsample the generated image by patch-wise processing. (By the way, Figure 3 is difficult to understand and Eq. 4 is not explained). This method is essentially not much different from other operations that use diffusion for upsampling. So what is the advantage of this method? Why is this method feasible? 3. The role of FFT Transformation is not well demonstrated. No exploratory experiments are used to demonstrate the motivation and effect of this method.

Questions

See Weaknesses please.

Rating

4

Confidence

5

Soundness

2

Presentation

1

Contribution

2

Limitations

The authors claim they discussed the limitation but I didn't find it in the text. Correct me if I am wrong.

Reviewer SvAB2024-08-13

Response to the rebuttal

I have read the author's response, as well as the comments and discussions with other reviewers. The author has partially addressed my concerns. However, I still think that the presentation of the paper is lacking at this stage. I will improve my score. However, since I cannot see the revised paper, I cannot judge whether the final presentation meets the requirements of NeurIPS.

Authorsrebuttal2024-08-14

Thank you for the comment! We appreciate the opportunity to continue the discussion. > The author has partially addressed my concerns. Can you please tell us which concerns have not been addressed so we can attend to them as well? > However, I still think that the presentation of the paper is lacking at this stage. In our original reply, we kindly asked for specific examples where the language was not up to the NeurIPS level, as it is important to us to have this constructive feedback. Could we please extend the original request here and ask for specifics on where the paper is lacking as well? We have revised the entire paper and followed all suggestions from all reviewers. It would greatly benefit our work to know which parts are lacking, as we understand that this is the main (remaining) issue with the review. Examples of language before the revision (referring to W1) and the presentation after all the implemented changes would help our work, as we believe in the NeurIPS review process and the best practices to help promote research. > However, since I cannot see the revised paper, I cannot judge whether the final presentation meets the requirements of NeurIPS. As we cannot upload the full paper, I can kindly point you to the Method section, which is in the global response, and the Experiments section, which is a reply to reviewer wkBK. The Related Work and Foundations sections that were changed can be found below. The rest of the paper remains unchanged, so the full paper is now shown in our replies. > > # Related Work > Pre-trained DMs are following a ... each new version [30]. It is clear that there is demand for increasingly higher resolution generation. > Currently, generating images ... application. > ## Trained models > (unchanged) > ## Adapted models > (unchanged) > # Foundations > ## Diffusion models > DMs [15,42] are probabilistic generative models that first add noise to a distribution during diffusion and then learn to remove this noise during denoising. During training, a Gaussian probability distribution is learned, and during inference, sampling from the Gaussian leads to the data probability distribution. Executing this process in the latent space [37] is more resource-efficient, allowing for faster training and inference times. > In formal terms, for a Latent Diffusion Model, if $z_0$ represents the data point in the latent space, given $z_0 \sim q(z_0)$ and $q$ being the diffusion process, then for timesteps t $\in$ $\{1,T\}$ $z_1,\ldots,z_T$ noisy latents, with variance $\beta_t \in (0,1)$, are produced, defining a joint distribution conditioned on $z_0$: > >(unchanged equation 1) > >(unchanged equation 2) > > The training estimates an isotropic Gaussian distribution for $z_T$. During the denoising process we sample from $z_T \sim \mathcal{N}(0,{I})$ and remove the added noise using a neural network: > >(unchanged equation 3) > ## Patch sampling > The default denoising process of an LDM involves sampling the entire latent space at each timestep. While this approach works for lower resolutions, it becomes increasingly resource-intensive as the resolution increases. > Instead, we modify the default process to denoise patches of a fixed dimension $128^{2}$ as introduced by DiffInfinite [1]. At each timestep, random patches are selected for denoising, and this process is repeated until the entire latent space is denoised (see Appendix Patch sampling). The DiffInfinite process relies on segmentation masks to condition each individual patch, providing rich spatial information. In a text-to-image DM, where the text-prompt is global for the entire latent space, using this method means that each patch is denoised with the same condition—the text-prompt. This leads to multiple repetitions of the condition and results in poor-quality generations. To address this, we implement a series of key components that enable scaling a pre-trained DM to resolutions never achieved before. The above changes result in considerably smaller pre-Method sections, allowing the new, better-explained Method section to fit and enhance readability. The previous full text of this section will become a new appendix, titled "Patch sampling," with the addition of our explanation to question 2 of Rv T7y4. Nothing else changes or is added, so this appendix will not be new to you.

Reviewer snt92024-08-09

I would like to thank the authors for their feedback.

Reviewer T7y42024-08-14

Official Comment by Reviewer T7y4

Thanks for the authors rebuttal. Most of my questions have been addressed, however, the reason why the imaginary part of the fourier transform corresponds to the low-frequency of the signal is still not illustrated well with deeper analysis, and depend on the ablation experiments, which may undermining the potential inspiration, therefore, I will not raise my score.

Authorsrebuttal2024-08-14

Thank you for the comment! We appreciate the opportunity to continue the discussion. >Most of my questions have been addressed, however, the reason why the imaginary part of the fourier transform corresponds to the low-frequency of the signal is still not illustrated well with deeper analysis, and depend on the ablation experiments, which may undermining the potential inspiration As stated in our global response, all the references have been changed to reflect the concerns raised. For example, in Lines 187 and 248, where it was previously mentioned that the imaginary part of the Fourier transform corresponds to the low-frequency information of the signal, this has now been removed. In our revised paper, we do not state a connection between the imaginary part and the low-frequency information and we base our choice of the imaginary part on the results of the ablation experiments.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC