PrimDiffusion: Volumetric Primitives Diffusion for 3D Human Generation

We present PrimDiffusion, the first diffusion-based framework for 3D human generation. Devising diffusion models for 3D human generation is difficult due to the intensive computational cost of 3D representations and the articulated topology of 3D humans. To tackle these challenges, our key insight is operating the denoising diffusion process directly on a set of volumetric primitives, which models the human body as a number of small volumes with radiance and kinematic information. This volumetric primitives representation marries the capacity of volumetric representations with the efficiency of primitive-based rendering. Our PrimDiffusion framework has three appealing properties: 1) compact and expressive parameter space for the diffusion model, 2) flexible 3D representation that incorporates human prior, and 3) decoder-free rendering for efficient novel-view and novel-pose synthesis. Extensive experiments validate that PrimDiffusion outperforms state-of-the-art methods in 3D human generation. Notably, compared to GAN-based methods, our PrimDiffusion supports real-time rendering of high-quality 3D humans at a resolution of $512\times512$ once the denoising process is done. We also demonstrate the flexibility of our framework on training-free conditional generation such as texture transfer and 3D inpainting.

Paper

Similar papers

Peer review

Reviewer GMsy4/10 · confidence 4/52023-06-13

Summary

This paper introduces a method to generate various 3D human geometry and appearance. To this end, it introduces a new design of diffusion model that learns to generate the parameters of 3D primitives of humans: a generalizable encoder constructs motion and appearance features and their attentions are fed to predict scale, color, and density of each primitive. The evaluation is performed on the render people dataset, and the authors demonstrate a number of applications such as reposing and 3D inpainting.

Strengths

+ This paper effectively combines existing 3D diffusion model and existing 3D representation to perform generalizable 3D human generation. It is the first attempt to combine diffusion model and primitive representation for 3D generative humans. It can serve as good initial baseline for future research. + The 3D inpainting application is interesting, which allows reconstructing a complete 3D human from a partial body.

Weaknesses

1] The motivation of this paper is not clear. In particular, it is not clear why the volume primitive representation is ideal (other than efficiency) for handling complex human articulations. Other than computational efficiency, why does this representation fundamentally enables better quality of 3D human generation than other representation (e.g., point clouds, sdf, occupancy, and so on)? Simply showing the ablation study (without thorough analysis) does not account for this fundamental motivation, and I would like to request the authors to explain above question with fundamental reasons. Without strong motivation, this work will fall into the work that just simply combines existing 3D diffusion with existing representation as one try of a baseline. 2] Misleading claims and overclaims: 2-1) In L45, existing methods (nerf or tri-plane) do not model humans as coarse volumes. They model the geometry in the continuous space, which can lead to the fine-results. In fact, the coarseness is quite proportional to the resolution (e.g., ray resolution, volume resolution). In that sense, the proposed method could be considered as coarse reconstruction method (also, the geometry and texture quality seem coarse). 2-2) Claiming the proposed method as real-time approach sounds overstated since the generation time of diffusion model is quite slow. Please claim the computational performance in light of the whole pipeline. 3] Weak clarity in implementation. 3-1) It is not clear how the authors repose the human from the generated 3D volume primitives while preserving identity. 3-2) While the number of parameters are five, the authors describe it as six-degree of freedom. Why six-degree of freedom? 3-3) Figure 2 is not clear. The {input, output} modality of Unet should be 2D image space. However, the current modality is not image but the parameters of {T,R,s,c,sigma}. How did you handle the modality for such parameter prediction? 3-4) In L165, it is not clear how the method can handle the texture outside the SMPL body parts. 4] Low-quality results. 4-1) The quality of the generated 3D human geometry and texture is low. In particular, the high-frequency details are highly lacking for both geometry and texture. 4-2) Also, the texture is not reflective of geometry. For example, in Figure 1, the shade in the red dress of the women (of the upper right corner) does not reflect the depth information at all. 4-3) Compared to existing work [1], the 3D generation quality is low. Are they all mainly due to the weakness of the 3D primitive representation? 5] Weak validation, comparison, demonstration, and analysis: 5-1) missing baseline for 3D human generation. In particular, this paper “must” be compared to gDNA [1] paper, which can generate 3D human geometry conditioned by SMPL model whose objective is exactly same as the one in this paer. Without this comparison, this paper should not claim “state-of-the-art” method; 5-2) Since this paper claims good “computational efficiency”, it should provide the measure of inference time including denoising process, not only the rendering part; 5-3) Due to the use of synthetic data, the human generation results look always synthetic and fake. It would be nice if the authors provide the results of the model trained on real multiview data, e.g., Thuman data [2]. 5-4) In L263 and Figure 6, it is unexpected that the voxel-based approach fails that much. Please provide a thorough insight and analysis why the voxel-based approach fails. [1] gDNA: Towards Generative Detailed Neural Avatars [2] Function4D: Real-time Human Volumetric Capture from Very Sparse RGBD Sensors

Questions

Please address the questions in the weakness section.

Rating

4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

3 good

Contribution

2 fair

Limitations

This paper sufficiently discussed the limitations.

Reviewer aNCP6/10 · confidence 4/52023-07-01

Summary

The paper proposes a generative model for 3D humans. The work is based on volumetric rendering of geometric primitives and diffusion model. Given multi-view images of humans as training data, the method first learns a volumetric primitives representation, and then applies diffusion model framework to model the representation distribution. The authors evaluate their method and compare to SOTA approaches on a relatively small synthetic multi-view dataset. Results show favor towards proposed approach.

Strengths

1. The paper tackles an interesting, practical and high-impact problem: generating clothed 3D humans. 2. The paper is overall well written, easy to follow. 3. The use of cross attention to fuse pose info and texture info is appreciated and demonstrated good performance. 4. I'm glad to see the attempt of using volumetric primitives and diffusion model to create generative models of 3D humans.

Weaknesses

1. The bibliography needs to be better formatted. a) The same conference is referred in different format, e.g. CVPR vs IEEE Conference on Computer... b) Capitalization of words, e.g. 3d -> 3D, Shapenet -> ShapeNet c) Others, e.g. [30] two commas, [31] where is it published? 2. As a reference, I'd like to have a fidelity evaluation of "ground truth" volumetric primitives, better added into Table 1 in the updated version. This helps to understand the upper limit of the diffusion process in the proposed method and see where the bottle neck lies. 3. It looks like all the generated results are blurry. Is it due to the memory limit? What is the GPU memory consumption for W=32 (assuming it is the setting of all the experiments)? Is the memory consumption a limitation for the proposed method compared to EVA3D? In other words, if one had a large size of multi-view human data, would it be likely that EVA3D performs much better than the proposed method given current limit of computational power? 4. Most of the generated results presented in the submission (including the video) contain mono-color clothing. There are only two or three exceptions. Is it a bias of the dataset or is it a result of low resolution volumetric primitives? Could the authors provide a statistics on the ratio of mono-color clothing in the Renderpeople dataset? 5. What is the difference of texture transfer and reposing? It looks to me that transfer the source texture to the target subject is the same as repose (and reshape the body of) the target subject to the source pose (and source body shape). If this is true, please remove this application as it is basically in 4.4 already.

Questions

I would like the authors to answer Weaknesses 2, 3, 4, and 5. The answers may not change my final decision as long as no major flaw is found later. The lack of details in the generated results prevent me from giving strong support of the submission. But the attempt to address 3D generative human models with diffusion model and volumetric primitives should be encouraged.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

Yes, both are tackled in the submission.

Reviewer PWaT5/10 · confidence 5/52023-07-04

Summary

The paper presents a diffusion-based approach, PrimDiffusion, for 3D human generation. The method leverages the representation of MVP (mixture of volumetric primitives) and a design of volumetric primitives diffusion to generate high-quality 3D human models using multi-view images as training data. The authors propose a two-stage training process: the first stage involves learning generalizable primitives from multi-view images, and the second stage involves a diffusion process to generate/sample these primitives. The paper also presents an encoder design tailored with cross-modal attention for learning volumetric primitives from images across identities. This encoder, which consists of an appearance branch and a motion branch, is trained end-to-end across identities, eliminating the need for per-subject optimization. The cross-modal attention module within the encoder effectively captures dependencies between appearance and motion, contributing to the generation of high-quality 3D humans. The paper includes extensive experimental results demonstrating the effectiveness of the proposed method in comparison with existing techniques.

Strengths

(+) A novel way of generating 3D human models using volumetric primitives diffusion. (+) The paper is well-structured and clearly written. The authors provide a clear explanation of their method. The supplementary document provides additional technical details, and the supplementary video also helps to showcase the quality that the proposed method achieves. (+) The proposed method achieves superior results compared to existing techniques on the RenderPeople multiview dataset.

Weaknesses

(-) The proposed method is a 2-stage approach, compared to previous GAN-based methods such as EVA3D, it requires **multi-view images** as training data to learn generalizable primitives in the first stage. This is a significant difference as collecting large-scale real-world multi-view human data can be challenging, while 2D image collections are relatively easier to gather. This distinction is not clearly articulated in the paper. Considering a single-stage solution might address this issue. (-) The experiments lack a comparison with a recent baseline: Jiang, Suyi, Haoran Jiang, Ziyu Wang, Haimin Luo, Wenzheng Chen, and Lan Xu. "Humangen: Generating human radiance fields with explicit priors." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12543-12554. 2023. (-) Please refer to the questions section for more points.

Questions

- In Figure 3, top-left, the input is denoted as "Body Model $M(\beta, \theta)$", but the figure appears to depict a detailed model rather than a SMPL model. Could the authors clarify this discrepancy? - When projecting image features and pose features to the SMPL UV space, how are the **"empty"** UV regions handled? What are the initial values for the primitive positions corresponding to these empty regions? - What is the memory consumption of the proposed method during training and testing? - The training data includes 20 different human poses. How were these poses sampled? What criteria were used to ensure a comprehensive coverage of possible poses?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The proposed method requires the use of multi-view images as training data. This requirement, when compared to previous works that do not have such a constraint, should be discussed in detail as a limitation of the current approach.

Reviewer wB9H5/10 · confidence 5/52023-07-06

Summary

The paper presents, PrimDiffusion, an approach for generative modeling of 3D humans. PrimDiffusion is a diffusion-based model that operates on a set of volumetric primitives representing the human body. Each volumetric primitive consists of rotation, translation, scale, radiance, and color information. PrimDiffusion is a multi-stage approach. Given a training dataset with multiview images, the first stage fits volumetric primitives on training images using reconstruction losses. The second stage then uses the fitted volumetric primitives to train a diffusion-based generative model where the results from the first stage are used as the ground truth for the diffusion process. The use of volumetric primitives allows decoder-free rendering of the images which allows real-time rendering once the diffusion process is done. It also provides explicit control over the body pose and viewpoint. The paper also shows its application for texture transfer and unconditional infilling. Experiments are performed on the synthetic RenderPeople dataset where PrimDiffusion is shown to outperform other baselines.

Strengths

- The paper addresses the challenging problem of generative modeling of 3D humans. - The use of volumetric primitives to represent complex human bodies, clothing, and hairs makes sense and is intuitive. - The main challenge for learning 3D diffusion models is the availability of 3D ground truths. The paper overcomes this challenge by first fitting volumetric primitives on multiview images. - The paper inherits the advantages of volumetric primitives including decoder-free rendering and explicit control over viewpoint and body pose. - The paper demonstrates two interesting applications of the proposed method including texture transfer and infilling. - The paper is well-written and easy to read.

Weaknesses

### Novelty - The main novelty of the proposed method comes from the combination of existing bits and pieces. The volumetric primitives have already been used in [25] to model human faces. The paper extends it to the full body by incorporating the SMPL body model. The idea of anchoring neural primitives to the SMPL body model is similar to NeuralBody [a] `(missing citation)`, and generative modeling using diffusion models is standard practice these days. While there is no technical novel contribution of the paper, I acknowledge that the proposed combination is not straightforward and this is not the most crucial limitation of the paper. ### Comparison with existing methods - This is the most crucial limitation of the paper. The paper compares with EVA3D, StyleSDF, and EG3D. The main advantage of these methods is that they can learn 3D representation from a large collection of 2D images. However, it seems that this paper has retrained them from scratch using only the multi-view images from the RenderPeople dataset `(please correct if my understanding is wrong)`. Hence, their results are significantly worse than what is shown in the original paper, in particular the results of EVA3D. I find this comparison unfair for these methods since it suppresses their advantages and conceals the limitations of the proposed method. For a fair comparison, these methods should be fine-tuned on the RenderPeople dataset rather than training from scratch. It is the limitation of the proposed method that it requires 3D ground truths for training and hence can only be trained on multiview data. Other methods, on the other hand, can use a large collection of 2D images hence they should be used by default for these methods. ### Blurry results - The qualitative results are quite blurry. Would be great if the authors can comment on this during the rebuttal. ### Missing references [a] Implicit Neural Representations with Structured Latent Codes, CVPR'21 ### Conclusion Overall, it is a good paper with a reasonable approach. However, the experiments section of the paper is a bit weak, in particular, the comparison with existing methods is unfair. Hence, my current rating for the paper is `Weak Reject`. I would be happy to upgrade my rating if the authors can address my concerns during the rebuttal.

Questions

### Missing Experiments - The paper learns 3D primitives using reconstruction losses and multiview images. Since the ground-truth meshes are available for the RenderPeople dataset, an interesting comparison would be to use them to obtain volumetric primitives. It will help us understand how much information is lost during the learning of volumetric primitives using multi-view images. - I would have liked to see a baseline that is similar to NeuralBody i.e., replace volumetric primitives with learnable latent codes. - Please explain how the baseline methods are trained.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

- As mentioned above, reliance on 3D ground-truths is a significant limitation of the proposed method as compared to existing methods. Hence, it should be discussed in the `Limitations`.

Reviewer oDDi6/10 · confidence 4/52023-07-10

Summary

The paper introduces a novel framework called PrimDiffusion, which utilizes diffusion models to generate 3D human models. The framework leverages a Mixture of Volumetric Primitives (MVP) to represent 3D humans, enabling efficient rendering and incorporating human priors for retargeting. This representation offers a compact and expressive 2D parameter space, making it well-suited for the diffusion operation. Experimental results demonstrate that the proposed model outperforms previous approaches in terms of both quality and efficiency. Furthermore, the proposed method exhibits flexibility in training-free conditional generation tasks, including texture transfer and 3D inpainting.

Strengths

The paper clearly proposes to represent 3D humans using volumetric primitives, effectively transforming the 3D representation into a 2D parameter space that is well-suited for 2D diffusion. This method outperforms previous approaches for 3D human face/body generation, which mostly rely on triplane representation. Further, volumetric primitives technique also provides easier animation and conditional generation capabilities. Through experiments, the proposed method demonstrates superior qualitative and quantitative performance compared to previous works. These experiments highlight the advantages of using volumetric primitives representation for 3D human generation, specifically, the paper compared it with voxel-based representations.

Weaknesses

In lines 139-140, the proposed method only considers color, density, and delta scale factor as optimized parameters, while delta translation and delta rotation are not included. It would be beneficial to understand the reasons behind this omission, especially considering that the original MVP paper optimized these parameters. Figure 2 illustrates the diffusion process of rotation (R), translation (T), and scale (s) during training, which appears to contradict the claim that the learnable parameters are solely color, density, and delta scale factor. And it seems that there is a discrepancy, as the expected parameters should be delta rotation, delta translation, and delta scale. Clarification on this point would be appreciated. The motion encoder in the proposed method solely considers joint rotations, which may not be sufficient for accurate fitting of 3D humans. Shape parameters also play a crucial role in this process. It would be helpful to hear an explanation for the decision to exclude shape parameters from the motion encoder. There are typos in Function (7), which are inconsistent with the function presented in the DDPM paper [16]. It is important to address these inconsistencies and ensure the correctness and coherence of the equations. The loss function (8) differs from the one described in the DDPM paper [16]. The original function aimed to estimate noise from V_t instead of directly estimating the ground truth. It would be beneficial to provide an explanation for this modification and discuss its implications.

Questions

The paper conducted a comparison between volumetric primitives and voxel representations in diffusion-based 3D human generation. However, I am also interested in understanding how volumetric primitives compare to triplane representations in diffusion. It would greatly enhance the experimental analysis if this comparison were included. Regarding the ablation study, the paper compared predefined rotation and translation values derived from a template human mesh with identity-specific ones. However, the paper does not provide a detailed explanation of the identity-specific rotation and translation. It would be beneficial to include a thorough explanation of this approach to improve the understanding of the study.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

The proposed method is innovative as it combines volumetric primitives with diffusion model. However, there is room for improvement in terms of the writing. There is still a noticeable disparity between the ground truth and the fitting results. It seems that there is significant room for improvement in stage 1 of the method.

Reviewer GMsy2023-08-15

Dear authors, thanks for the detailed rebuttals. I don't have additional questions and will make a final decision based on the discussion with other reviewers. Best,

Authorsrebuttal2023-08-16

Thanks for your feedback. We are encouraged to see that our responses address your concerns above. In addition, we would like to emphasize our contributions as follows: 1) We propose the first diffusion model for 3D human generation. 2) We design a compact and expressive parameter space on top of volumetric primitives to enable tractable training of diffusion models and high-performance rendering in a decoder-free manner. 3) We propose a scalable framework with cross-modal attention for learning volumetric primitives without per-subject optimization on large-scale dataset. 4) We demonstrate the great potential of our work in training-free generative tasks like texture transfer and 3D inpainting, which surpasses the limitations of prior approaches that necessitate retraining. Should there be any further concerns from you that might hinder the recommendation of our paper for acceptance, we would be glad in addressing them.

Reviewer wB9H2023-08-15

Dear Authors, Thank you for providing the fine-tuning experiment for EVA3D. It would be nice to provide further details about the training process. Is it trained only using the GAN losses? If so, I feel that it is still unfair to EVA3D since the proposed method heavily relies on 3D supervision whereas EVA3D is only provided with weak 2D supervision in the form of GAN loss, even the reconstruction loss is not used. I would have liked to see a baseline method that utilizes a similar amount of 3D supervision as the proposed approach. Regarding novelty, I acknowledge that the proposed combination is not straightforward and somewhat novel. Sincerely, wB9H

Authorsrebuttal2023-08-16

Thanks for your feeback and for acknowledging our technical contributions. We appreciate the opportunity to provide further clarification on the training process and address the concerns you raised. Regarding the training process, we would like to emphasize that our method does not require any explicit 3D supervision, such as ground-truth meshes or normals, during training. All methods in our experiments, are trained using a consistent experimental setup with input training data consisting of multiview 2D images and camera poses. This ensures that both EVA3D and our model, are provided with weak 2D image-based supervisions during training. The finetuned EVA3D is trained using the same configuration as the official code release. The training objective for the finetuned model involves a combination of non-saturating GAN loss, minimum offset loss, and eikonal loss. We utilized the pre-trained EVA3D model, which was initially trained on the DeepFashion dataset, and performed the finetuning on our RenderPeople dataset. During our experimentation, we observed that introducing a 2D image-based reconstruction loss to the training process of EVA3D led to mode collapse, likely due to the sensitivity of adversarial training. Too strong reconstruction loss will degenerate the model into a autodecoder. We also want to highlight that, as the first diffusion model for 3D human generation, direct comparisons with prior works in the exact same setting are challenging due to the novelty of our approach. However, to provide meaningful baselines for comparison, we included results from voxel-based and NeuralBody-based methods. These baselines utilize the same 2D image-based supervision as our approach, enabling a fair assessment of our method's performance. Furthermore, the comparison of 3D diffusion models with GAN-based methods on multiview images is a widely accepted practice in recent diffusion models for 3D rigid object generation [1,2,3]. We follow this practice for comparisons of 3D human (non-rigid body) generation, which allows for meaningful evaluations and insights into the capabilities of different methods in generating 3D representations from 2D inputs. We hope this clarification addresses your concerns and provides a more comprehensive understanding of our experiment and method. We are open to any further discussions or questions you may have, and we appreciate your consideration of our paper for acceptance. [1] N. Muller et al. DiffRF: Rendering-Guided 3D Radiance Field Diffusion. In CVPR, 2023. [2] T. Wang et al. Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion. In CVPR, 2023. [3] E. Ntavelis et al. Autodecoding Latent 3D Diffusion Models. In arXiv, 2023.

Reviewer wB9H2023-08-22

Thank you for providing a detailed response. What I meant was that EVA3D can probably benefit from adding multiview consistency losses, in addition to the GAN loss that it currently uses for monocular images. Nonetheless, most of my concerns have been addressed. Thank you!

Reviewer PWaT2023-08-17

Dear Authors, Thank you for providing a rebuttal in response to the initial review. However, there remain some unresolved concerns: - For my "Q1: The proposed method is a 2-stage approach that requires multi-view images", the primary concern was the method's dependency on **multi-view images for training**. This was not explicitly addressed in the authors' response. - Regarding my "Q5: How the 20 poses sampled for training data?", employing a simple random sampling strategy may not yield a representative pose distribution from the AMASS dataset. Considering the **imbalanced motion distribution within AMASS**, this strategy might compromise the effectiveness of the proposed method. It would be beneficial if you could provide further insights or clarifications. Best, Reviewer PWaT

Authorsrebuttal2023-08-19

Thank you for your valuable feedback and follow-up comments. We appreciate the opportunity to provide additional clarifications in response to your concerns regarding Q1 and Q5. **Q1**: We acknowledge the consideration that multiview images incur higher costs compared to image collections. However, we argue that training with multiview images offers distinct advantages, namely 1) learning comprehensive and coherent 3D representation of human body (e.g., texture on the back of human body) and 2) enabling view synthesis with free viewpoints. Notably, existing approaches like ENARF-GAN [1] and EVA3D [2] opt for image collections for training due to their cost-effectiveness, yet they are constrained to generating forward-facing humans within limited camera view variations. In contrast, our model offers a promising way to fully utilize the multiview knowledge during training to generate coherent 3D representation of humans, enabling free-view synthesis and efficient decoder-free rendering. Moreover, the concurrent work HumanGen also achieved 360 degree rendering by incorporating explicit 3D priors from the model trained on multiview images. This substantiates the effectiveness and necessity of utilizing multiview data for comprehensive 3D human generation. In essence, both image collections and multiview data serve as distinct task settings tailored to different objectives. The former can leverage vast in-the-wild knowledge in a cheaper way but the camera views are bounded by the training data. The latter, albeit with a higher cost associated with multiview setup, facilitates the learning of genuinely 3D human representations, enabling free-view rendering. **Q5**: Thanks for your insightful comments. We note that the generalizability of our method to novel poses has been validated in our experiments (supplementary video) where all novel poses are out-of-distribution poses instead of in-distribution unseen poses. This feature is enabled by our careful design of decoder-free rendering, which exclude the implicit dependence of pose during rendering. Therefore, the diversity of human poses does not significantly impact our training process. Furthermore, we conducted experiments of three different sampling strategies on AMASS dataset. In these experiments, we sample $20\times 796 = 15920$ poses (equals to the number of poses used in our RenderPeople dataset) from AMASS for each method and compute the Fréchet distance [3] on the pose parameters of SMPL-X against the source data. It's important to emphasize that the Fréchet distance serves as a measure of distribution similarity. We investigate three different pose sampling strategies: 1) uniform: sample poses from the dataset with a fixed interval, 2) random: sample poses from the dataset randomly, and 3) cluster: we divide the dataset into 30 pose clusters using K-means algorithm, and randomly sample poses in each cluster. The results shown below indicate that the difference between sampling strategies is minor given the number of sampled poses we used for training is already large enough. |Methods|Fréchet distance ($\times 10^{-3}$)| |:--:|:--:| |uniform|0.99| |random|1.13| |cluster|4.48| [1] A. Noguchi et al. Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representations. In ECCV, 2022. [2] F. Hong et al. EVA3D: Compositional 3D Human Generation from 2D Image Collections. In ICLR, 2023. [3] D.C Dowson et al. The Fréchet distance between multivariate normal distributions. In Journal of Multivariate Analysis, 1982.

Reviewer aNCP2023-08-18

Typo in GPU Memory

Thank you very much for the rebuttal and clearance of most the questions. It seems the GPU Memory consumption in the above table has wrong unit. Instead of 20K+ GB it should be 20K+ MB. Please note that when adding it into revised version.

Authorsrebuttal2023-08-19

Thank you for your feedback. We appreciate your attention to detail. In the revised version, we will ensure that the GPU Memory consumption unit in the table is corrected to 'MB' and will include it in the supplementary material.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC