Continuous diffusion models have demonstrated their effectiveness in addressing the inherent uncertainty and indeterminacy in monocular 3D human pose estimation (HPE). Despite their strengths, the need for large search spaces and the corresponding demand for substantial training data make these models prone to generating biomechanically unrealistic poses. This challenge is particularly noticeable in occlusion scenarios, where the complexity of inferring 3D structures from 2D images intensifies. In response to these limitations, we introduce the Discrete Diffusion Pose ($\text{Di}^2\text{Pose}$), a novel framework designed for occluded 3D HPE that capitalizes on the benefits of a discrete diffusion model. Specifically, $\text{Di}^2\text{Pose}$ employs a two-stage process: it first converts 3D poses into a discrete representation through a \emph{pose quantization step}, which is subsequently modeled in latent space through a \emph{discrete diffusion process}. This methodological innovation restrictively confines the search space towards physically viable configurations and enhances the model's capability to comprehend how occlusions affect human pose within the latent space. Extensive evaluations conducted on various benchmarks (e.g., Human3.6M, 3DPW, and 3DPW-Occ) have demonstrated its effectiveness.
Paper
Similar papers
Peer review
Summary
This paper introduces a discrete diffusion method for 3D human pose estimation (HPE). Recent works have successfully applied diffusion models for HPE. However, they need a lot of training data and sometimes output non-anthropomorphic poses. In this work, the authors propose to use discrete diffusions, which leverage a quantized latent space and model the diffusion process as transitions between discrete states. The hope is that the result will be more constrained to the space of plausible poses as it can only take a limited number of values. The main contributions of this paper are the following: (1) The authors propose a VQ-VAE for quantizing human pose and modeling the dependencies between the body joints; (2) A discrete diffusion model is designed to do HPE in the quantized latent space; (3) The "Occlude and replace" strategy is proposed to model occlusions, further improving the results.
Strengths
- The approach is new: this is the first work using discrete diffusion for human pose estimation. This approach is promising and performs well, especially in the presence of occlusions. - I found the paper quite well written. The weakness of prior works is clearly pointed out, and we understand how the authors hope to address them. - There are plenty of details about the models and their mathematical ground. Calculations are well-detailed, which makes them easy to follow. - The experiments and numerous ablations help capture the strengths of the introduced approach. The proposed approach outperforms SOTA approaches.
Weaknesses
The method section could be improved to make it more clear: - Giving the dimensions of $F$ and $T$ from the beginning (L137) would help understanding the VQ-VAE. - (L183) I find it very surprising that the loss for training the VQ-VAE is a cross-entropy. Usually, we use reconstruction loss for training such models (here, it could have been an MSE or some MPJPE). It is unclear why this is not the case for the proposed model. - (L185) It is unclear if $k$ is the sequence of $N$ tokens or only one token. Given the Equation 2, I guess it is the sequence? - (L193) The notation $k_i$ is already used in Equation 2 (and I think this does not denote the same thing as here it is temporal). - (L270) Using the same variable name for 2 different values is not a good practice. Even if this paper is the first to apply discrete diffusion models to HPE, the methodology section does not bring much novelty. Section 3.2 could be considered preliminary as this directly applies [23]. From my understanding, the only difference is that the "Mask and replace" operation of [23] is renamed "Occlude and replace" (but it does the same thing). I think it is good to use prior works when designing a new method, but it should be made clear which part is new and which part is an application of prior works. The introduction justifies this work by saying continuous diffusion models have high dimensions and that relationships between the joints are ignored. From my understanding, the latent representation of a pose is composed of $N=100$ tokens of dimension $d=5$. Each of the $d_i$ dimensions can take a certain number of values (respectively 7,5,5,5,5). That means that the codebook has 4375 tokens. The transition matrix $M$ between consecutive intermediate predictions is then of dimension $4375 \times 4375$ if we suppose that the $N$ tokens are independent (which is not supposed to be the case if we want the pose to be globally coherent). In the end, I am not sure that the proposed model's dimension is smaller than continuous diffusion models and that the relationships between the joints are better modeled. I also have some doubts about the fairness of experiments. The proposed model uses a ViT backbone, while all others (except [20]) have CNN or GCN backbones with much fewer parameters. I believe this is a crucial component of the model and that it is impossible to evaluate the contributions if basic components differ from other approaches. The material used for experiments is given, but this is valuable only if we have running time for experiments (which is not the case).
Questions
- Is $k$ (L185) a sequence? - Are there some changes between the proposed diffusion model and the model from [23]? - Is the relationship between the $N=100$ tokens modeled? - Is the plausibility of the results due to the quantization or the discrete diffusion process? I know this is linked, but for instance, would it be possible to do a latent diffusion process on the continuous latent representation of dimension $N \times d$ (which is much smaller) and do the quantization just before decoding? - Were other backbones tested to quantify the role of this component in the results? - What are the running times (training and testing)? Is it comparable to other methods? - In Section 4, where are the error bars and the information about the statistical significance? Were the experiments run multiple times? - One of the advantages of diffusion models is that they can generate diverse outputs given a single 2D observation, which helps address the ambiguity of predicting 3D from a single 2D POV and occlusions. However, no analysis is performed on that; it seems that a single prediction is made given an image. Is there a reason for that?
Rating
5
Confidence
4
Soundness
2
Presentation
3
Contribution
2
Limitations
The section on the limitations is quite limited. I believe the problem of predictions under heavy occlusion could have been solved by making multiple predictions given an image, which is straightforward with generative models such as diffusions. Limitations could also mention the running time of the diffusion-based approaches. Limitations include the fact that the experiments ran for a single time, which is particularly concerning since diffusion models introduce stochasticity in predictions. The societal impact could include environmental considerations.
Replying to Official Comment by Reviewer ACgE
We sincerely appreciate your thorough consideration of our responses and are delighted to address all questions and concerns. ## Q1:Dimensions of 𝐹 and 𝑇 We apologize for this misunderstanding. The confusion arose due to the word limit in our rebuttal, which prevented us from clearly illustrating this point. We originally intended to convey that we fully agree with your suggestion, and we will revise the original manuscript to include explicit descriptions of the dimensions of $\mathbf{F}$ and $\mathbf{T}$ (e.g., ) starting from the beginning (L137), such as $\mathbf{F} = (\mathbf{f}_1, \mathbf{f}_2, \cdots, \mathbf{f}_N)$ ($\mathbf{f}_i \in \mathbb{R}^{D}$), and $\mathbf{T}=(\mathbf{t}_1, \mathbf{t}_2, \cdots, \mathbf{t}_N)$ ($\mathbf{t}_i \in \mathbb{R}^{D}$) . This will make our presentation clearer and more intuitive. ## Q2: Dependencies between sub-structures? **We agree that considering the dependencies between sub-structures is an important problem**. Our approach follows the ***codebook learning paradigm (i.e., the encoder-decoder framework in VQ-VAE style)***, where an encoder encode the complete pose into discrete tokens, and a decoder decodes these tokens back to a reconstrcuted pose. ***Within this encoder-decoder framework, the interdependencies among sub-structures are mainly considered by the decoder, e.g., the discrete tokens (local sub-structure) can be reconstructed into the original 3D pose (global structure)***. Since our pose quantization step adheres to this pipeline, we have not implemented explicit strategies to further enhance the dependencies between sub-structures. **The motivation behind the design of the Local-MLP (L143-L145):** Our primary goal was to address a specific limitation observed in previous work [20 in the original paper], where MLPs were used to globally model the relationships between all joints. In the context of Di²Pose, our focus is on learning tokens that effectively represent different pose sub-structures. For each token, the key is to capture the relationships between the joints within a sub-structure. This is why we designed the Local-MLP to specifically model the dependencies between adjacent joints within each sub-structure. ## Q3: Reply to the comment on "Q2" We sincerely appreciate your insights on this matter. **We completely agree with your point**, and we have acknowledged this in our rebuttal where we stated, “We acknowledge the importance of distinguishing our contributions from prior work and will clarify and discuss these differences in the Method section.” Regarding your concern about the presentation of the method section, we would like to clarify that when writing this paper, we recognized that discrete diffusion models are a popular and widely-used framework. Our "Occlude and Replace" mechanism was specifically designed for occluded 3D HPE, and we chose this name to reflect its targeted application. However, we did not sufficiently discuss previous discrete diffusion model techniques in our original manuscript. This oversight could indeed cause some confusion, and we sincerely apologize for that. **To address the issues mentioned above, we will provide several revisions as follows:** 1. ***In the Related Work section***, we will include a discussion of the current development and applications of discrete diffusion models (such as [1-4]), highlighting both the connections and differences between our work and prior research. 2. ***In the Method section***, to avoid potential confusion, we will clearly state that **our "Occlude and Replace" transition matrix is inspired by prior work** [1]. We will also emphasize the distinctions and specific contributions of our approach in comparison. Once again, we are grateful for your constructive comments, which have greatly contributed to improving the clarity and quality of our paper. If you have any further questions or concerns, please don't hesitate to discuss them with us. [1] Vector Quantized Diffusion Model for Text-to-Image Synthesis. CVPR 2022 [2] Global Context with Discrete Diffusion in Vector Quantised Modelling for Image Generation. ICCV 2022 [3] Priority-centric human motion generation in discrete latent space. ICCV 2023 [4] Layoutdm: Discrete diffusion model for controllable layout generation. CVPR 2023
Thanks a lot for this detailed answer. I increase my rating as most of my concerns are addressed. However, I am still not entirely convinced by the technical novelty. Is there any technical difference between the model of [23] and DI^2Pose? From my understanding of the paper and our discussion, I feel the only difference is the application domain.\ What is the difference between "Occlude and replace" and "Mask and replace"? I agree that this mechanism is well-suited to the problem of occluded 3D HPE, but changing its name does not make it a contribution if it works exactly like "Mask and replace" in [23].
Thanks for reviewing our work
We would like to express our sincere gratitude to you for your thorough review of our work. It is our pleasure to have addressed most of your concerns, and we greatly appreciate your recognition of our efforts by increasing the rating. We acknowledge that the "Occlude and Replace" mechanism shares similarities with the "Mask and Replace" technique from [23] on a technical level. However, our key contribution is being the first to effectively apply the discrete diffusion model to the occluded 3D HPE task, using the "Occlude and Replace" mechanism to simulate the full process of a 3D pose transitioning from occluded to recovered. Moreover, directly applying this technique in 3D HPE is not straightforward. The success of our approach depends on learning effective quantized tokens that represent sub-structures of 3D poses. Thus, another key contribution is the integration of the discrete diffusion process with a specifically designed pose quantization step, which together provide an effective solution for occluded 3D HPE. Once again, we sincerely thank you for your review and insightful suggestions, which have been instrumental in helping us improve the quality of our work.
Summary
The work aims to solve the occlusion in 3D human pose estimation, which is an interesting and inherent topic in this field. The authors critiqued that the current continuous diffusion-based pose estimation method requires a large amount of data in the training, while 3D pose datasets are commonly insufficient and thus, hurt the accurate pose generation, especially for occluded cases. To address the problem, the authors first designed a Local-mlp block to quantize poses into tokens, while being aware of the local connection between each keypoints. Then, a diffusion model is exploited to learn the generation of tokens in the condition of a given image. To better address the occluded case, the author designed two addition tokens named "occ" to simulate the occlusion. All combined, the method shows on-par results in 3D pose estimations and the state-of-art performance in the occluded dataset.
Strengths
1. The design of the "occ" tokens is interesting. Previous data augmentation is majorly in the image level, i.e. blocking a few key points in the image by adding noise or set to black. However, this paper does it at the token-level, which potentially leads to a more natural simulation of the occlusion. 2. Combining diffusion with 3D poses is an interesting topic. Especially generate the pose at a token-level, which leverages a similar idea to latent diffusion. 3. The paper is well-written and easy to follow.
Weaknesses
1. In the introduction, the authors criticized the previous method, which requires a large amount of 3D pose data in training, while the proposed solution is to use the quantization of 3D poses. It is unclear why intuitively using a quantized representation of poses could be a solution to data dependency. 2. The target of this paper is also unclear. In my understanding of the author's argument, it is the data dependency of the discrete diffusion model that leads to inaccurate pose estimation in occlusion. However, instead of addressing the data dependency problem, the authors develop occ. token and replacement strategy to address the occlusion problem. This is confusing to me. It is like claiming there is a reason A for problem P, but instead of addressing A, the authors propose another method B completely irrelevant to A to address P from another perspective. 3. Confusing definitions of "Continuous" and "Discrete". There are two pairs of "Continuous" and "Discrete" in the paper. The first pair is the continuous and discrete representation of 3D poses. The second pair is a continuous and discrete diffusion model, as current DDPM or DDIM could be considered as a discretization of a continuous SDE or ODE. It is confusing why it is called continuous in the paper.
Questions
1. For local-map, could there be a more clear explanation of how it exactly captures the interrelation between points? 2. Is it possible to visualize a few quantitative samples in the training process? Especially for poses that get replaced by occ tokens or some tokens get transferred to different tokens. I suspect such a replacement is potentially to be a good method to simulate the person-to-person occlusions. 3. What is the performance of directly using VQ-VAE to encode the 3D poses? Without these results, it is difficult to know the effectiveness of the proposed local-mlp.
Rating
6
Confidence
3
Soundness
3
Presentation
3
Contribution
4
Limitations
No potential negative societal impact of the work is found.
Summary
This work claims that the 3D human pose of a single frame is discrete, and learns the local pairing relationship between joint points to generate the human pose under occlusion. At the method level, VQ-VAE is used for human skeleton quantization, and then combined with the diffusion model to solve this discrete relationship. The main contributions are as follows: 1. The Di2Pose framework is proposed, which integrates the inherent discreteness of 3D pose data into the diffusion model, providing a new paradigm for 3D HPE under occlusion. 2. The designed pose quantization step represents the 3D pose in a combinatorial manner, effectively captures the local correlation between joints, and limits the search space to reasonable configurations. 3. The constructed discrete diffusion process simulates the complete process of 3D pose from occlusion to recovery, and introduces the impact of occlusion into the pose estimation process.
Strengths
1. The structure and writing of the article are relatively clear. 2. The inherent discreteness of 3D pose data is integrated into the diffusion model. Although this is a downstream task of VQ-VAE+Diffusion, it is indeed a new idea for 3D HPE of monocular 2D images. 3. The reasoning process of Di^2Pose in Chapter 3 is very clear, which helps to understand the principle of the overall method. 3. The influence of occlusion is introduced into the pose estimation process, and the codebook that encodes the human pose is back-diffused, which is also a solution to occluded human pose.
Weaknesses
1. From an intuitive level, the human skeleton in this work still uses the simplest H3.6M skeleton with only 17 joints, not even the version with 32 key points. The error tolerance rate is very high for pose estimation of models with such a small number of joints. There is not even any obvious joint change. The Average MPJPE value in the Human3.6M list of 17 joint skeletons has stabilized within 30. The human skeleton in this work needs to use more complex SMPL, SMPL-X, and STAR to see the effect better. 2. The image and skeleton prediction are independent. I hope the author can render the skeleton into the image and map it in the image to better observe the subtle differences in movement. It is difficult to observe the 3D skeleton alone, especially in Figure 4. I can't even find any obvious difference in the human posture in Figure a. In addition, when estimating a single-frame 3D skeleton, it is best to give results from multiple perspectives. I think the perspectives shown in the article are too casual. 3. Although the method is clearly written, I think Di^2Pose is just a downstream task of VQ-VAE+Diffusion. In terms of design, it is not even as clever as [1]. I think Di^2Pose is just DiffPose with the addition of VQ-VAE. 4. The dataset used in the experiment is too outdated. I hope to see more new datasets for verification. Like EMDB[2], SLOPER4D[3], CIMI4D[4], etc. [1] Feng H, Ma W, Gao Q, et al. Stratified Avatar Generation from Sparse Observations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 153-163. [2] Kaufmann M, Song J, Guo C, et al. Emdb: The electromagnetic database of global 3d human pose and shape in the wild[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023: 14632-14643. [3] Yan M, Wang X, Dai Y, et al. Cimi4d: A large multimodal climbing motion dataset under human-scene interactions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023: 12977-12988. [4] Dai Y, Lin Y T, Lin X P, et al. Sloper4d: A scene-aware dataset for global 4d human pose estimation in urban environments[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023: 682-692.
Questions
I hope to see the results on new datasets and SMPL renders to images. I will consider improving my score.
Rating
5
Confidence
4
Soundness
2
Presentation
4
Contribution
2
Limitations
See weaknesses.
Summary
The paper presents novel diffusion-based framework for occluded 3D Human Pose Estimation (HPE) that operates in discrete space. Di2Pose leverages a two-stage process: a pose quantization step and a discrete diffusion process. The pose quantization step captures the local interactions between joints and represents the 3D pose as multiple quantized tokens. These tokens are then modeled in the latent space through a discrete diffusion process. This approach allows the framework to effectively manage occlusions by simulating the transition of a 3D pose from occluded to recovered, enhancing the reliability of pose estimation under occlusion conditions. Extensive evaluations on benchmarks like Human3.6M, 3DPW, and 3DPW-Occ demonstrate that Di2Pose outperforms state-of-the-art methods, particularly in occluded scenarios.
Strengths
Introduction of Di2Pose Framework Di2Pose integrates the inherent discreteness of 3D pose data into the diffusion model, providing a novel paradigm for addressing 3D HPE under occlusions. This framework leverages a two-stage process involving pose quantization and discrete diffusion to confine the search space to physically plausible configurations and simulate the transition from occluded to recovered poses. Pose Quantization Step: The designed pose quantization step effectively captures local correlations between joints by representing 3D poses in a compositional manner. This step confines the search space to reasonable configurations by learning from real 3D human poses, ensuring that the model generates physically plausible poses even under severe occlusions. Discrete Diffusion Process The discrete diffusion process simulates the complete transition of a 3D pose from occluded to recovered, incorporating the impact of occlusions into the pose estimation process. This process models the quantized pose tokens in latent space, enhancing the model’s capability to understand and predict occluded parts of the human pose. Strengths claimed and shown include Effectiveness in Occluded Scenarios: Di2Pose demonstrates significant improvements in 3D HPE accuracy under occlusions compared to state-of-the-art methods, highlighting its superior occlusion-handling capabilities. Physically Plausible Pose Generation: By integrating pose quantization and discrete diffusion, Di2Pose confines the search space to physically reasonable configurations, ensuring the generation of biomechanically valid poses even in challenging occluded scenarios. Comprehensive Evaluation: The framework has been extensively evaluated on multiple challenging benchmarks, consistently yielding lower errors and demonstrating its robustness and generalizability across different datasets. Use of Discrete Diffusion: The introduction of a discrete diffusion process tailored for 3D HPE provides a new perspective in the field, aligning more closely with the inherent discreteness of 3D pose data. These contributions collectively position Di2Pose as a robust and innovative solution for 3D HPE, particularly in the presence of occlusions, advancing the state-of-the-art in this challenging domain.
Weaknesses
Mechanistic Insights: The paper does not delve deeply into the mechanistic aspects of how the discrete diffusion process works in conjunction with pose quantization. Detailed explanations of the underlying mechanisms and how they contribute to the observed improvements would enhance understanding. Comparative Benchmarking: While Di2Pose shows improvements over existing methods, a direct comparison with other contemporary methods such as the Pose Relation Transformer (Chi et al., ICRA 2023) and InfoGCN (Chi et al., CVPR 2022) would provide a clearer picture of its relative strengths and weaknesses. Citation of Related Works: Incorporating references to other successful approaches in the field, such as the above These citations would position Di2Pose within the broader context of ongoing research and highlight its unique contributions relative to other leading methods. Addressing these areas would strengthen the paper, providing deeper insights into the framework’s operation and situating it more firmly within the current landscape of 3D HPE research.
Questions
See above
Rating
7
Confidence
3
Soundness
3
Presentation
3
Contribution
2
Limitations
Limitations are not explored or discussed. It follows the same formula - compare against a benchmark and do ablation studies and state it is better than state of art.
Thanks a lot to the authors for this rebuttal, which addressed many of my concerns. However, some of them are still unsolved: # Q1 1) I cannot see the dimensions of $F$ and $T$ L137... The dimension of $F$ is given indirectly L149, and the dimension of T is unknown until L178. 3) Thank you for this explanation. I understand the different $k$ better now. I believe using different letters would be even clearer, but it is ok to keep it like that. 4) One of the motivations for the proposed method is to model the dependencies between the body joints (L32). Why is it ok to ignore the dependencies between sub-structures? # Q2 I understand that the application is completely different. However, the model is the same.\ I believe it is a great idea to use discrete diffusions for 3D HPE, and the components initially proposed for image generation do have a nice interpretation to solve occlusions.\ The main reproach I make is that the method section is presented in a way that lets the reader believe that all these techniques (for instance, the "Occlude and replace") mechanisms were never done before for discrete diffusions, which is not true.
Thank you for your detailed reply. The response has largely addressed my concerns. However, I have one additional concern, which I have outlined below. Regarding Q3, I disagree with the statement that “Continuous diffusion models initialize the 3D pose from random noise, where each joint can be sampled from the continuous 3D space. This aligns with the general understanding of continuous diffusion processes, such as those modeled by continuous SDEs or ODEs.” In my understanding, the term “continuous” in continuous diffusion models refers to the time variable t, not the space of the joints (either inputs or conditions). Take a traditional diffusion model DDPM as an example, which transitions pure white noise X_T at time T to X_0 at time 0, the “continuous diffusion” refers to considering X_t as a continuous function over time t, i.e., from X_t to X(t). The transition from X_{t} to X_{t-1} then becomes an SDE between two continuous functions X(t) and X(t-1) rather than a recursive formula. This concept is not about where X is sampled no matter this X is sampled from a continuous space (e.g., a multivariate Gaussian distribution) or a discrete space (e.g discrete latent space). I believe this difference in understanding of “continuous” may cause confusion among readers from the field of probabilistic generative models. I suggest that the authors provide further clarification on this point. Regarding the final rating, I noticed that the ratings are quite diverse. However, I place the most value on the review from reviewer ACgE, as another 7-level acceptance appears to have been autonomously generated by ChatGPT, which I am unsure whether to consider as a valid reference. Follow the discussion between the authors and reviewer ACgE, my major concern lies in the novelty of the work, as the “Occlude and Replacement” technique is not originally from this paper. Applying or adapting an existing method to a specific domain offers a different level of novelty compared to proposing an entirely new method. Therefore, although I am still inclined to support acceptance, I would like to adjust my rating to “weakly accept.” I would also be interested in hearing the responses from reviewer tz5g and any further response from ACgE on the issue of novelty.
Thanks for reviewing our work
Thank you for your continued support and for providing a thoughtful assessment of our work. We appreciate your inclination to support acceptance, and we value your feedback on the relevant statements regarding “continuous diffusion.” ### **Avoid confusion about “continuous diffusion”** We fully agree with your suggestion that the term "continuous" as used in the context of “continuous diffusion” may cause confusion to readers. To avoid misunderstanding, we will revise our statements in Introduction as follows: > Prior diffusion-based 3D HPE methods initialize the 3D pose from random noise at the begining of the diffusion process, where each joint can be sampled from the continuous 3D space. Since the continuous 3D space has an infinite number of points, training such diffusion-based models requires a large amount of 3D pose data to achieve optimal outcomes. > This clarification should help readers directly grasp the point we intend to convey and avoid any confusion regarding the “continuous diffusion”. In addition, we will revise other relevant parts about “continuous diffusion” in our paper accordingly to avoid confusing. ### **Clarify our contributions** Regarding the novelty of our work, we acknowledge that the "Occlude and Replace" mechanism shares similarities with prior techniques. However, our key contribution is being the first to effectively apply the discrete diffusion model to the occluded 3D HPE task, using the "Occlude and Replace" mechanism to simulate the full process of a 3D pose transitioning from occluded to recovered. Moreover, directly applying this technique in 3D HPE is not straightforward. The success of our approach depends on learning effective quantized tokens that represent sub-structures of 3D poses. Thus, another key contribution is the integration of the discrete diffusion process with a specifically designed pose quantization step, which together provide an effective solution for occluded 3D HPE. Once again, we appreciate your thoughtful feedback and suggestions, which have been instrumental in refining our work.
Thanks for the author's reply. --- - Regarding the question of skeleton-based and mesh-based methods, what I want to express is that skeletons can be redirected to achieve migration between skeletons. Just like the H3.6M skeleton with 17 joints can be migrated to the SMPL skeleton with 24 key points. Many papers have implemented such methods, which is no longer a difficult problem. - The reason why I hope to see a mesh-based human body model is that the skeleton method is monotonous in visualization. Predicting 10 more Beta parameters can take the visualization effect to a higher level, which I think is necessary to implement. - Discrete expression is the focus of this paper, and the network structure also determines the tone of discrete data. Although the author has made a comprehensive explanation, just like the target detection task in autonomous driving, the single-frame-based method will never be able to obtain more and more coherent information than the continuous-frame-based method. This is also my concern. --- This paper is excellent in terms of writing and methodological integrity, but like the old school papers, I would like to see more reasonable and interesting innovations.
Thanks for reviewing our work
Thanks for your thoughtful consideration and feedback. We fully acknowledge the point you raised about the possibility of redirecting skeletons to achieve migration between different skeleton models. We also agree that extending our Di²Pose framework to mesh-based 3D HPE is a feasible direction. However, due to the structural differences between skeleton and mesh models, this still require certain modifications to our framework, such as redesigning the pose quantization step to incorporate shape parameters and other aspects specific to SMPL or similar models. This extension would require structural redesign and extensive hyperparameter tuning. Due to the limited time of rebuttal period, we do not have enough time to fully explore these aspects. However, we acknowledge its importance and consider it a valuable direction for future work. Regarding the concern about the single-frame-based method not capturing as much coherent information as continuous-frame-based methods, we agree that continuous-frame approaches inherently have an advantage in this regard. However, our single-frame-based framework can serve as a foundational step for future work that extends to continuous-frame-based methods, which would incorporate temporal coherence and continuity into more complex and comprehensive frameworks. Once again, we sincerely appreciate your thorough considerations, which have been instrumental in refining our work.
Decision
Accept (poster)