FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editing

Text-driven motion generation has achieved substantial progress with the emergence of diffusion models. However, existing methods still struggle to generate complex motion sequences that correspond to fine-grained descriptions, depicting detailed and accurate spatio-temporal actions. This lack of fine controllability limits the usage of motion generation to a larger audience. To tackle these challenges, we present FineMoGen, a diffusion-based motion generation and editing framework that can synthesize fine-grained motions, with spatial-temporal composition to the user instructions. Specifically, FineMoGen builds upon diffusion model with a novel transformer architecture dubbed Spatio-Temporal Mixture Attention (SAMI). SAMI optimizes the generation of the global attention template from two perspectives: 1) explicitly modeling the constraints of spatio-temporal composition; and 2) utilizing sparsely-activated mixture-of-experts to adaptively extract fine-grained features. To facilitate a large-scale study on this new fine-grained motion generation task, we contribute the HuMMan-MoGen dataset, which consists of 2,968 videos and 102,336 fine-grained spatio-temporal descriptions. Extensive experiments validate that FineMoGen exhibits superior motion generation quality over state-of-the-art methods. Notably, FineMoGen further enables zero-shot motion editing capabilities with the aid of modern large language models (LLM), which faithfully manipulates motion sequences with fine-grained instructions. Project Page: https://mingyuan-zhang.github.io/projects/FineMoGen.html

Paper

References (23)

Scroll for more · 11 remaining

Similar papers

Peer review

Reviewer 4S8a5/10 · confidence 4/52023-06-30

Summary

Existing methods often struggle to generate intricate motion sequences that align with fine-grained descriptions. Addressing this, the authors introduce FineMoGen, a diffusion-based motion generation and editing framework capable of synthesizing intricate, detailed motions that align with user instructions. FineMoGen employs a diffusion model coupled with a unique transformer architecture called Spatio-Temporal Mixture Attention (SAMI). This architecture optimizes global attention template creation by: 1) explicitly modeling the constraints of spatio-temporal composition, and 2) using a sparsely-activated mixture-of-experts to adaptively extract fine-grained features. In addition, the authors offer the HuMMan-MoGen dataset, designed to facilitate the study of fine-grained motion generation and editing. Extensive experimental validation suggests that FineMoGen outperforms state-of-the-art methods in motion generation quality.

Strengths

1. The authors' contribution of a novel dataset is significant and could stimulate further research in this field. A fine-grained labeled dataset extends the potential for human motion generation applications and aligns with intuitive human approaches to generating and editing human motion. 2. The motivation behind the development of the Spatio-Temporal Mixture Attention model is robust and justified.

Weaknesses

1. The relationship between SAMI and MEA is somewhat unclear, particularly from the perspective of notations. The association between F' and B is not well-explained. 2. The factorization in Equation 4 could benefit from clearer explanation. The current description of G' as a time-varied signal seems insufficient, and the formulation of G* in Equation 4 lacks clarity, with minimal elaboration provided in the main paper. 3. Figure 2 does not adequately represent the primary contributions of this paper. It solely incorporates the LLM module, which, while new, is not the main contribution. The SAMI and MoE, which are crucial to this paper, are omitted. 4. The motivation behind the Sparse-activated Mixture-of-Expert is unclear. I failed to see the inherent correlation between the Sparse-activated Mixture-of-Expert and SAMI. 5. On Line 214, there appears to be a typographical error - "vGiven" should likely be corrected.

Questions

Please refer to weaknesses.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

2 fair

Limitations

The authors mention the limitation and suggest a way to resolve it.

Authorsrebuttal2023-08-17

Follow-up

Dear Reviewer 4S8a, Thank you very much again for your time and efforts in reviewing our paper. As per your suggestion, we have included a new figure to enhance the clarity of the architectural design and have provided a detailed explanation regarding the utilization of MoE. We just wonder whether there is any further concern and hope to have a chance to respond before the discussion phase ends. Best regards, Authors of paper 1891

Area Chair oLs72023-08-17

Request for your feedback in light of authors' feedback

Thank you for your valuable insights and expertise which have contributed significantly to the review process. Following the initial review, the authors have provided a detailed rebuttal addressing the feedback and comments provided by our esteemed reviewers, including yourself. I kindly request that you take the time to carefully review the authors' rebuttal and assess its impact on your initial evaluation. Please share your thoughts and any additional points you may have after reading the authors' rebuttal. Thank you very much!

Reviewer CFoF6/10 · confidence 3/52023-07-03

Summary

The paper proposes a method for diffusion-based motion generation and editing framework for synthesizing fine-grained motions. It takes fine-grained spatio-temporal description. The framework supports zero-shot and fully-supervised generation scenario. The utilize MOE for generating global template. They have also introduced a dataset with fine-grained text annotation. Their method is called spatio-temporal mixture attention (SAMI). To summarize, The fine-grained descriptions are first processed by frozen CLIP followed by two layers of learnable transformer encoders. Next, a text feature matrix is created, and then sent to a diffusion model-based motion generative network to generate corresponding motion sequence.

Strengths

The zero-shot generation is encouraging. The results on HumanML3D and KIT-ML show improved and comparable results with existing methods. The superior temporal composition results show the effectiveness of SAMI. Ablation experiments are a plus.

Weaknesses

The dependencies on the LLM might have two way effect. The natural description of the fine-grained motion may not be best utilized in this scenario.

Questions

Like most other generative methods, it also should have concern for its potential negative impacts. Can some sort of user attribution be incorporated in the model itself?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The dependencies on the LLM might have two way effect. The natural description of the fine-grained motion may not be best utilized in this scenario. Like most other generative methods, it also should have concern for its potential negative impacts. Can some sort of user attribution be incorporated in the model itself?

Reviewer DsXZ5/10 · confidence 3/52023-07-05

Summary

The paper targets an interesting problem of fine-grained spatial-temporal motion generation and editing. It introduces a new transformer structure with spatio-temporal mixturee attention. Also, a large scale dataset, called HuMMan-MoGen, has been introduced to train the proposed algorithm. Attractive experimental results have been reported in the paper.

Strengths

1. A new large-scale dataset with fine-grained spatio-temporal descriptions have been introduced in the paper. 2. Attractive performance has been reported on the motion generation benchmark. 3. The problem of fine-grained motion generation and edting would be helpful for the industry.

Weaknesses

1. The proposed HuMMan-MoGen dataset is originally from the HuMMan dataset. Is it possible to the dataset convering sufficient diversity of the finegrained motion clips? 2. The annotations of different groups would make it easy to represent the motions. But some motions have been defined as a whole body rather than a combination of motions from different body parts. Is there any potential negative impact on this problem? 3. I would suggest to include the dataset contruction process in the paper rather than leaving it in the supplementary.

Questions

Please address the questions in the weakness section. Also, will the dataset be public released together with the finegrained description?

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The paper already discussed the limitations in the paper.

Reviewer C9Az6/10 · confidence 4/52023-07-06

Summary

The paper introduces a compositional text-to-motion synthesis approach that allows for control up to body part level. For fine-grained spatio-temporal editing, the motion of each body part from a predetermined set (e.g., head, arms, legs, etc.) can be described. Furthermore, for zero-shot motion control, the model has an interface with LLMs to support high-level motion descriptions, which are converted to part-specific commands. The paper introduces FineMoGen, a diffusion-based model with novel spatio-temporal mixture attention (SAMI). Specifically, each body part is represented with a spatial attention block and conditioned only on the corresponding part’s text feature to ensure spatial independence. Similarly, the temporal block aims to make smooth transitions between different stages (i.e., different atomic motions). The proposed model performs competitively on the ***atomic text-to-motion benchmarks such as HumanML3D and KIT-ML. In the ***compositional text-to-motion task, it performs quantitatively better than the baselines on the BABEL and the proposed HuMMan-MoGen datasets. There are major artifacts in the synthesized motions (e.g., implausible movements and foot skating), though. *** I distinguished two tasks by tagging them as "atomic" and "compositional". While the former focuses on generating a short motion snippet from a single instruction describing an atomic event, the latter also involves transitions between semantically different motion descriptions, requiring the composition of several instructions.

Strengths

**originality** The problem setting is interesting and practical. The proposed model is arguably novel. Ideas from the prior works are nicely extended. **quality** The paper is well-motivated, and the concerns are addressed in the design of the proposed model. The decoupling of the spatial and temporal blocks and the corresponding inputs (i.e., part-specific text commands and descriptions for different stages, respectively) seems to be effective. **clarity** Although the presentation of the technical details could be improved, it is straightforward to get an overview of the motivation and the proposed approach. **significance** The proposed model performs well in evaluations in both the atomic text-to-motion and compositional text-to-motion tasks by allowing for fine-grained editing. I believe this is an interesting direction and could be valuable for the community.

Weaknesses

* The motion quality of the synthetic samples is one of the major weaknesses of this work. While the motion samples in the first part of the supplementary video (examples 1, 2, and 3) are okay, the rest, including the qualitative comparison and the editing sections, have major artifacts. Unfortunately, the inference settings (i.e., the dataset, type of text commands, etc.) are not provided. (**Q1**) Could the authors explain why this is the case? * (**Q2**) Similarly, the paper does not provide details on the underlying motion representation. Is it 3D positions or rotations? If the latter, which representation? What do the "raw motion vectors" look like? * I find it not straightforward to follow the technical details in Section 3.3. At least the transition from the "Efficient Attention" paragraph to Section 3.3 was not smooth. I think a revision would help. Moreover, an intuitive figure could be highly useful for conveying the message.

Questions

I have listed my questions regarding the weaknesses of the paper above (Q1 and Q2). Here I am raising additional questions for further clarification. **Q3**) It is not clear to me how interactive editing works. Is the motion initialized with the output of the previous pass or generated from scratch as an additional stage? **Q4**) The supplementary video does not have any results for the atomic text-to-motion task. I am curious if the artifacts also exist in this task. Could the authors provide a qualitative comparison with the baselines?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

2 fair

Contribution

3 good

Limitations

Yes, the paper briefly mentions limitations and the broader impact.

Reviewer DsXZ2023-08-14

The rebuttal addressed most of my concerns and I will keep the postive view on the paper.

Reviewer C9Az2023-08-16

Post-rebuttal Comment

I thank the authors for their rebuttal. My concerns are addressed. I read other reviews as well. It looks like no other major concerns have been raised. I am lenient towards acceptance. I'll revise my score after the reviewer discussion period.

Authorsrebuttal2023-08-17

Thanks to Reviewer C9Az

Dear Reviewer C9Az, We sincerely appreciate the comments and raising the score of our paper. Best regards, Authors of paper 1891

Reviewer CFoF2023-08-16

post-rebuttal comments

I thank the authors for responding to reviewers' questions. I am leaning towards accept

Authorsrebuttal2023-08-17

Thanks to Reviewer CFoF

Dear Reviewer CFoF, We really appreciate you spending the effort to interact with us and provide valuable, constructive feedback about our paper. Best regards, Authors of paper 1891

Reviewer 4S8a2023-08-18

Post-rebuttal discussions

Dear Authors of Paper 1891, Please accept my apologies for my delayed response. I have carefully read your rebuttal and would like to provide the following comments: 1. **Regarding the difference between MEA and SAMI**: I appreciate the effort you took to illustrate the distinction between these two concepts with a detailed figure. It certainly clarifies the subject in a visually intuitive way. I strongly recommend including this figure in the manuscript to assist the readers. 2. **Regarding the Sparse-activated Mixture-of-Experts (MoE)**: While I understand that previous designs may introduce complexities to the model's learning process, the reasoning behind employing the Sparse-activated MoE technique to "broaden the overall network structure to enhance learning capability" still raises questions for me. Specifically: - ① Could you elucidate the particular features of the Sparse-activated MoE that make it uniquely suited to this problem? - ② Is the "Sparse-activated" design crucial? Is it possible to employ the naive softmax attention instead? Is the concept of mixture-of-experts similar to multi-head in Transformer? - ③ I noticed an inconsistency in terminology between "sparsely-activated" in the abstract and "sparse-activated" in the main text. Please align these terms for clarity, if possible. I look forward to your revisions. Kind regards, Reviewer 4S8a

Authorsrebuttal2023-08-21

Additional Rebuttal

We sincerely appreciate your additional suggestions. We intend to delve further into the discussion concerning the Sparsely-activated MoE component. > **Q4-1** Could you elucidate the particular features of the Sparse-activated MoE that make it uniquely suited to this problem? **A4-1** Our primary objective is to enhance the model's learning capacity as much as possible without significantly increasing testing time. The Sparsely-activated MoE perfectly addresses our dual requirements for speed and accuracy. By leveraging the 'expert' approach, it offers greater flexibility in extracting meaningful information, and during testing, it employs only a minimal number of experts, thus ensuring a modest increase in testing time. To the best of our knowledge, Sparsely-activated MoE could be the optimal choice. > **Q4-2** Is the "Sparse-activated" design crucial? Is it possible to employ the naive softmax attention instead? Is the concept of mixture-of-experts similar to multi-head in Transformer? **A4-2** We show additional experiments on HuMMan-MoGen test set as below: | Method | Number of routing | Number of Experts | R Precision$\uparrow$ | FID$\downarrow$ | Computation Cost | |:-:|:-:|:-:|:-:|:-:|:-:| | Real motions | - | - | $0.49$ | $0.007$ | - | | - | - | - | $0.37$ | $1.95$ | 1x | | - | 1 | 4 | $0.40$ | $1.57$ | ~1.4x | | - | 2 | 4 | $0.40$ | $1.49$ | ~2.6x | | - | 4 | 4 | $0.41$ | $1.41$ | ~5.0x | | - | 1 | 8 | $0.40$ | $1.54$ | ~1.6x | | Ours | 2 | 8 | $0.41$ | $1.34$ | ~2.9x | | - | 4 | 8 | $0.41$ | $1.30$ | ~5.5x | | - | 8 | 8 | $0.42$ | $1.25$ | ~10.8x | The term 'sparsely-activated' does not play a critical role in generating quality. As shown in this table, employing more experts during the testing phase could potentially yield better generation results. Therefore, if precision is the sole consideration, 'sparsely-activated' might not be the optimal choice. However, we also observe that the most noticeable accuracy improvement occurs when transitioning from no MoE usage to using 1 routing, and from using 1 routing to 2 routings. Considering computational complexity, we have opted for a compromise solution. > **Q4-3** I noticed an inconsistency in terminology between "sparsely-activated" in the abstract and "sparse-activated" in the main text. Please align these terms for clarity, if possible. **A4-3** Thanks for pointing out this issue. We will revise the entire document and consistently use 'sparsely-activated'.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC