Summary
The paper introduces a compositional text-to-motion synthesis approach that allows for control up to body part level. For fine-grained spatio-temporal editing, the motion of each body part from a predetermined set (e.g., head, arms, legs, etc.) can be described. Furthermore, for zero-shot motion control, the model has an interface with LLMs to support high-level motion descriptions, which are converted to part-specific commands. The paper introduces FineMoGen, a diffusion-based model with novel spatio-temporal mixture attention (SAMI). Specifically, each body part is represented with a spatial attention block and conditioned only on the corresponding part’s text feature to ensure spatial independence. Similarly, the temporal block aims to make smooth transitions between different stages (i.e., different atomic motions). The proposed model performs competitively on the ***atomic text-to-motion benchmarks such as HumanML3D and KIT-ML. In the ***compositional text-to-motion task, it performs quantitatively better than the baselines on the BABEL and the proposed HuMMan-MoGen datasets. There are major artifacts in the synthesized motions (e.g., implausible movements and foot skating), though.
*** I distinguished two tasks by tagging them as "atomic" and "compositional". While the former focuses on generating a short motion snippet from a single instruction describing an atomic event, the latter also involves transitions between semantically different motion descriptions, requiring the composition of several instructions.
Strengths
**originality**
The problem setting is interesting and practical. The proposed model is arguably novel. Ideas from the prior works are nicely extended.
**quality**
The paper is well-motivated, and the concerns are addressed in the design of the proposed model. The decoupling of the spatial and temporal blocks and the corresponding inputs (i.e., part-specific text commands and descriptions for different stages, respectively) seems to be effective.
**clarity**
Although the presentation of the technical details could be improved, it is straightforward to get an overview of the motivation and the proposed approach.
**significance**
The proposed model performs well in evaluations in both the atomic text-to-motion and compositional text-to-motion tasks by allowing for fine-grained editing. I believe this is an interesting direction and could be valuable for the community.
Weaknesses
* The motion quality of the synthetic samples is one of the major weaknesses of this work. While the motion samples in the first part of the supplementary video (examples 1, 2, and 3) are okay, the rest, including the qualitative comparison and the editing sections, have major artifacts. Unfortunately, the inference settings (i.e., the dataset, type of text commands, etc.) are not provided. (**Q1**) Could the authors explain why this is the case?
* (**Q2**) Similarly, the paper does not provide details on the underlying motion representation. Is it 3D positions or rotations? If the latter, which representation? What do the "raw motion vectors" look like?
* I find it not straightforward to follow the technical details in Section 3.3. At least the transition from the "Efficient Attention" paragraph to Section 3.3 was not smooth. I think a revision would help. Moreover, an intuitive figure could be highly useful for conveying the message.
Questions
I have listed my questions regarding the weaknesses of the paper above (Q1 and Q2). Here I am raising additional questions for further clarification.
**Q3**) It is not clear to me how interactive editing works. Is the motion initialized with the output of the previous pass or generated from scratch as an additional stage?
**Q4**) The supplementary video does not have any results for the atomic text-to-motion task. I am curious if the artifacts also exist in this task. Could the authors provide a qualitative comparison with the baselines?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Limitations
Yes, the paper briefly mentions limitations and the broader impact.