Human Motion Diffusion Model

29 Sep 2022  ·  Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or, Amit H. Bermano ·

Natural and expressive human motion generation is the holy grail of computer animation. It is a challenging task, due to the diversity of possible motion, human perceptual sensitivity to it, and the difficulty of accurately describing it. Therefore, current generative solutions are either low-quality or limited in expressiveness. Diffusion models, which have already shown remarkable generative capabilities in other domains, are promising candidates for human motion due to their many-to-many nature, but they tend to be resource hungry and hard to control. In this paper, we introduce Motion Diffusion Model (MDM), a carefully adapted classifier-free diffusion-based generative model for the human motion domain. MDM is transformer-based, combining insights from motion generation literature. A notable design-choice is the prediction of the sample, rather than the noise, in each diffusion step. This facilitates the use of established geometric losses on the locations and velocities of the motion, such as the foot contact loss. As we demonstrate, MDM is a generic approach, enabling different modes of conditioning, and different generation tasks. We show that our model is trained with lightweight resources and yet achieves state-of-the-art results on leading benchmarks for text-to-motion and action-to-motion. .

PDF Abstract

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Motion Synthesis HumanAct12 MDM FID 0.08 # 1
Accuracy 0.99 # 1
Multimodality 2.58 # 1
Motion Synthesis HumanML3D T2M FID 1.067 # 2
Diversity 9.188 # 2
Multimodality 2.09 # 2
Motion Synthesis HumanML3D Text2Gesture FID 7.664 # 3
Diversity 6.409 # 4
Motion Synthesis HumanML3D JL2P FID 11.02 # 4
Diversity 7.676 # 3
Motion Synthesis HumanML3D MDM FID 0.544 # 1
Diversity 9.559 # 1
Multimodality 2.927 # 1