ChoreoMaster : Choreography-Oriented Music-Driven Dance Synthesis

SIGGRAPH 2021

Kang Chen1

Zhipeng Tan1

Jin Lei1

Song-Hai Zhang2

Yuan-Chen Guo2

Weidong Zhang1

Shi-Min Hu2

Kang Chen, NetEase Games AI LAB

Zhipeng Tan, NetEase Games AI LAB

Jin Lei, NetEase Games AI LAB

Song-Hai Zhang, Tsinghua University

Yuan-Chen Guo, Tsinghua University

Weidong Zhang, NetEase Games AI LAB

Shi-Min Hu, Tsinghua University

1NetEase Games AI LAB

2Tsinghua University

Paper
Supplemental Document
User Study Video
Data

Abstract

Despite strong demand in the game and film industry, automatically synthesizing high-quality dance motions remains a challenging task. In this paper, we present ChoreoMaster, a production-ready music-driven dance motion synthesis system. Given a piece of music, ChoreoMaster can automatically generate a high-quality dance motion sequence to accompany the input music in terms of style, rhythm and structure.To achieve this goal, we introduce a novel choreography-oriented choreomusical embedding framework, which successfully constructs a unified choreomusical embedding space for both style and rhythm relationships between music and dance phrases. The learned choreomusical embedding is then incorporated into a novel choreography-oriented graph-based motion synthesis framework, which can robustly and efficiently generate high-quality dance motions following various choreographic rules. As a production-ready system, ChoreoMaster is sufficiently controllable and comprehensive for users to produce desired results. Experimental results demonstrate that dance motions generated by ChoreoMaster are accepted by professional artists.


More Results


BibTeX

@article{choreomaster2021,
    author = {Chen Kang, Zhipeng Tan, Jin Lei, Song-Hai Zhang, Yuan-Chen Guo, Weidong Zhang, Shi-Min Hu},
    title = {ChoreoMaster : Choreography-Oriented Music-Driven Dance Synthesis},
    journal = {ACM Transactions on Graphics (TOG)},
    volume = {40},
    number = {4},
    year = {2021},
    publisher = {ACM}
  }