MoCap-Solver: A Neural Solver for Optical Motion Capture Data

SIGGRAPH 2021

Kang Chen1

Yupan Wang1

Song-Hai Zhang2

Sen-Zhe Xu2

Weidong Zhang1

Shi-Min Hu2

Kang Chen, NetEase Games AI LAB

Yupan Wang, NetEase Games AI LAB

Song-Hai Zhang, Tsinghua University

Sen-Zhe Xu, Tsinghua University

Weidong Zhang, NetEase Games AI LAB

Shi-Min Hu, Tsinghua University

1NetEase Games AI LAB

2Tsinghua University

Paper
Supplemental Document
Code&Data

Abstract

In a conventional optical motion capture (MoCap) workflow, two processes are needed to turn captured raw marker sequences into correct skeletal animation sequences. Firstly, various tracking errors present in the markers must be fixed (cleaning or refining). Secondly, an agent skeletal mesh must be prepared for the actor/actress, and used to determine skeleton information from the markers (re-targeting or solving). The whole process, normally referred to as solving MoCap data, is extremely time-consuming, labor-intensive, and usually the most costly part of animation production. Hence, there is a great demand for automated tools in industry. In this work, we present MoCap-Solver, a production-ready neural solver for optical MoCap data. It can directly produce skeleton sequences and clean marker sequences from raw MoCap markers, without any tedious manual operations. To achieve this goal, our key idea is to make use of neural encoders concerning three key intrinsic components: the template skeleton, marker configuration and motion, and to learn to predict these latent vectors from imperfect marker sequences containing noise and errors. By decoding these components from latent vectors, sequences of clean markers and skeletons can be directly recovered. Moreover, we also provide a novel normalization strategy based on learning a pose-dependent marker reliability function, which greatly improves system robustness. Experimental results demonstrate that our algorithm consistently outperforms the state-of-the-art on both synthetic and real-world datasets.


BibTeX

@article{mocapsolver2021,
    author = {Chen Kang, Yupan Wang, Song-Hai Zhang, Sen-Zhe Xu, Weidong Zhang, Shi-Min Hu},
    title = {MoCap-Solver: A Neural Solver for Optical Motion Capture Data},
    journal = {ACM Transactions on Graphics (TOG)},
    volume = {40},
    number = {4},
    year = {2021},
    publisher = {ACM}
  }