MM-Hand: 3D-Aware Multi-Modal Guided Hand Generation for 3D Hand Pose Synthesis

Zhenyu Wu, Duc Hoang, Shih Yao Lin, Yusheng Xie, Liangjian Chen, Yen Yu Lin, Zhangyang Wang, Wei Fan

研究成果: Conference contribution同行評審

9 引文 斯高帕斯(Scopus)

摘要

Estimating the 3D hand pose from a monocular RGB image is important but challenging. A solution is training on large-scale RGB hand images with accurate 3D hand keypoint annotations. However, it is too expensive in practice. Instead, we develop a learning-based approach to synthesize realistic, diverse, and 3D pose-preserving hand images under the guidance of 3D pose information. We propose a 3D-aware multi-modal guided hand generative network (MM-Hand), together with a novel geometry-based curriculum learning strategy. Our extensive experimental results demonstrate that the 3D-annotated images generated by MM-Hand qualitatively and quantitatively outperform existing options. Moreover, the augmented data can consistently improve the quantitative performance of the state-of-the-art 3D hand pose estimators on two benchmark datasets. The code will be available at https://github.com/ScottHoang/mm-hand.

原文English
主出版物標題MM 2020 - Proceedings of the 28th ACM International Conference on Multimedia
發行者Association for Computing Machinery, Inc
頁面2508-2516
頁數9
ISBN(電子)9781450379885
DOIs
出版狀態Published - 12 10月 2020
事件28th ACM International Conference on Multimedia, MM 2020 - Virtual, Online, United States
持續時間: 12 10月 202016 10月 2020

出版系列

名字MM 2020 - Proceedings of the 28th ACM International Conference on Multimedia

Conference

Conference28th ACM International Conference on Multimedia, MM 2020
國家/地區United States
城市Virtual, Online
期間12/10/2016/10/20

指紋

深入研究「MM-Hand: 3D-Aware Multi-Modal Guided Hand Generation for 3D Hand Pose Synthesis」主題。共同形成了獨特的指紋。

引用此