Occlusion Resistant Network for 3D Face Reconstruction

Hitika Tiwari, Vinod K. Kurmi, K. S. Venkatesh, Yong Sheng Chen

研究成果: Conference contribution同行評審

2 引文 斯高帕斯(Scopus)

摘要

3D face reconstruction from a monocular face image is a mathematically ill-posed problem. Recently, we observed a surge of interest in deep learning-based approaches to address the issue. These methods possess extreme sensitivity towards occlusions. Thus, in this paper, we present a novel context-learning-based distillation approach to tackle the occlusions in the face images. Our training pipeline focuses on distilling the knowledge from a pre-trained occlusion-sensitive deep network. The proposed model learns the context of the target occluded face image. Hence our approach uses a weak model (unsuitable for occluded face images) to train a highly robust network towards partially and fully-occluded face images. We obtain a landmark accuracy of 0.77 against 5.84 of recent state-of-the-art-method for real-life challenging facial occlusions. Also, we propose a novel end-to-end training pipeline to reconstruct 3D faces from multiple variations of the target image per identity to emphasize the significance of visible facial features during learning. For this purpose, we leverage a novel composite multi-occlusion loss function. Our multi-occlusion per identity model shows a dip in the landmark error by a large margin of 6.67 in comparison to a recent state-of-the-art method. We deploy the occluded variations of the CelebA validation dataset and AFLW2000-3D face dataset: naturally-occluded and artificially occluded, for the comparisons. We comprehensively compare our results with the other approaches concerning the accuracy of the reconstructed 3D face mesh for occluded face images.

原文English
主出版物標題Proceedings - 2022 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022
發行者Institute of Electrical and Electronics Engineers Inc.
頁面297-306
頁數10
ISBN(電子)9781665409155
DOIs
出版狀態Published - 2022
事件22nd IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022 - Waikoloa, United States
持續時間: 4 1月 20228 1月 2022

出版系列

名字Proceedings - 2022 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022

Conference

Conference22nd IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022
國家/地區United States
城市Waikoloa
期間4/01/228/01/22

指紋

深入研究「Occlusion Resistant Network for 3D Face Reconstruction」主題。共同形成了獨特的指紋。

引用此