Improving Single-View Mesh Reconstruction for Unseen Categories via Primitive-Based Representation and Mesh Augmentation

Yu Liang Kuo, Wei Jan Ko, Chen Yi Chiu, Wei Chen Chiu

研究成果: Conference contribution同行評審

摘要

As most existing works of single-view 3D reconstruction aim at learning the better mapping functions to directly transform the 2D observation into the corresponding 3D shape for achieving state-of-the-art performance, there often comes a potential concern on having the implicit bias towards the seen classes learnt in their models (i.e. reconstruction intertwined with the classification) thus leading to poor generalizability for the unseen object categories. Moreover, such implicit bias typically stemmed from adopting the object-centered coordinate in their model designs, in which the reconstructed 3D shapes of the same class are all aligned to the same canonical pose regardless of different view-angles in the 2D observations. To this end, we propose an end-to-end framework to reconstruct the 3D mesh from a single image, where the reconstructed mesh is not only view-centered (i.e. its 3D pose respects the viewpoint of the 2D observation) but also preliminarily represented as a composition of volumetric 3D primitives before being further deformed into the fine-grained mesh to capture the shape details. In particular, the usage of volumetric primitives is motivated from the assumption that there generally exists some similar shape parts shared across various object categories, learning to estimate the primitive-based 3D model thus becomes more generalizable to the unseen categories. Furthermore, we advance to propose a novel mesh augmentation strategy, CvxRearrangement, to enrich the distribution of training shapes, which contributes to increasing the robustness of our proposed model and achieves better generalization. Extensive experiments demonstrate that our proposed method provides superior performance on both unseen and seen classes in comparison to several representative baselines of single-view 3D reconstruction.

原文English
主出版物標題IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022
發行者Institute of Electrical and Electronics Engineers Inc.
頁面2001-2008
頁數8
ISBN(電子)9781665479271
DOIs
出版狀態Published - 2022
事件2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022 - Kyoto, 日本
持續時間: 23 10月 202227 10月 2022

出版系列

名字IEEE International Conference on Intelligent Robots and Systems
2022-October
ISSN(列印)2153-0858
ISSN(電子)2153-0866

Conference

Conference2022 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2022
國家/地區日本
城市Kyoto
期間23/10/2227/10/22

指紋

深入研究「Improving Single-View Mesh Reconstruction for Unseen Categories via Primitive-Based Representation and Mesh Augmentation」主題。共同形成了獨特的指紋。

引用此