Finding Robust 2D-to-3D Correspondence with LSTM Score Estimation for Camera Localization

Tsu Kuan Huang, Po Heng Chen, Li Yang Wang, Kuan Wen Chen*

*此作品的通信作者

研究成果: Conference contribution同行評審

摘要

2D-to-3D correspondence estimation is the key step of 3D model-based image localization, and most of the existing research in this field focuses on improving the feature matching performance. Even with the best feature matching method, there are still some outliers, and thus, almost all the methods simply apply the RANSAC algorithm to select the inliers and estimate the camera pose afterwards. However, the reliability of RANSAC depends considerably on the inlier ratio. Once the inlier ratio decreases, for example a challenging scenario occurs, it will be unable to select the inliers well and lead to a worse camera pose. In this study, we attempted to build a neural network to learn the geometric relationship between 2D images and the 3D model to select the correct correspondence from the initial 2D-to-3D matching results to improve the performance of camera localization. Because the number of inputs, i.e., the number of 2D-to-3D correspondences, is unknown and different for each image, we propose a PointNet-based Geometric Consistency Network (GCC-Net) for the correct correspondence estimation and an LSTM-based Hypothesis Rating Network (HR-Net) to enhance GCC-Net with the camera localization loss. Experimental results showed that the proposed method outperforms RANSAC considerably on the camera pose estimation, particularly when the inlier ratio of the initial correspondence was low.

原文English
主出版物標題IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2021
發行者Institute of Electrical and Electronics Engineers Inc.
頁面2232-2238
頁數7
ISBN(電子)9781665417143
DOIs
出版狀態Published - 2021
事件2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2021 - Prague, Czech Republic
持續時間: 27 9月 20211 10月 2021

出版系列

名字IEEE International Conference on Intelligent Robots and Systems
ISSN(列印)2153-0858
ISSN(電子)2153-0866

Conference

Conference2021 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2021
國家/地區Czech Republic
城市Prague
期間27/09/211/10/21

指紋

深入研究「Finding Robust 2D-to-3D Correspondence with LSTM Score Estimation for Camera Localization」主題。共同形成了獨特的指紋。

引用此