Colorization of Depth Map via Disentanglement

Chung Sheng Lai*, Zunzhi You, Ching Chun Huang, Yi Hsuan Tsai, Wei Chen Chiu


研究成果: Conference contribution同行評審


Vision perception is one of the most important components for a computer or robot to understand the surrounding scene and achieve autonomous applications. However, most of the vision models are based on the RGB sensors, which in general are vulnerable to the insufficient lighting condition. In contrast, the depth camera, another widely-used visual sensor, is capable of perceiving 3D information and being more robust to the lack of illumination, but unable to obtain appearance details of the surrounding environment compared to RGB cameras. To make RGB-based vision models workable for the low-lighting scenario, prior methods focus on learning the colorization on depth maps captured by depth cameras, such that the vision models can still achieve reasonable performance on colorized depth maps. However, the colorization produced in this manner is usually unrealistic and constrained to the specific vision model, thus being hard to generalize for other tasks to use. In this paper, we propose a depth map colorization method via disentangling appearance and structure factors, so that our model could 1) learn depth-invariant appearance features from an appearance reference and 2) generate colorized images by combining a given depth map and the appearance feature obtained from any reference. We conduct extensive experiments to show that our colorization results are more realistic and diverse in comparison to several image translation baselines.

主出版物標題Computer Vision – ECCV 2020 - 16th European Conference, 2020, Proceedings
編輯Andrea Vedaldi, Horst Bischof, Thomas Brox, Jan-Michael Frahm
發行者Springer Science and Business Media Deutschland GmbH
出版狀態Published - 2020
事件16th European Conference on Computer Vision, ECCV 2020 - Glasgow, United Kingdom
持續時間: 23 八月 202028 八月 2020


名字Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
12352 LNCS


Conference16th European Conference on Computer Vision, ECCV 2020
國家/地區United Kingdom


深入研究「Colorization of Depth Map via Disentanglement」主題。共同形成了獨特的指紋。