Uncertainty-Encoded Multi-Modal Fusion for Robust Object Detection in Autonomous Driving

Yang Lou, Qun Song, Qian Xu, Rui Tan, Jianping Wang

研究成果: Conference contribution同行評審

摘要

Multi-modal fusion has shown initial promising results for object detection of autonomous driving perception. However, many existing fusion schemes do not consider the quality of each fusion input and may suffer from adverse conditions on one or more sensors. While predictive uncertainty has been applied to characterize single-modal object detection performance at run time, incorporating uncertainties into the multi-modal fusion still lacks effective solutions due primarily to the uncertainty's cross-modal incomparability and distinct sensitivities to various adverse conditions. To fill this gap, this paper proposes Uncertainty-Encoded Mixture-of-Experts (UMoE) that explicitly incorporates single-modal uncertainties into LiDAR-camera fusion. UMoE uses individual expert network to process each sensor's detection result together with encoded uncertainty. Then, the expert networks' outputs are analyzed by a gating network to determine the fusion weights. The proposed UMoE module can be integrated into any proposal fusion pipeline. Evaluation shows that UMoE achieves a maximum of 10.67%, 3.17%, and 5.40% performance gain compared with the state-of-the-art proposal-level multi-modal object detectors under extreme weather, adversarial, and blinding attack scenarios.

原文English
主出版物標題ECAI 2023 - 26th European Conference on Artificial Intelligence, including 12th Conference on Prestigious Applications of Intelligent Systems, PAIS 2023 - Proceedings
編輯Kobi Gal, Kobi Gal, Ann Nowe, Grzegorz J. Nalepa, Roy Fairstein, Roxana Radulescu
發行者IOS Press BV
頁面1593-1600
頁數8
ISBN(電子)9781643684369
DOIs
出版狀態Published - 28 9月 2023
事件26th European Conference on Artificial Intelligence, ECAI 2023 - Krakow, Poland
持續時間: 30 9月 20234 10月 2023

出版系列

名字Frontiers in Artificial Intelligence and Applications
372
ISSN(列印)0922-6389

Conference

Conference26th European Conference on Artificial Intelligence, ECAI 2023
國家/地區Poland
城市Krakow
期間30/09/234/10/23

指紋

深入研究「Uncertainty-Encoded Multi-Modal Fusion for Robust Object Detection in Autonomous Driving」主題。共同形成了獨特的指紋。

引用此