Integrating Low-Cost LiDAR and Stereo Camera for 3D Object Detection with Lidar-Stereo Fusion and Cost Volume Enhancement

We propose a fusion network that integrates LiDAR and stereo images multiple times for 3D object detection. It fuses projected left/right LiDAR maps with stereo camera images at the beginning of the network, then adopts the LiDAR maps again for cost volume enhancement with a novel parallel fusion network (PFNet). The PFNet combines two distinctive strategies: a learnable strategy and a physical modeling strategy. The learnable design modifies HeirCCVNorm (Hierarchical Conditional Cost Volume Normalization). The physical modeling involves Gaussian embedding of LiDAR signals. Our simulations showed that the proposed network has better 3D object detection performance than a recent acclaimed state-of-the-art method when using the same dataset for training.