Deformable Feature Fusion Network for Multi-Modal 3D Object Detection

LiDAR and cameras are two widely used sensors in 3D object detection. LiDAR point clouds show geometry knowledge of objects, while RGB images provide semantic information, such as color and texture. How to effectively fuse their features is the key to improving detection performance. This paper proposes a Deformable Feature Fusion Network, which performs LiDAR-camera fusion in a flexible way. We present multi-modal features in the bird's-eye view(BEV), and build a Deformable-Attention Fusion(DAF) module to conduct feature fusion. Besides fusion methods, feature alignment is also important in multi-modal detection. Data augmentation of point clouds may change the projection relationship between RGB images and LiDAR point clouds and causes feature misalignment. We introduce a Feature Alignment Transform(FAT) module and alleviate the problem without introducing any trainable parameters. We conduct experiments on the KITTI dataset to evaluate the effectiveness of proposed modules and the experiment results show that our method outperforms most existing methods.