Real-time Interpolation Method For Sparse LiDAR Point Cloud Using RGB Camera

LiDAR (Light Detection and Ranging) sensor-based mapping and navigation is one of the fundamental techniques for achieving autonomous driving capabilities in urban scenarios. LiDARs can generate long-distance omni-directional measurements of its surrounding that is crucial for object detection and obstacle avoidance for autonomous vehicles. Currently, the most popular LiDAR sensors used for the application of autonomous driving are 360-degree rotating multi-layer type sensor. These are expensive for general use and mainly suffer from poor vertical resolution. In this study, we propose a method of LiDAR point cloud interpolation in real-time using information from an RGB camera. We propose a method to treat sparse point cloud as a depth image which enables us to apply depth enhancement methods to the point cloud. Additionally, we propose a new depth enhancement method with image segmentation for point cloud and compare its accuracy with existing methods. From the results, we present the usefulness of introducing depth image and applying the new depth enhancement method for LiDAR point cloud.