End-to-End Variable-Rate Learning-Based Depth Compression Guided by Deep Correlation Features

The progress in the field of 3D video, particularly depth maps, is leading to the emergence of various technologies such as augmented, virtual, and mixed reality that have a wide range of applications in smart cities, intelligent transportation, AI-enabled farms, healthcare, education, industry, and more. Additionally, the future development of the Internet of Things (IoT) heavily depends on incorporating 3D vision and depth perception into machines like autonomous cars, robots, and drones, so that they effectively perceive their surroundings similar to how humans do. However, traditional compression methods that focus only on texture are not suitable for efficiently handle the large volume of depth maps due to the distinct features between texture and depth. To tackle this challenge, we aim to propose a model for compressing depth maps. Our approach utilizes a learning variable-rate method combined with a conditional quality-controllable autoencoder. The model consists of an encoder that automatically extracts features from depth maps using an optimized Convolutional Neural Network. This latter consists of an initial layer that uses predetermined wedgelet filters, succeeded by a VGG19 model. Additionally, we utilize a technique for classifying image styles based on Learnt Deep Correlation Features in order to learn deep features that distinguish depth maps from texture images. Our model objective is to optimize a loss function with multiple terms, which maintains the accuracy of depth discontinuities in the reconstructed output while also ensuring high-quality synthesis. By capturing and preserving deep features specific to depth maps, our end-to-end network achieves better R/D compression performances compared to related methods and depth-oriented 3D-HEVC standard.