Diffusion-based remote sensing image fusion for classification

The objective of remote sensing image fusion is to merge remote sensing images from multiple data sources to generate high-quality images with elevated spatial and spectral resolution. The resulting images of superior quality can enhance the geometric precision of remote sensing images, augment the quantity and detail of feature information, augment classification accuracy, and facilitate dynamic monitoring across many applications. In fields such as agriculture, forestry, urban planning, and environmental monitoring, high-quality images can enhance the precision and resolution of interpretation and facilitate the extraction of target information. A typical example of remote sensing image fusion is the fusion of low-resolution hyperspectral image (HSI) and light detection and ranging (LiDAR) data. The majority of the various remote sensing image fusion methodologies that have been put forth thus far conduct fusion studies regarding the specific data characteristics of HSI and LiDAR, with only a limited focus on the correlation between the two regarding their spatial distribution. To address this issue, this paper proposes an image fusion classification network based on the denoising diffusion probabilistic model (DDPM). DDPM can be trained to learn the data distribution of an image and generate a new image with the same distribution by inverse diffusion, it is frequently employed in the field of image generation research, yet its application to image fusion research remains unexplored. Therefore, we use DDPM to extract the spatial feature distribution of correlations derived from HSI-LiDAR data pairs, fuse them with hyperspectral features from HSI, and then train them jointly. Experimental results demonstrate that intermediate activation at a specific time step in the inverse diffusion process can effectively extract feature information from HSI and LiDAR, resulting in a significant improvement in classification accuracy after fusion with hyperspectral features of HSI. Furthermore, even when replacing the inputs with multispectral images (MSI) and synthetic aperture radar (SAR) data, the model still maintains considerable performance.