Near-infrared and visible fusion for image enhancement based on multi-scale decomposition with rolling WLSF

Generally, the visible (VIS) image (RGB image is used here) captured by the VIS band sensor is often blurred by the influence of haze, strong light, or dark environment. However, the near-infrared (NIR) band sensor can avoid the interference of harsh environments, and the captured NIR image has advantages in details that the VIS image does not have. In this paper, we propose a multi-scale decomposition (MSD) framework based on rolling weighted least squares filter (Rolling WLSF) for image fusion. This method combines the details of the VIS and NIR images to generate an enhanced fusion image. The proposed method first dehazes the original VIS image and extracts the details hidden under the influence of haze and other factors. Then, the designed MSD module decomposes the VIS image and the NIR image into detail layers of different scales, which can well retain the detailed texture of the image, and the multi-scale structure can avoid artifacts. In addition, a detail enhancement module (Boosting) is used to enhance the medium detail layer, which can well enhance the large-scale detail contour in the image. The weighted fusion of the detail layer can prevent the degraded NIR image from affecting the details of the fused image. We have used a variety of evaluation methods to evaluate its performance, and experiments have verified that the proposed method has excellent performance and can better enhance the detailed texture of the image.