Real time demosaicking and superresolution of multispectral images

The human visual system registers electromagnetic waves lying in a 390 to 700 nm wavelength range. While visible light provides humans with sufficient guidance for everyday activities, a large amount of information remains unregistered. However, electromagnetic radiation outside the visible range can be registered using cameras and sensors. Due to the multiplexing of visible light and additional wavelengths, the resolution drops significantly. To improve the resolution, we propose a GPU based joint method for demosaicking, denoising and superresolution. In order to interpolate missing pixel values for all four wavelengths, we first extract high pass image features from all types of pixels in the mosaic. Using this information we perform directional interpolation, to preserve continuities of edges present in all four component images. After the initial interpolation, we introduce high spatial content from other frequency bands, giving preference to original over the interpolated edges. Moreover, we perform the refinement and upsampling of the demosaicked image by introducing information from previous frames. Motion compensation relies on a subpixel block-based motion estimation algorithm, relying on all 4 chromatic bands, and performs regularization to reduce estimation errors and related artifacts in the interpolated images. We perform experiments using the mosaic consisting of red, green, blue and near-infrared pixels (850nm). The proposed algorithm is implemented on Jetson TX 2 platform, achieving 120 fps at QVGA resolution. It operates recursively, requiring only one additional frame buffer for the previous results. The results of the proposed method compared favorably to the state-of-the-art multispectral demosaicing methods.