Video colorization method based on fusion of multi-source colorization results using dual reference frames

To better leverage reference frame information,a colorization method for black-and-white videos based on fusion of multi-source colorization results using dual reference frames is proposed. First,a hard attention fusion submodule is employed to fuse color information from two reference frames,preventing color blurring caused by unreasonable reference information during the coloring of the dual-frame semantic matching module. Then,a multi-source colorization result fusion module fuses colorization results obtained through the dual-frame optical flow propagation module,the dual-frame semantic matching module and occlusion information,thereby producing a better colorization result. Experimental results show that the peak signal to noise (PSNR),the structural similarity (SSIM),and the color distribution consistency index (CDC)of the method on the Davis30 test dataset are 37. 36 dB,0. 980 5,0. 003 748,respectively. This demonstrates that the method can fully utilize information from dual reference frames to colorize grayscale frames through multiple fusion methods and generate aesthetically pleasing and temporally consistent colorization results. © 2024 Southeast University. All rights reserved.