Bit allocation for video coding with temporal-spatial tradeoff

Bit allocation with joint temporal-spatial consideration for video coding and motion-compensated frame interpolation at low bit rates is investigated. In contrast with traditional approaches where the frame-rate control and the quantization parameter selection are treated separately, they are jointly optimized in this work. In our framework, an affine warping based on triangular patch is adopted to interpolate skipped frames. Both skipped and coded frames contribute to the operational rate-distortion (R-D) function, and bit allocation is formulated as an optimization problem. A heuristic approach using iterative greedy pruning is proposed to solve this problem. It provides a suboptimal solution at a reasonable complexity. Experimental results show that, with various temporal (frame skipping) and spatial (quantizer) combinations, the proposed algorithm can achieve an average PSNR that is 0.3 similar to 1.0 dB better than that of ITU-T H.263+/TMN8. Moreover, consistency of visual quality has significantly improved as evidenced by the reduced PSNR variance among frames.