Key frame extraction method with global information balance

Key frame extraction can provide evidence for traffic violation detection, which is essential to support administrative punishment. However, the existing key frame extraction methods failed to model context information in complex semantic cases, such as failing to yield to pedestrian. To address this problem, we have proposed a key frame extraction model with global information balance (GIB), an intelligent vehicle violation screenshot method based on balancing the global information of video frames. The proposed GIB extracts three screenshots from the videos of vehicles failing to yield to pedestrians at crosswalks without signals. First, the proposed GIB defines the extraction of global information based on trajectories, comprising spatial structure and motion attributes as feature factors. Then, based on semantic correlation analysis for global information, relational entity filtering is implemented to avoid the interference of non-key entities and improve the effectiveness of the features. Finally, a search and pruning policy prioritizing mutual information is designed to maximize the global information entropy among preserved nodes to ensure the optimal prediction solution in case of a large global search solution space. The policy is implemented in the key frame prediction task in the Seq2Seq model based on the attention mechanism. The results of several experiments confirm the superior performance of the proposed method compared to conventional methods in terms of the evaluation of frame-time differential, perceptual hashing, and subjective scoring. For example, the perceptual hashing values of the proposed method were 10.5% and 6.7% greater than semantic correlation extraction and image similarity extraction, respectively, which are baseline methods based on local information.