Shot boundary detection in video using dual-stage optimized VGGNet based feature fusion and classification

Shot boundary detection (SBD) in video sequences is a key process in the analysis, retrieval, and summarization tasks of video content. The major goal of SBD is to detect transition and their boundaries among the subsequent shots by analyzing the spatial appearance and temporal motion information of the video. This paper proposed a deep learning-based intelligent SBD model, which can detect abrupt transition (AT) and gradual transition (GT) concurrently from video sequences. The proposed model follows the Dual Stage Fused Feature Extraction(DSFFE) process using an optimized VGGNet architecture. Initially, the input video data is converted into several frames, and then a pre-processing step is performed using the Improved Bilateral Filter (IBF). Then, Dual Stage Fused Feature Extraction is performed using VGGNet for extracting deep and spatial appearance-temporal motion features from video sequences. Further, a continuity matrix is created using the Inter-frame Euclidean Threshold (IET) to find the dissimilarity measure. Finally, shot transitions are classified using the Softmax Classifier, which categorizes AT and GT transitions as Fade in/out, cut, and dissolve. Especially the VGG network model weights are updated using an algorithm called Red Fox Optimization (RFO) to minimize the loss function. The proposed model is implemented using TRECVID and VideoSeg datasets on the MATLAB platform. The performance outcomes show that the proposed SBD model achieves an average recall, precision, and f1-score of 98.89%, 98.15%, and 98.86%, respectively, which is comparatively better than other models.