There are numerous applications for visual object tracking in computer vision, and it aims to attain the highest tracking reliability and accuracy depending on the applications' varied evaluation criteria. Although DCF tracking algorithms have been used in the past and achieved great results, they are still unable to provide robust tracking under difficult conditions such as occlusion, scale fluctuation, quick motion, and motion blur. To address the instability during tracking brought on by various challenging issues in complex sequences, we present a novel framework termed improved spatial-temporal regularized correlation filters (I-STRCF) to integrate with instantaneous motion estimation and Kalman filter for visual object tracking which can minimize the possible tracking failure during tracking as the tracking model update itself with Kalman filter throughout the video sequence. We also include a unique scale estimate criterion called average peak-to-correlation energy to address the issue of target loss brought on by scale change. Using the previously calculated motion data, the suggested method predicts the potential scale region of the target in the current frame, and then the target model updates the target object's position in successive frames. Additionally, we examine the factors affecting how well the suggested framework performs in extensive experiments. The experimental results show that this proposed framework achieves the best visual tracking for computer vision and performs better than STRCF on Temple Color-128 datasets for object tracking attributes. Our framework produces greater AUC improvements for the scale variation, background clutter, lighting variation, occlusion, out-of-plane rotation, and deformation properties when compared to STRCF. Our system gets much better improvements than its rivals in terms of performance and robustness for sporting events.