In this fast world, the researchers are trying to develop strong artificial intelligence (AI) applications such as autonomous vehicles, self-driving cars, improved closed circuit television (CCTV) systems, etc. in the transportation field to ensure safety and a sophisticated life. In this scenario, the object detection is performed by the You Look Only Once (YOLO) Version 2 (Yolov2) and YOLO Version 3 (Yolov3) algorithms for detecting the objects. However, no method is available to fulfil the current requirements in terms of efficiency and effectiveness. To overcome these issues, this paper proposes an improved YOLO Version 4 (Yolov4) algorithm to improve the accuracy and detection speed of vehicles. Moreover, a new Mosiac data augmentation technique that incorporates a Bidirectional Feature Pyramid Network (BiFPN) is implemented in the existing Yolov4 algorithm for identifying the faster vehicles, and it is also used to reduce the false detection rate with rapid detection speed. In addition, a refined DeepSort technique for multiobject tracking (MOT) is also used to avoid common errors like occlusion, similar objects, etc. in this work. The proposed system was evaluated by using live videos and images and obtained 88.75% accuracy, 87.37% precision, and 89.21% recall that are superior to other systems.