Traffic sign detection plays an essential role in the technology of self-driving vehicles. Recently, deep learning methods have significantly advanced the field of traffic sign recognition. Nevertheless, faced with increasingly complex traffic scenarios, practical applications of traffic sign detection still encounter challenges, including false detections, missed detections, and reduced accuracy. To tackle these challenges, we introduce an enhanced algorithm for traffic sign detection built on the YOLOv8 model, aimed at improving performance and accuracy. Firstly, a Multi-Scale Convolutional Attention (MSCA) module is embedded into the backbone architecture to improve the model's feature extraction capabilities at multiple scales, enhancing its focus on target areas. Furthermore, a small object detection layer is added during the detection phase, effectively reducing the false positive and missed detection rates for small objects. Finally, we present the Inner-WIoU loss function for bounding boxes, which integrates a dynamic non-monotonic focusing mechanism with auxiliary boxes. This boosts the model's capability to identify objects and enhances overall detection performance. The findings from the experiments demonstrate that the enhanced algorithm obtains an mAP0.5 value of 83.8% on the TT100K dataset, indicating a 7.8% increase compared to the baseline YOLOv8 algorithm. When compared to existing algorithms, the proposed method demonstrates competitive performance.