Automatic Camera-based Road Defect Detection with Neural Networks

The perception of the road condition is crucial to the autonomous driving task and is a main factor for the comfort and safety of the driver. Real time detection of dangerous hazards and road defects is a key element for achieving this goal. At the Global Road Damage Detection Challenge 2020 (GRDDC 2020) participants are presented with a dataset of over 10000 labeled images of international roads with classified and localized damages. After successfully partaking in the challenge, Dr. Felix Kortmann and Kevin Talits carried out studies for the harmonization of speed and accuracy of a YOLOv5 detection model for road defects. Impacts of single optimization tools are investigated and combinations suitable for the field of application are presented. For a better estimation of the performance in a car, a Jetson Nano was used as an edge device with minimal computational requirements and compared to the detection performance of a Nvidia GTX 1080 Ti. As evaluation metric mAP and F1-Score are used to compare the findings to other networks. The best performing network from the GRDDC 2020 scored a F1-Score of 0.676 with 3.08 FPS on a Nvidia GTX 1080 Ti. After the application of the best suited tools, even on the weak Jetson Nano a model with a comparable F1-Score of 0.653 can be run at 1.76 FPS and with lowering the accuracy to a still good performing 0.577 a detection rate of 8.40 FPS is possible. These networks run respectively at 36.10 FPS and 107.53 FPS on the Nvidia GTX 1080 Ti, achieving real time capability. The best network with a F1-Score of 0.715 can only run on the Nvidia GTX 1080 Ti with 1.58 FPS. In consideration of the driving situation a higher accuracy with slower detection speed or vice versa can be feasible. A car on a straight, free country road with speeds up to 100 km/h can trade off some detection speed for a higher accuracy, because of the good visible road ahead. But with 50 km/h in a city with cars in the front and no free view of the street ahead a faster detection is desirable. Different networks for changing demands can be helpful in managing the diverse driving challenges. © 2022, VDI Verlag GMBH. All rights reserved.