Fine-Grained Detection of Pavement Distress Based on Integrated Data Using Digital Twin

The automated detection of distress such as cracks or potholes is a key basis for assessing the condition of pavements and deciding on their maintenance. A fine-grained pavement distress-detection algorithm based on integrated data using a digital twin is proposed to solve the challenges of the insufficiency of high-quality negative samples in specific scenarios An asphalt pavement background model is created based on UAV-captured images, and a lightweight physical engine is used to randomly render 5 types of distress and 3 specific scenarios to the background model, generating a digital twin model that can provide virtual distress data. The virtual data are combined with real data in different virtual-to-real ratios (0:1 to 5:1) to form an integrated dataset and used to fully train deep object detection networks for fine-grained detection. The results show that the YOLOv5 network with the virtual-to-real ratio of 3:1 achieves the best average precision for 5 types of distress (asphalt pavement MAP: 75.40%), with a 2-fold and 1.5-fold improvement compared to models developed without virtual data and with traditional data augmentation, respectively, and achieves over 40% recall in shadow, occlusion and blur. The proposed approach could provide a more reliable and refined automated method for pavement analysis in complex scenarios.