Automated Pixel-Level Pavement Crack Detection on 3D Asphalt Surfaces with a Recurrent Neural Network

A recurrent neural network (RNN) called CrackNet-R is proposed in the article for fully automated pixel-level crack detection on three-dimensional (3D) asphalt pavement surfaces. In the article, a new recurrent unit, gated recurrent multilayer perceptron (GRMLP), is proposed to recursively update the internal memory of CrackNet-R. Unlike the widely used long short-term memory (LSTM) and gated recurrent unit (GRU), GRMLP is intended for deeper abstractions on the inputs and hidden states by conducting multilayer nonlinear transforms at gating units. CrackNet-R implements a two-phase sequence processing: sequence generation and sequence modeling. Sequence generation is specifically developed in the study to find the best local paths that are most likely to form crack patterns. Sequence modeling predicts timely probabilities of the input sequence being a crack pattern. In terms of sequence modeling, GRMLP slightly outperforms LSTM and GRU by using only one more nonlinear layer at each gate. In addition to sequence processing, an output layer is proposed to produce pixel probabilities based on timely probabilities predicted for sequences. The proposed output layer is critical for pixel-perfect accuracy, as it accomplishes the transition from sequence-level learning to pixel-level learning. Using 3,000 diverse 3D images, the training of CrackNet-R is completed through optimizing sequence modeling, sequence generation, and the output layer serially. The experiment using 500 testing pavement images shows that CrackNet-R can achieve high Precision (88.89%), Recall (95.00%), and F-measure (91.84%) simultaneously. Compared with the original CrackNet, CrackNet-R is about four times faster and introduces tangible improvements in detection accuracy.