Automatic Pavement Defect Detection and Classification Using RGB-Thermal Images Based on Hierarchical Residual Attention Network.

A convolutional neural network based on an improved residual structure is proposed to implement a lightweight classification model for the recognition of complex pavement conditions, which uses RGB-thermal as input and embeds an attention module to adjust the spatial, as well as channel, information of the images. The best prediction accuracy of the proposed model is 98.88%, while the RGB-thermal is used as input and an attention mechanism is used. The attention mechanism increases the attention to detail of the image and regulates the use of image channels, which enhances the final performance of the model. It is also compared with state-of-the-art (SOTA) deep learning models, indicating our model has fewer parameters, shorter training time, and higher recognition accuracy compared to existing image classification models. A visualization method incorporating gradient-weighted class activation mapping (Grad-CAM) is proposed to analyze the classification results, comparing the data the model learns from the images under different input data.

Deep Learning Based Infrared Thermal Image Analysis of Complex Pavement Defect Conditions Considering Seasonal Effect.

Deep learning techniques underpinned by extensive data sources encompassing complex pavement features have proven effective in early pavement damage detection. With pavement features exhibiting temperature variation, inexpensive infra-red imaging technology in combination with deep learning techniques can detect pavement damages effectively. Previous experiments based on pavement data captured during summer sunny conditions when subjected to SA-ResNet deep learning architecture technique demonstrated 96.47% prediction accuracy. This paper has extended the same deep learning approach to a different dataset comprised of images captured during winter sunny conditions to compare the prediction accuracy, sensitivity and recall score with summer conditions. The results suggest that irrespective of the prevalent weather season, the proposed deep learning algorithm categorises pavement features around 92% accurately (95.18% in summer and 91.67% in winter conditions), suggesting the beneficial replacement of one image type with other. The data captured in sunny conditions during summer and winter show prediction accuracies of DC = 96.47% > MSX = 95.24% > IR-T = 93.83% and DC = 94.14% > MSX = 90.69% > IR-T = 90.173%, respectively. DC images demonstrated a sensitivity of 96.47% and 94.20% for summer and winter conditions, respectively, to demonstrate that reliable categorisation is possible with deep learning techniques irrespective of the weather season. However, summer conditions showing better overall prediction accuracy than winter conditions suggests that inexpensive IR-T imaging cameras with medium resolution levels can still be an economical solution, unlike expensive alternate options, but their usage has to be limited to summer sunny conditions.