Interpretable & Explainable Machine Learning for Ultrasonic Defect Sizing.

Despite its popularity in literature, there are few examples of machine learning (ML) being used for industrial nondestructive evaluation (NDE) applications. A significant barrier is the 'black box' nature of most ML algorithms. This paper aims to improve the interpretability and explainability of ML for ultrasonic NDE by presenting a novel dimensionality reduction method: Gaussian feature approximation (GFA). GFA involves fitting a 2D elliptical Gaussian function an ultrasonic image and storing the seven parameters that describe each Gaussian. These seven parameters can then be used as inputs to data analysis methods such as the defect sizing neural network presented in this paper. GFA is applied to ultrasonic defect sizing for inline pipe inspection as an example application. This approach is compared to sizing with the same neural network, and two other dimensionality reduction methods (the parameters of 6 dB drop boxes and principal component analysis), as well as a convolutional neural network applied to raw ultrasonic images. Of the dimensionality reduction methods tested, GFA features produce the closest sizing accuracy to sizing from the raw images, with only a 23% increase in RMSE, despite a 96.5% reduction in the dimensionality of the input data. Implementing ML with GFA is implicitly more interpretable than doing so with principal component analysis or raw images as inputs, and gives significantly more sizing accuracy than 6 dB drop boxes. Shapley additive explanations (SHAP) are used to calculate how each feature contributes to the prediction of an individual defect's length. Analysis of SHAP values demonstrates that the GFA-based neural network proposed displays many of the same relationships between defect indications and their predicted size as occur in traditional NDE sizing methods.

Exploring the potential of using simulation games for engaging with sheep farmers about lameness recognition.

Introduction: Computer simulation games are increasingly being used in agriculture as a promising tool to study, support and influence real-life farming practices. We explored the potential of using simulation games to engage with sheep farmers on the ongoing challenge of reducing lameness. Working with UK stakeholders, we developed a game in which players are challenged with identifying all the lame sheep in a simulated flock. Here, we evaluate the game's potential to act as a tool to help assess, train and understand farmers' ability to recognize the early signs of lameness. Methods: Participants in the UK were invited to play the game in an online study, sharing with us their in-game scores alongside information relating to their real-life farming experience, how they played the game, and feedback on the game. Mixed methods were used to analyze this information in order to evaluate the game. Quantitative analyses consisted of linear modeling to test for statistical relationships between participants' in-game recall (% of the total number of lame sheep that were marked as lame), and the additional information they provided. Qualitative analyses of participants' feedback on the game consisted of thematic analysis and a Likert Scale questionnaire to contextualize the quantitative results and identify additional insights from the study. Results: Quantitative analyses identified no relationships between participants' (n = 63) recall scores and their real life farming experience, or the lameness signs they looked for when playing the game. The only relationship identified was a relationship between participants' recall score and time spent playing the game. Qualitative analyses identified that participants did not find the game sufficiently realistic or engaging, though several enjoyed playing it and saw potential for future development. Qualitative analyses also identified several interesting and less-expected insights about real-life lameness recognition practices that participants shared after playing the game. Discussion: Simulation games have potential as a tool in livestock husbandry education and research, but achieving the desired levels of realism and/or engagingness may be an obstacle to realizing this. Future research should explore this potential further, aided by larger budgets and closer collaboration with farmers, stockpeople, and veterinarians.

Uncertainty Quantification for Deep Learning in Ultrasonic Crack Characterization.

Deep learning for nondestructive evaluation (NDE) has received a lot of attention in recent years for its potential ability to provide human level data analysis. However, little research into quantifying the uncertainty of its predictions has been done. Uncertainty quantification (UQ) is essential for qualifying NDE inspections and building trust in their predictions. Therefore, this article aims to demonstrate how UQ can best be achieved for deep learning in the context of crack sizing for inline pipe inspection. A convolutional neural network architecture is used to size surface breaking defects from plane wave imaging (PWI) images with two modern UQ methods: deep ensembles and Monte Carlo dropout. The network is trained using PWI images of surface breaking defects simulated with a hybrid finite element / ray-based model. Successful UQ is judged by calibration and anomaly detection, which refer to whether in-domain model error is proportional to uncertainty and if out of training domain data is assigned high uncertainty. Calibration is tested using simulated and experimental images of surface breaking cracks, while anomaly detection is tested using experimental side-drilled holes and simulated embedded cracks. Monte Carlo dropout demonstrates poor uncertainty quantification with little separation between in and out-of-distribution data and a weak linear fit ( R=0.84 ) between experimental root-mean-square-error and uncertainty. Deep ensembles improve upon Monte Carlo dropout in both calibration ( R=0.95 ) and anomaly detection. Adding spectral normalization and residual connections to deep ensembles slightly improves calibration ( R=0.98 ) and significantly improves the reliability of assigning high uncertainty to out-of-distribution samples.

Domain Adapted Deep-Learning for Improved Ultrasonic Crack Characterization Using Limited Experimental Data.

Deep learning is an effective method for ultrasonic crack characterization due to its high level of automation and accuracy. Simulating the training set has been shown to be an effective method of circumventing the lack of experimental data common to nondestructive evaluation (NDE) applications. However, a simulation can neither be completely accurate nor capture all variability present in the real inspection. This means that the experimental and simulated data will be from different (but related) distributions, leading to inaccuracy when a deep learning algorithm trained on simulated data is applied to experimental measurements. This article aims to tackle this problem through the use of domain adaptation (DA). A convolutional neural network (CNN) is used to predict the depth of surface-breaking defects, with in-line pipe inspection as the targeted application. Three DA methods across varying sizes of experimental training data are compared to two non-DA methods as a baseline. The performance of the methods tested is evaluated by sizing 15 experimental notches of length (1-5 mm) and inclined at angles of up to 20° from the vertical. Experimental training sets are formed with between 1 and 15 notches. Of the DA methods investigated, an adversarial approach is found to be the most effective way to use the limited experimental training data. With this method, and only three notches, the resulting network gives a root-mean-square error (RMSE) in sizing of 0.5 ± 0.037 mm, whereas with only experimental data the RMSE is 1.5 ± 0.13 mm and with only simulated data it is 0.64 ± 0.044 mm.

Deep Learning for Ultrasonic Crack Characterization in NDE.

Machine learning for nondestructive evaluation (NDE) has the potential to bring significant improvements in defect characterization accuracy due to its effectiveness in pattern recognition problems. However, the application of modern machine learning methods to NDE has been obstructed by the scarcity of real defect data to train on. This article demonstrates how an efficient, hybrid finite element (FE) and ray-based simulation can be used to train a convolutional neural network (CNN) to characterize real defects. To demonstrate this methodology, an inline pipe inspection application is considered. This uses four plane wave images from two arrays and is applied to the characterization of cracks of length 1-5 mm and inclined at angles of up to 20° from the vertical. A standard image-based sizing technique, the 6-dB drop method, is used as a comparison point. For the 6-dB drop method, the average absolute error in length and angle prediction is ±1.1 mm and ±8.6°, respectively, while the CNN is almost four times more accurate at ±0.29 mm and ±2.9°. To demonstrate the adaptability of the deep learning approach, an error in sound speed estimation is included in the training and test set. With a maximum error of 10% in shear and longitudinal sound speed, the 6-dB drop method has an average error of ±1.5 mmm and ±12°, while the CNN has ±0.45 mm and ±3.0°. This demonstrates far superior crack characterization accuracy by using deep learning rather than traditional image-based sizing.