Computerized Ultrasonic Imaging Inspection: From Shallow to Deep Learning.

For many decades, ultrasonic imaging inspection has been adopted as a principal method to detect multiple defects, e.g., void and corrosion. However, the data interpretation relies on an inspector's subjective judgment, thus making the results vulnerable to human error. Nowadays, advanced computer vision techniques reveal new perspectives on the high-level visual understanding of universal tasks. This research aims to develop an efficient automatic ultrasonic image analysis system for nondestructive testing (NDT) using the latest visual information processing technique. To this end, we first established an ultrasonic inspection image dataset containing 6849 ultrasonic scan images with full defect/no-defect annotations. Using the dataset, we performed a comprehensive experimental comparison of various computer vision techniques, including both conventional methods using hand-crafted visual features and the most recent convolutional neural networks (CNN) which generate multiple-layer stacking for representation learning. In the computer vision community, the two groups are referred to as shallow and deep learning, respectively. Experimental results make it clear that the deep learning-enabled system outperformed conventional (shallow) learning schemes by a large margin. We believe this benchmarking could be used as a reference for similar research dealing with automatic defect detection in ultrasonic imaging inspection.

Computerized Hammer Sounding Interpretation for Concrete Assessment with Online Machine Learning.

Developing efficient Artificial Intelligence (AI)-enabled systems to substitute the human role in non-destructive testing is an emerging topic of considerable interest. In this study, we propose a novel hammering response analysis system using online machine learning, which aims at achieving near-human performance in assessment of concrete structures. Current computerized hammer sounding systems commonly employ lab-scale data to validate the models. In practice, however, the response signal patterns can be far more complicated due to varying geometric shapes and materials of structures. To deal with a large variety of unseen data, we propose a sequential treatment for response characterization. More specifically, the proposed system can adaptively update itself to approach human performance in hammering sounding data interpretation. To this end, a two-stage framework has been introduced, including feature extraction and the model updating scheme. Various state-of-the-art online learning algorithms have been reviewed and evaluated for the task. To conduct experimental validation, we collected 10,940 response instances from multiple inspection sites; each sample was annotated by human experts with healthy/defective condition labels. The results demonstrated that the proposed scheme achieved favorable assessment accuracy with high efficiency and low computation load.

Drone-based displacement measurement of infrastructures utilizing phase information.

Drone-based inspections provide an efficient and flexible approach to assessing aging infrastructures while prioritizing safety. Here, we present a pioneering framework that employs drone cameras for high-precision displacement measurement and achieves sub-millimeter accuracy, meeting the requirements for on-site inspections. Inspired by the principles of human auditory equilibrium, we have developed an effective scheme using a group of strategical reference markers on the bridge girders to measure structural displacements in the bridge. Our approach integrates the phase-based sampling moiré technique with four degrees-of-freedom geometric modeling to accurately delineate the desired bridge displacements from camera motion-induced displacements. The proposed scheme demonstrates favorable precision with accuracy reaching up to 1/100th of a pixel. Real-world validations further confirmed the reliability and efficiency of this technique, making it a practical tool for bridge displacement measurement. Beyond its current applications, this methodology holds promise as a foundational element in shaping the landscape of future autonomous infrastructure inspection systems.

CTG-Net: Cross-task guided network for breast ultrasound diagnosis.

Deep learning techniques have achieved remarkable success in lesion segmentation and classification between benign and malignant tumors in breast ultrasound images. However, existing studies are predominantly focused on devising efficient neural network-based learning structures to tackle specific tasks individually. By contrast, in clinical practice, sonographers perform segmentation and classification as a whole; they investigate the border contours of the tissue while detecting abnormal masses and performing diagnostic analysis. Performing multiple cognitive tasks simultaneously in this manner facilitates exploitation of the commonalities and differences between tasks. Inspired by this unified recognition process, this study proposes a novel learning scheme, called the cross-task guided network (CTG-Net), for efficient ultrasound breast image understanding. CTG-Net integrates the two most significant tasks in computerized breast lesion pattern investigation: lesion segmentation and tumor classification. Further, it enables the learning of efficient feature representations across tasks from ultrasound images and the task-specific discriminative features that can greatly facilitate lesion detection. This is achieved using task-specific attention models to share the prediction results between tasks. Then, following the guidance of task-specific attention soft masks, the joint feature responses are efficiently calibrated through iterative model training. Finally, a simple feature fusion scheme is used to aggregate the attention-guided features for efficient ultrasound pattern analysis. We performed extensive experimental comparisons on multiple ultrasound datasets. Compared to state-of-the-art multi-task learning approaches, the proposed approach can improve the Dice's coefficient, true-positive rate of segmentation, AUC, and sensitivity of classification by 11%, 17%, 2%, and 6%, respectively. The results demonstrate that the proposed cross-task guided feature learning framework can effectively fuse the complementary information of ultrasound image segmentation and classification tasks to achieve accurate tumor localization. Thus, it can aid sonographers to detect and diagnose breast cancer.

Point defect detection and strain mapping in Si single crystal by two-dimensional multiplication moiré method.

Although defect detection is critical for evaluating the manufacturing processes of semiconductor materials and metals, the detection of crystal defects, especially point defects, over a large field of view still faces considerable challenges. Herein, we report on the development of a two-dimensional (2D) multiplication moiré method using digital image processing to simultaneously detect point and line defects in a wide field of view. Defect locations were automatically detected by employing the concept of a hybrid strain, that is, the absolute value of the product of the strain distributions in different principal directions. To demonstrate a typical application of the proposed method, the hybrid strain distribution in a Si single crystal was measured, and point defects were successfully detected by transmission electron microscopy. The effectiveness of the proposed method was experimentally verified based on the enlarged views of atomic structures at several detected defect locations. This method is capable of visualizing defects by magnifying the lattice distortion in situ, which is a good solution to the problem faced by traditional methods in detecting point defects. This study paves the way for the detection of vacancies, interstitial atoms, substitutional atoms, dislocations, slips, and interfaces in various crystal structures and 2D materials.

Potential for improving the local realization of coordinated universal time with a convolutional neural network.

The time difference between coordinated universal time (UTC) and a hydrogen maser, which is a master oscillator for the local realization of UTC at the National Metrology Institute of Japan (NMIJ), has been predicted by using one of the deep learning techniques called a one-dimensional convolutional neural network (1D-CNN). Regarding the prediction result obtained by the 1D-CNN, we have observed improvement in the accuracy of prediction compared with that obtained by the Kalman filter. Although more investigations are required to conclude that the 1D-CNN can work as a good predictor, the present results suggest that the computational approach based on the deep learning technique may become a versatile method for improving the synchronous accuracy of UTC(NMIJ) relative to UTC.