Artificial Intelligence-Assisted Ultrasound Diagnosis on Infant Developmental Dysplasia of the Hip Under Constrained Computational Resources.

OBJECTIVES: Ultrasound (US) is important for diagnosing infant developmental dysplasia of the hip (DDH). However, the accuracy of the diagnosis depends heavily on expertise. We aimed to develop a novel automatic system (DDHnet) for accurate, fast, and robust diagnosis of DDH. METHODS: An automatic system, DDHnet, was proposed to diagnose DDH by analyzing static ultrasound images. A five-fold cross-validation experiment was conducted using a dataset containing 881 patients to verify the performance of DDHnet. In addition, a blind test was conducted on 209 patients (158 normal and 51 abnormal cases). The feasibility and performance of DDHnet were investigated by embedding it into ultrasound machines at low computational cost. RESULTS: DDHnet obtained reliable measurements and accurate diagnosis predictions. It reported an intra-class correlation coefficient (ICC) on α angle of 0.96 (95% CI: 0.93-0.97), ß angle of 0.97 (95% CI: 0.95-0.98), FHC of 0.98 (95% CI: 0.96-0.99) and PFD of 0.94 (95% CI: 0.90-0.96) in abnormal cases. DDHnet achieved a sensitivity of 90.56%, specificity of 100%, accuracy of 98.64%, positive predictive value (PPV) of 100%, and negative predictive value (NPV) of 98.44% for the diagnosis of DDH. For the measurement task on the US device, DDHnet took only 1.1 seconds to operate and complete, whereas the experienced senior expert required an average 41.4 seconds. CONCLUSIONS: The proposed DDHnet demonstrate state-of-the-art performance for all four indicators of DDH diagnosis. Fast and highly accurate DDH diagnosis is achievable through DDHnet, and is accessible under constrained computational resources.

Searching collaborative agents for multi-plane localization in 3D ultrasound.

3D ultrasound (US) has become prevalent due to its rich spatial and diagnostic information not contained in 2D US. Moreover, 3D US can contain multiple standard planes (SPs) in one shot. Thus, automatically localizing SPs in 3D US has the potential to improve user-independence and scanning-efficiency. However, manual SP localization in 3D US is challenging because of the low image quality, huge search space and large anatomical variability. In this work, we propose a novel multi-agent reinforcement learning (MARL) framework to simultaneously localize multiple SPs in 3D US. Our contribution is four-fold. First, our proposed method is general and it can accurately localize multiple SPs in different challenging US datasets. Second, we equip the MARL system with a recurrent neural network (RNN) based collaborative module, which can strengthen the communication among agents and learn the spatial relationship among planes effectively. Third, we explore to adopt the neural architecture search (NAS) to automatically design the network architecture of both the agents and the collaborative module. Last, we believe we are the first to realize automatic SP localization in pelvic US volumes, and note that our approach can handle both normal and abnormal uterus cases. Extensively validated on two challenging datasets of the uterus and fetal brain, our proposed method achieves the average localization accuracy of 7.03∘/1.59mm and 9.75∘/1.19mm. Experimental results show that our light-weight MARL model has higher accuracy than state-of-the-art methods.

Learn Fine-Grained Adaptive Loss for Multiple Anatomical Landmark Detection in Medical Images.

Automatic and accurate detection of anatomical landmarks is an essential operation in medical image analysis with a multitude of applications. Recent deep learning methods have improved results by directly encoding the appearance of the captured anatomy with the likelihood maps (i.e., heatmaps). However, most current solutions overlook another essence of heatmap regression, the objective metric for regressing target heatmaps and rely on hand-crafted heuristics to set the target precision, thus being usually cumbersome and task-specific. In this paper, we propose a novel learning-to-learn framework for landmark detection to optimize the neural network and the target precision simultaneously. The pivot of this work is to leverage the reinforcement learning (RL) framework to search objective metrics for regressing multiple heatmaps dynamically during the training process, thus avoiding setting problem-specific target precision. We also introduce an early-stop strategy for active termination of the RL agent's interaction that adapts the optimal precision for separate targets considering exploration-exploitation tradeoffs. This approach shows better stability in training and improved localization accuracy in inference. Extensive experimental results on two different applications of landmark localization: 1) our in-house prenatal ultrasound (US) dataset and 2) the publicly available dataset of cephalometric X-Ray landmark detection, demonstrate the effectiveness of our proposed method. Our proposed framework is general and shows the potential to improve the efficiency of anatomical landmark detection.

Agent With Warm Start and Adaptive Dynamic Termination for Plane Localization in 3D Ultrasound.

Accurate standard plane (SP) localization is the fundamental step for prenatal ultrasound (US) diagnosis. Typically, dozens of US SPs are collected to determine the clinical diagnosis. 2D US has to perform scanning for each SP, which is time-consuming and operator-dependent. While 3D US containing multiple SPs in one shot has the inherent advantages of less user-dependency and more efficiency. Automatically locating SP in 3D US is very challenging due to the huge search space and large fetal posture variations. Our previous study proposed a deep reinforcement learning (RL) framework with an alignment module and active termination to localize SPs in 3D US automatically. However, termination of agent search in RL is important and affects the practical deployment. In this study, we enhance our previous RL framework with a newly designed adaptive dynamic termination to enable an early stop for the agent searching, saving at most 67% inference time, thus boosting the accuracy and efficiency of the RL framework at the same time. Besides, we validate the effectiveness and generalizability of our algorithm extensively on our in-house multi-organ datasets containing 433 fetal brain volumes, 519 fetal abdomen volumes, and 683 uterus volumes. Our approach achieves localization error of 2.52mm/10.26° , 2.48mm/10.39° , 2.02mm/10.48° , 2.00mm/14.57° , 2.61mm/9.71° , 3.09mm/9.58° , 1.49mm/7.54° for the transcerebellar, transventricular, transthalamic planes in fetal brain, abdominal plane in fetal abdomen, and mid-sagittal, transverse and coronal planes in uterus, respectively. Experimental results show that our method is general and has the potential to improve the efficiency and standardization of US scanning.