Real-Time Motion Analysis With 4D Deep Learning for Ultrasound-Guided Radiotherapy.

Motion compensation in radiation therapy is a challenging scenario that requires estimating and forecasting motion of tissue structures to deliver the target dose. Ultrasound offers direct imaging of tissue in real-time and is considered for image guidance in radiation therapy. Recently, fast volumetric ultrasound has gained traction, but motion analysis with such high-dimensional data remains difficult. While deep learning could bring many advantages, such as fast data processing and high performance, it remains unclear how to process sequences of hundreds of image volumes efficiently and effectively. We present a 4D deep learning approach for real-time motion estimation and forecasting using long-term 4D ultrasound data. Using motion traces acquired during radiation therapy combined with various tissue types, our results demonstrate that long-term motion estimation can be performed markerless with a tracking error of 0.35±0.2 mm and with an inference time of less than 5 ms. Also, we demonstrate forecasting directly from the image data up to 900 ms into the future. Overall, our findings highlight that 4D deep learning is a promising approach for motion analysis during radiotherapy.

Systematic analysis of volumetric ultrasound parameters for markerless 4D motion tracking.

OBJECTIVES: Motion compensation is an interesting approach to improve treatments of moving structures. For example, target motion can substantially affect dose delivery in radiation therapy, where methods to detect and mitigate the motion are widely used. Recent advances in fast, volumetric ultrasound have rekindled the interest in ultrasound for motion tracking. We present a setup to evaluate ultrasound based motion tracking and we study the effect of imaging rate and motion artifacts on its performance. METHODS: We describe an experimental setup to acquire markerless 4D ultrasound data with precise ground truth from a robot and evaluate different real-world trajectories and system settings toward accurate motion estimation. We analyze motion artifacts in continuously acquired data by comparing to data recorded in a step-and-shoot fashion. Furthermore, we investigate the trade-off between the imaging frequency and resolution. RESULTS: The mean tracking errors show that continuously acquired data leads to similar results as data acquired in a step-and-shoot fashion. We report mean tracking errors up to 2.01 mm and 1.36 mm on the continuous data for the lower and higher resolution, respectively, while step-and-shoot data leads to mean tracking errors of 2.52 mm and 0.98 mm. CONCLUSIONS: We perform a quantitative analysis of different system settings for motion tracking with 4D ultrasound. We can show that precise tracking is feasible and additional motion in continuously acquired data does not impair the tracking. Moreover, the analysis of the frequency resolution trade-off shows that a high imaging resolution is beneficial in ultrasound tracking.

Ultrasound Shear Wave Elasticity Imaging With Spatio-Temporal Deep Learning.

Ultrasound shear wave elasticity imaging is a valuable tool for quantifying the elastic properties of tissue. Typically, the shear wave velocity is derived and mapped to an elasticity value, which neglects information such as the shape of the propagating shear wave or push sequence characteristics. We present 3D spatio-temporal CNNs for fast local elasticity estimation from ultrasound data. This approach is based on retrieving elastic properties from shear wave propagation within small local regions. A large training data set is acquired with a robot from homogeneous gelatin phantoms ranging from 17.42 kPa to 126.05 kPa with various push locations. The results show that our approach can estimate elastic properties on a pixelwise basis with a mean absolute error of 5.01(437) kPa. Furthermore, we estimate local elasticity independent of the push location and can even perform accurate estimates inside the push region. For phantoms with embedded inclusions, we report a 53.93% lower MAE (7.50 kPa) and on the background of 85.24% (1.64 kPa) compared to a conventional shear wave method. Overall, our method offers fast local estimations of elastic properties with small spatio-temporal window sizes.

Rupture Detection During Needle Insertion Using Complex OCT Data and CNNs.

OBJECTIVE: Soft tissue deformation and ruptures complicate needle placement. However, ruptures at tissue interfaces also contain information which helps physicians to navigate through different layers. This navigation task can be challenging, whenever ultrasound (US) image guidance is hard to align and externally sensed forces are superimposed by friction. METHODS: We propose an experimental setup for reproducible needle insertions, applying optical coherence tomography (OCT) directly at the needle tip as well as external US and force measurements. Processing the complex OCT data is challenging as the penetration depth is limited and the data can be difficult to interpret. Using a machine learning approach, we show that ruptures can be detected in the complex OCT data without additional external guidance or measurements after training with multi-modal ground-truth from US and force. RESULTS: We can detect ruptures with accuracies of 0.94 and 0.91 on homogeneous and inhomogeneous phantoms, respectively, and 0.71 for ex-situ tissues. CONCLUSION: We propose an experimental setup and deep learning based rupture detection for the complex OCT data in front of the needle tip, even in deeper tissue structures without the need for US or force sensor guiding. SIGNIFICANCE: This study promises a suitable approach to complement a robust robotic needle placement.