ULTRA-SR Challenge: Assessment of Ultrasound Localization and TRacking Algorithms for Super-Resolution Imaging.

With the widespread interest and uptake of super-resolution ultrasound (SRUS) through localization and tracking of microbubbles, also known as ultrasound localization microscopy (ULM), many localization and tracking algorithms have been developed. ULM can image many centimeters into tissue in-vivo and track microvascular flow non-invasively with sub-diffraction resolution. In a significant community effort, we organized a challenge, Ultrasound Localization and TRacking Algorithms for Super-Resolution (ULTRA-SR). The aims of this paper are threefold: to describe the challenge organization, data generation, and winning algorithms; to present the metrics and methods for evaluating challenge entrants; and to report results and findings of the evaluation. Realistic ultrasound datasets containing microvascular flow for different clinical ultrasound frequencies were simulated, using vascular flow physics, acoustic field simulation and nonlinear bubble dynamics simulation. Based on these datasets, 38 submissions from 24 research groups were evaluated against ground truth using an evaluation framework with six metrics, three for localization and three for tracking. In-vivo mouse brain and human lymph node data were also provided, and performance assessed by an expert panel. Winning algorithms are described and discussed. The publicly available data with ground truth and the defined metrics for both localization and tracking present a valuable resource for researchers to benchmark algorithms and software, identify optimized methods/software for their data, and provide insight into the current limits of the field. In conclusion, Ultra-SR challenge has provided benchmarking data and tools as well as direct comparison and insights for a number of the state-of-the art localization and tracking algorithms.

Ultrasound localization microscopy.

Ultrasound Localization Microscopy (ULM) is an emerging technique that provides impressive super-resolved images of microvasculature, i.e., images with much better resolution than the conventional diffraction-limited ultrasound techniques and is already taking its first steps from preclinical to clinical applications. In comparison to the established perfusion or flow measurement methods, namely contrast-enhanced ultrasound (CEUS) and Doppler techniques, ULM allows imaging and flow measurements even down to the capillary level. As ULM can be realized as a post-processing method, conventional ultrasound systems can be used for. ULM relies on the localization of single microbubbles (MB) of commercial, clinically approved contrast agents. In general, these very small and strong scatterers with typical radii of 1-3 µm are imaged much larger in ultrasound images than they actually are due to the point spread function of the imaging system. However, by applying appropriate methods, these MBs can be localized with sub-pixel precision. Then, by tracking MBs over successive frames of image sequences, not only the morphology of vascular trees but also functional information such as flow velocities or directions can be obtained and visualized. In addition, quantitative parameters can be derived to describe pathological and physiological changes in the microvasculature. In this review, the general concept of ULM and conditions for its applicability to microvessel imaging are explained. Based on this, various aspects of the different processing steps for a concrete implementation are discussed. The trade-off between complete reconstruction of the microvasculature and the necessary measurement time as well as the implementation in 3D are reviewed in more detail, as they are the focus of current research. Through an overview of potential or already realized preclinical and clinical applications - pathologic angiogenesis or degeneration of vessels, physiological angiogenesis, or the general understanding of organ or tissue function - the great potential of ULM is demonstrated.

Ultrasound Imaging.

Ultrasound imaging plays an important role in oncological imaging for more than five decades now. It can be applied in all tissues that are not occluded by bone or gas-filled regions. The quality of ultrasound images benefitted strongly from improved electronics and increased computational power. To the morphological imaging, several functional imaging methods were added: Flow visualization became possible by Doppler techniques and as a recent addition the elastic properties of tissues can be imaged by elastographic methods with transient shear wave imaging. In the beginning of molecular imaging, ultrasound with its contrast based on mechanical tissue properties was an unlikely candidate to play a role. However, with contrast agents consisting of micrometer-sized gas bubbles, which can be imaged with high sensitivity, ligands addressing targets in the vascular wall could be used. Because even single bubbles can be detected, this led to various ultrasound molecular imaging techniques and the ongoing development of clinical molecular contrast media. In this chapter, the basic properties of ultrasonic imaging like its contrast mechanisms and spatiotemporal resolution are discussed. The image formation and its ongoing change from line-oriented scanning to full-volume reconstructions are explained. Then, the ultrasound contrast media and imaging techniques are introduced and emerging new methods like super-resolution vascular imaging demonstrate the ongoing development in this field.

Assessing Vessel Reconstruction in Ultrasound Localization Microscopy by Maximum Likelihood Estimation of a Zero-Inflated Poisson Model.

In clinical applications of super-resolution ultrasound imaging, it is often not possible to achieve a full reconstruction of the microvasculature within a limited measurement time. This makes the comparison of studies and quantitative parameters of vascular morphology and perfusion difficult. Therefore, saturation models were proposed to predict adequate measurement times and estimate the degree of vessel reconstruction. Here, we derive a statistical model for the microbubble counts in super-resolution voxels by a zero-inflated Poisson (ZIP) process. In this model, voxels either belong to vessels with probability Pv and count events with Poisson rate Λ , or they are empty and remain zero. In this model, Pv represents the vessel voxel density in the super-resolution image after infinite measurement time. For the parameters Pv and Λ , we give Cramér-Rao lower bounds (CRLBs) for the estimation variance and derive maximum likelihood estimators (MLEs) in a novel closed-form solution. These can be calculated with knowledge of only the counts at the end of the acquisition time. The estimators are applied to preclinical and clinical data and the MLE outperforms alternative estimators proposed before. The estimated degree of reconstruction lies between 38% and 74% after less than 90 s. Vessel probability Pv ranged from 4% to 20%. The rate parameter Λ was estimated in the range of 0.5-1.3 microbubbles/voxel. For these parameter ranges, the CRLB gives standard deviations of less than 2%, which supports that the parameters can be estimated with good precision already for limited acquisition times.

Sonographic visibility of cannulas using convex ultrasound transducers.

The key for safe ultrasound (US)-guided punctures is a good visibility of the cannula. When using convex transducers for deep punctures, the incident angle between US beam and cannula varies along the cannula leading to a complex visibility pattern. Here, we present a method to systematically investigate the visibility throughout the US image. For this, different objective criteria were defined and applied to measurement series with varying puncture angles and depths of the cannula. It is shown that the visibility not only depends on the puncture angle but also on the location of the cannula in the US image when using convex transducers. In some image regions, an unexpected good visibility was observed even for steep puncture angles. The systematic evaluation of the cannula visibility is of fundamental interest to sensitise physicians to the handling of convex transducers and to evaluate new techniques for further improvement.

Clinical Pilot Application of Super-Resolution US Imaging in Breast Cancer.

Recently, we proved in the first measurements of breast carcinomas the feasibility of super-resolution ultrasound (US) imaging by motion-model ultrasound localization microscopy in a clinical setup. Nevertheless, pronounced in-plane and out-of-plane motions, a nonoptimized microbubble injection scheme, the lower frame rate and the larger slice thickness made the processing more complex than in preclinical investigations. Here, we compare the results of state-of-the-art contrast-enhanced to super-resolution US imaging and systematically analyze the measurements to get indications for the improvement of image acquisition and processing in the future clinical studies. In this regard, the application of a saturation model to the reconstructed vessels is shown to be a valuable tool not only to estimate the measurement times necessary to adequately reconstruct the microvasculature but also for the validation of the measurements. The parameters from this model can also serve to optimize contrast agent concentration and injection protocols. Finally, for the measurements of well-perfused tumors, we observed between 28% and 50% filling for 90-s examination times.

Motion model ultrasound localization microscopy for preclinical and clinical multiparametric tumor characterization.

Super-resolution imaging methods promote tissue characterization beyond the spatial resolution limits of the devices and bridge the gap between histopathological analysis and non-invasive imaging. Here, we introduce motion model ultrasound localization microscopy (mULM) as an easily applicable and robust new tool to morphologically and functionally characterize fine vascular networks in tumors at super-resolution. In tumor-bearing mice and for the first time in patients, we demonstrate that within less than 1 min scan time mULM can be realized using conventional preclinical and clinical ultrasound devices. In this context, next to highly detailed images of tumor microvascularization and the reliable quantification of relative blood volume and perfusion, mULM provides multiple new functional and morphological parameters that discriminate tumors with different vascular phenotypes. Furthermore, our initial patient data indicate that mULM can be applied in a clinical ultrasound setting opening avenues for the multiparametric characterization of tumors and the assessment of therapy response.

Low-Energy Ultrasound Treatment Improves Regional Tumor Vessel Infarction by Retargeted Tissue Factor.

OBJECTIVES: To enhance the regional antitumor activity of the vascular-targeting agent truncated tissue factor (tTF)-NGR by combining the therapy with low-energy ultrasound (US) treatment. METHODS: For the in vitro US exposure of human umbilical vein endothelial cells (HUVECs), cells were put in the focus of a US transducer. For analysis of the US-induced phosphatidylserine (PS) surface concentration on HUVECs, flow cytometry was used. To demonstrate the differences in the procoagulatory efficacy of TF-derivative tTF-NGR on binding to HUVECs with a low versus high surface concentration of PS, we performed factor X activation assays. For low-energy US pretreatment, HT1080 fibrosarcoma xenotransplant-bearing nude mice were treated by tumor-regional US-mediated stimulation (ie, destruction) of microbubbles. The therapy cohorts received the tumor vessel-infarcting tTF-NGR protein with or without US pretreatment (5 minutes after US stimulation via intraperitoneal injection on 3 consecutive days). RESULTS: Combination therapy experiments with xenotransplant-bearing nude mice significantly increased the antitumor activity of tTF-NGR by regional low-energy US destruction of vascular microbubbles in tumor vessels shortly before application of tTF-NGR (P < .05). Mechanistic studies proved the upregulation of anionic PS on the outer leaflet of the lipid bilayer of endothelial cell membranes by low-energy US and a consecutive higher potential of these preapoptotic endothelial cells to activate coagulation via tTF-NGR and coagulation factor X as being a basis for this synergistic activity. CONCLUSIONS: Combining retargeted tTF to tumor vessels with proapoptotic stimuli for the tumor vascular endothelium increases the antitumor effects of tumor vascular infarction. Ultrasound treatment may thus be useful in this respect for regional tumor therapy.

Estimation of multipath transmission parameters for quantitative ultrasound measurements of bone.

When applying quantitative ultrasound (QUS) measurements to bone for predicting osteoporotic fracture risk, the multipath transmission of sound waves frequently occurs. In the last 10 years, the interest in separating multipath QUS signals for their analysis awoke, and led to the introduction of several approaches. Here, we compare the performances of the two fastest algorithms proposed for QUS measurements of bone: the modified least-squares Prony method (MLSP), and the space alternating generalized expectation maximization algorithm (SAGE) applied in the frequency domain. In both approaches, the parameters of the transfer functions of the sound propagation paths are estimated. To provide an objective measure, we also analytically derive the Cramér-Rao lower bound of variances for any estimator and arbitrary transmit signals. In comparison with results of Monte Carlo simulations, this measure is used to evaluate both approaches regarding their accuracy and precision. Additionally, with simulations using typical QUS measurement settings, we illustrate the limitations of separating two superimposed waves for varying parameters with focus on their temporal separation. It is shown that for good SNRs around 100 dB, MLSP yields better results when two waves are very close. Additionally, the parameters of the smaller wave are more reliably estimated. If the SNR decreases, the parameter estimation with MLSP becomes biased and inefficient. Then, the robustness to noise of the SAGE clearly prevails. Because a clear influence of the interrelation between the wavelength of the ultrasound signals and their temporal separation is observable on the results, these findings can be transferred to QUS measurements at other sites. The choice of the suitable algorithm thus depends on the measurement conditions.

A device for in vivo measurements of quantitative ultrasound variables at the human proximal femur.

Quantitative ultrasound (QUS) at the calcaneus has similar power as a bone mineral density (BMD)- measurement using DXA for the prediction of osteoporotic fracture risk. Ultrasound equipment is less expensive than DXA and free of ionizing radiation. As a mechanical wave, QUS has the potential of measuring different bone properties than dual X-ray absorptiometry (DXA,) which depends on X-ray attenuation and might be developed into a tool of comprehensive assessment of bone strength. However, site-specific DXA at the proximal femur shows best performance in the prediction of hip fractures. To combine the potential of QUS with measurements directly at the femur, we developed a device for in vivo QUS measurements at this site. Methods comprise ultrasound transmission through the bone, reflection from the bone surface, and backscatter from the inner trabecular structure. The complete area of the proximal femur can be scanned except at the femoral head, which interferes with the ilium. To avoid edge artifacts, a subregion of the proximal femur in the trochanteric region was selected as measurement region. First, in vivo measurements demonstrate a good signal to noise ratio and proper depiction of the proximal femur on an attenuation image. Our results demonstrate the feasibility of in vivo measurements. Further improvements can be expected by refinement of the scanning technique and data evaluation method to enhance the potential of the new method for the estimation of bone strength.

Model-based estimation of quantitative ultrasound variables at the proximal femur.

To improve the prediction of the osteoporotic fracture risk at the proximal femur we are developing a scanner for quantitative ultrasound (QUS) measurements at this site. Due to multipath transmission in this complex shaped bone, conventional signal processing techniques developed for QUS measurements at peripheral sites frequently fail. Therefore, we propose a model-based estimation of the QUS variables and analyze the performance of the new algorithm. Applying the proposed method to QUS scans of excised proximal femurs increased the fraction of evaluable signals from approx. 60% (using conventional algorithms) to 97%. The correlation of the standard QUS variables broadband ultrasound attenuation (BUA) and speed of sound (SOS) with the established variable bone mineral density (BMD) reported in previous studies is maintained (BUA/BMD: r(2) = 0.69; SOS/BMD: r(2) = 0.71; SOS+BUA/BMD: r(2) = 0.88). Additionally, different wave types could be clearly detected and characterized in the trochanteric region. The ability to separate superimposed signals with this approach opens up further diagnostic potential for evaluating waves of different sound paths and wave types through bone tissue.

In vivo measurements of ultrasound transmission through the human proximal femur.

Quantitative ultrasound (QUS) measurements can be used to estimate osteoporotic fracture risk. The commonly used variables are the speed of sound (SOS) and the frequency dependent sound attenuation (broadband ultrasound attenuation, [BUA]) of a wave propagating through the bone, preferably the calcaneus. The technology, so far, is less suitable for direct measurement in vivo at the spine or the femur for prediction of bone mineral density (BMD) or fracture risk at the main osteoporotic fracture sites. To improve the clinical performance of QUS, we built a device for direct QUS measurements at the human femur in vivo. In vivo images of ultrasound transmission at one of the main fracture sites, the proximal femur, could be acquired. The estimated precision of SOS measurements of 0.5% achieved at the femur is comparable with the precision of peripheral QUS devices.

Wavelet-based signal processing of in vitro ultrasonic measurements at the proximal femur.

To estimate osteoporotic fracture risk, several techniques for quantitative ultrasound (QUS) measurements at peripheral sites have been developed. As these techniques are limited in the prediction of fracture risk of the central skeleton, such as the hip, we are developing a QUS device for direct measurements at the femur. In doing so, we noticed the necessity to improve the conventional signal processing because it failed in a considerable number of measurements due to multipath transmission. Two sets of excised human femurs (n = 6 + 34) were scanned in transmission mode. Instead of using the conventional methods, the radio-frequency signals were processed with the continuous wavelet transform to detect their time-of-flights for the calculation of speed-of-sound (SOS) in bone. The SOS-values were averaged over a region similar to the total hip region of dual X-ray absorptiometry (DXA) measurements and compared with bone mineral density (BMD) measured with DXA. Testing six standard wavelets, this algorithm failed for only 0% to 6% of scan in test set 1 compared with 29% when using conventional algorithms. For test set 2, it failed for 2% to 12% compared with approximately 40%. SOS and BMD correlated significantly in both test sets (test set 1: r2 = 0.87 to 0.92, p < 0.007; test set 2: r2 = 0.68 to 0.79, p < 0.0001). The correlations are comparable with correlations recently reported. However, the number of evaluable signals could be substantially increased, which improves the perspectives of the in vivo measurements.