Task model-specific operator skill assessment in routine fetal ultrasound scanning.

PURPOSE: For highly operator-dependent ultrasound scanning, skill assessment approaches evaluate operator competence given available data, such as acquired images and tracked probe movement. Operator skill level can be quantified by the completeness, speed, and precision of performing a clinical task, such as biometry. Such clinical tasks are increasingly becoming assisted or even replaced by automated machine learning models. In addition to measurement, operators need to be competent at the upstream task of acquiring images of sufficient quality. To provide computer assistance for this task requires a new definition of skill. METHODS: This paper focuses on the task of selecting ultrasound frames for biometry, for which operator skill is assessed by quantifying how well the tasks are performed with neural network-based frame classifiers. We first develop a frame classification model for each biometry task, using a novel label-efficient training strategy. Once these task models are trained, we propose a second task model-specific network to predict two skill assessment scores, based on the probability of identifying positive frames and accuracy of model classification. RESULTS: We present comprehensive results to demonstrate the efficacy of both the frame-classification and skill-assessment networks, using clinically acquired data from two biometry tasks for a total of 139 subjects, and compare the proposed skill assessment with metrics of operator experience. CONCLUSION: Task model-specific skill assessment is feasible and can be predicted by the proposed neural networks, which provide objective assessment that is a stronger indicator of task model performance, compared to existing skill assessment methods.

Safety Indices of Ultrasound: Adherence to Recommendations and Awareness During Routine Obstetric Ultrasound Scanning.

PURPOSE: To analyze bioeffect safety indices and assess how often operators look at these indices during routine obstetric ultrasound. MATERIALS AND METHODS: Automated analysis of prospectively collected data including video recordings of full-length ultrasound scans coupled with operator eye tracking was performed. Using optical recognition, we extracted the Mechanical Index (MI), Thermal Index in soft tissue (TIs), and Thermal Index in bone (TIb) values and ultrasound mode. This allowed us to report the bioeffect safety indices during routine obstetric scans and assess adherence to professional organization recommendations. Eye-tracking analysis allowed us to assess how often operators look at the displayed bioeffect safety indices. RESULTS: A total of 637 ultrasound scans performed by 17 operators were included, of which 178, 216, and 243 scans were first, second, and third-trimester scans, respectively. During live scanning, the mean and range were 0.14 (0.1 to 3.0) for TIb, 0.2 (0.1 to 1.2) for TIs, and 0.9 (0.1 to 1.3) for MI. The mean and standard deviation of TIb were 0.15 ±â€Š0.03, 0.23 ±â€Š0.09, 0.32 ±â€Š0.24 in the first, second, and third trimester, respectively. For B-mode, the highest TIb was 0.8 in all trimesters. The highest TIb was recorded for pulsed-wave Doppler mode in all trimesters. The recommended exposure times were maintained in all scans. Analysis of eye tracking suggested that operators looked at bioeffect safety indices in only 27 (4.2 %) of the scans. CONCLUSION: In this study, recommended bioeffect indices were adhered to in all routine scans. However, eye tracking showed that operators rarely assessed safety indices during scanning.

Weakly-supervised convolutional neural networks for multimodal image registration.

One of the fundamental challenges in supervised learning for multimodal image registration is the lack of ground-truth for voxel-level spatial correspondence. This work describes a method to infer voxel-level transformation from higher-level correspondence information contained in anatomical labels. We argue that such labels are more reliable and practical to obtain for reference sets of image pairs than voxel-level correspondence. Typical anatomical labels of interest may include solid organs, vessels, ducts, structure boundaries and other subject-specific ad hoc landmarks. The proposed end-to-end convolutional neural network approach aims to predict displacement fields to align multiple labelled corresponding structures for individual image pairs during the training, while only unlabelled image pairs are used as the network input for inference. We highlight the versatility of the proposed strategy, for training, utilising diverse types of anatomical labels, which need not to be identifiable over all training image pairs. At inference, the resulting 3D deformable image registration algorithm runs in real-time and is fully-automated without requiring any anatomical labels or initialisation. Several network architecture variants are compared for registering T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients. A median target registration error of 3.6 mm on landmark centroids and a median Dice of 0.87 on prostate glands are achieved from cross-validation experiments, in which 108 pairs of multimodal images from 76 patients were tested with high-quality anatomical labels.

Automatic Lacunae Localization in Placental Ultrasound Images via Layer Aggregation.

Accurate localization of structural abnormalities is a precursor for image-based prenatal assessment of adverse conditions. For clinical screening and diagnosis of abnormally invasive placenta (AIP), a life-threatening obstetric condition, qualitative and quantitative analysis of ultrasonic patterns correlated to placental lesions such as placental lacunae (PL) is challenging and time-consuming to perform even for experienced sonographers. There is a need for automated placental lesion localization that does not rely on expensive human annotations such as detailed manual segmentation of anatomical structures. In this paper, we investigate PL localization in 2D placental ultrasound images. First, we demonstrate the effectiveness of generating confidence maps from weak dot annotations in localizing PL as an alternative to expensive manual segmentation. Then we propose a layer aggregation structure based on iterative deep aggregation (IDA) for PL localization. Models with this structure were evaluated with 10-fold cross-validations on an AIP database (containing 3,440 images with 9,618 labelled PL from 23 AIP and 11 non-AIP participants). Experimental results demonstrate that the model with the proposed structure yielded the highest mean average precision (mAP=35.7%), surpassing all other baseline models (32.6%, 32.2%, 29.7%). We argue that features from shallower stages can contribute to PL localization more effectively using the proposed structure. To our knowledge, this is the first successful application of machine learning to placental lesion analysis and has the potential to be adapted for other clinical scenarios in breast, liver, and prostate cancer imaging.

A computer-aided tracking and motion analysis with ultrasound (CAT & MAUS) system for the description of hip joint kinematics.

PURPOSE: Investigation of joint kinematics contributes to developing a better understanding of musculoskeletal conditions. However, the most commonly used optoelectronic motion analysis systems cannot determine the movements of underlying bone landmarks with high accuracy because of soft tissue artefacts. The aim of this paper was to present a computer-aided measurement system to track the underlying bone anatomy in a 3D global coordinate frame and describe hip joint kinematics of ten healthy volunteers during gait. METHODS: We have developed a measurement tool with an image-based computer-aided post-processing pipeline for automatic bone segmentation in ultrasound (US) images and a globally optimal 3D surface-to-surface registration method to quantify hip joint movements. The segmentation algorithm exploits US intensity profiles, including information about the integrated backscattering, acoustic shadows, and local phase features. A global optimization method is applied based on the traditional iterative closest point registration algorithm, which is robust to initialization. The International Society of Biomechanics recommended joint kinematics descriptor has been adapted to calculate the joint kinematics. RESULTS: The developed system prototype has been validated with a ball-joint femoral phantom and tested in vivo with 10 volunteers. The maximum Euclidean distance error of the automatic bone segmentation is less than 2 pixels (approximately 0.2 mm). The maximum absolute rotation angle error is less than [Formula: see text]. CONCLUSION: This computer-aided tracking and motion analysis with ultrasound (CAT & MAUS) system shows the feasibility of describing hip joint kinematics for clinical investigation and diagnosis using an image-based solution.

Heterogeneous Tissue Characterization Using Ultrasound: A Comparison of Fractal Analysis Backscatter Models on Liver Tumors.

Assessment of tumor tissue heterogeneity via ultrasound has recently been suggested as a method for predicting early response to treatment. The ultrasound backscattering characteristics can assist in better understanding the tumor texture by highlighting the local concentration and spatial arrangement of tissue scatterers. However, it is challenging to quantify the various tissue heterogeneities ranging from fine to coarse of the echo envelope peaks in tumor texture. Local parametric fractal features extracted via maximum likelihood estimation from five well-known statistical model families are evaluated for the purpose of ultrasound tissue characterization. The fractal dimension (self-similarity measure) was used to characterize the spatial distribution of scatterers, whereas the lacunarity (sparsity measure) was applied to determine scatterer number density. Performance was assessed based on 608 cross-sectional clinical ultrasound radiofrequency images of liver tumors (230 and 378 representing respondent and non-respondent cases, respectively). Cross-validation via leave-one-tumor-out and with different k-fold methodologies using a Bayesian classifier was employed for validation. The fractal properties of the backscattered echoes based on the Nakagami model (Nkg) and its extend four-parameter Nakagami-generalized inverse Gaussian (NIG) distribution achieved best results-with nearly similar performance-in characterizing liver tumor tissue. The accuracy, sensitivity and specificity of Nkg/NIG were 85.6%/86.3%, 94.0%/96.0% and 73.0%/71.0%, respectively. Other statistical models, such as the Rician, Rayleigh and K-distribution, were found to not be as effective in characterizing subtle changes in tissue texture as an indication of response to treatment. Employing the most relevant and practical statistical model could have potential consequences for the design of an early and effective clinical therapy.

Breast-lesion Segmentation Combining B-Mode and Elastography Ultrasound.

Breast ultrasound (BUS) imaging has become a crucial modality, especially for providing a complementary view when other modalities (i.e., mammography) are not conclusive in the task of assessing lesions. The specificity in cancer detection using BUS imaging is low. These false-positive findings often lead to an increase of unnecessary biopsies. In addition, increasing sensitivity is also challenging given that the presence of artifacts in the B-mode ultrasound (US) images can interfere with lesion detection. To deal with these problems and improve diagnosis accuracy, ultrasound elastography was introduced. This paper validates a novel lesion segmentation framework that takes intensity (B-mode) and strain information into account using a Markov Random Field (MRF) and a Maximum a Posteriori (MAP) approach, by applying it to clinical data. A total of 33 images from two different hospitals are used, composed of 14 cancerous and 19 benign lesions. Results show that combining both the B-mode and strain data in a unique framework improves segmentation results for cancerous lesions (Dice Similarity Coefficient of 0.49 using B-mode, while including strain data reaches 0.70), which are difficult images where the lesions appear with blurred and not well-defined boundaries.

Right ventricular strain by MR quantitatively identifies regional dysfunction in patients with arrhythmogenic right ventricular cardiomyopathy.

BACKGROUND: Analysis of regional wall motion of the right ventricle (RV) is primarily qualitative with large interobserver variation in clinical practice. Thus, the purpose of this study was to use feature tracking to analyze regional wall motion abnormalities in patients with arrhythmogenic right ventricular cardiomyopathy (ARVC). METHODS: We enrolled 110 subjects (39 overt ARVC [mutation+/phenotype+] (35.5%), 40 preclinical ARVC [mutation+/phenotype-] (36.3%), and 31 control subjects (28.2%)). Cine steady state free precession cardiac MR was performed with temporal resolution ≤40 ms in the horizontal long axis (HLA), axial, and short axis directions. Regional strain was analyzed using feature tracking software and reproducibility was assessed by means of intraclass correlation coefficient. Dunnett's test was used in univariate analysis for comparisons to control subjects; cumulative odds logistic regression was used for minimally and fully adjusted multivariate models. RESULTS: Strain was significantly impaired in overt ARVC compared with control subjects both globally (P < 0.01) and regionally (all segments of HLA view, P < 0.01). In the HLA view, regional reproducibility was excellent within (intraclass correlation coefficient [ICC] = 0.81) and moderate between (ICC = 0.62) observers. Using a threshold of -31% subtricuspid strain in the HLA view, the sensitivity and specificity for overt ARVC were 75.0% and 78.2%, respectively. In multivariable analysis involving all three groups, subtricuspid strain less than -31% (beta = 1.38; P = 0.014) and RV end diastolic volume index (beta = 0.06; P = 0.001) were significant predictors of disease presence. CONCLUSION: RV strain can be reproducibly assessed with MR feature tracking, and regional strain is abnormal in overt ARVC compared with control subjects.

Quantification of ultrasonic texture intra-heterogeneity via volumetric stochastic modeling for tissue characterization.

Intensity variations in image texture can provide powerful quantitative information about physical properties of biological tissue. However, tissue patterns can vary according to the utilized imaging system and are intrinsically correlated to the scale of analysis. In the case of ultrasound, the Nakagami distribution is a general model of the ultrasonic backscattering envelope under various scattering conditions and densities where it can be employed for characterizing image texture, but the subtle intra-heterogeneities within a given mass are difficult to capture via this model as it works at a single spatial scale. This paper proposes a locally adaptive 3D multi-resolution Nakagami-based fractal feature descriptor that extends Nakagami-based texture analysis to accommodate subtle speckle spatial frequency tissue intensity variability in volumetric scans. Local textural fractal descriptors - which are invariant to affine intensity changes - are extracted from volumetric patches at different spatial resolutions from voxel lattice-based generated shape and scale Nakagami parameters. Using ultrasound radio-frequency datasets we found that after applying an adaptive fractal decomposition label transfer approach on top of the generated Nakagami voxels, tissue characterization results were superior to the state of art. Experimental results on real 3D ultrasonic pre-clinical and clinical datasets suggest that describing tumor intra-heterogeneity via this descriptor may facilitate improved prediction of therapy response and disease characterization.

Quantification of the heterogeneity of prognostic cellular biomarkers in ewing sarcoma using automated image and random survival forest analysis.

Driven by genomic somatic variation, tumour tissues are typically heterogeneous, yet unbiased quantitative methods are rarely used to analyse heterogeneity at the protein level. Motivated by this problem, we developed automated image segmentation of images of multiple biomarkers in Ewing sarcoma to generate distributions of biomarkers between and within tumour cells. We further integrate high dimensional data with patient clinical outcomes utilising random survival forest (RSF) machine learning. Using material from cohorts of genetically diagnosed Ewing sarcoma with EWSR1 chromosomal translocations, confocal images of tissue microarrays were segmented with level sets and watershed algorithms. Each cell nucleus and cytoplasm were identified in relation to DAPI and CD99, respectively, and protein biomarkers (e.g. Ki67, pS6, Foxo3a, EGR1, MAPK) localised relative to nuclear and cytoplasmic regions of each cell in order to generate image feature distributions. The image distribution features were analysed with RSF in relation to known overall patient survival from three separate cohorts (185 informative cases). Variation in pre-analytical processing resulted in elimination of a high number of non-informative images that had poor DAPI localisation or biomarker preservation (67 cases, 36%). The distribution of image features for biomarkers in the remaining high quality material (118 cases, 104 features per case) were analysed by RSF with feature selection, and performance assessed using internal cross-validation, rather than a separate validation cohort. A prognostic classifier for Ewing sarcoma with low cross-validation error rates (0.36) was comprised of multiple features, including the Ki67 proliferative marker and a sub-population of cells with low cytoplasmic/nuclear ratio of CD99. Through elimination of bias, the evaluation of high-dimensionality biomarker distribution within cell populations of a tumour using random forest analysis in quality controlled tumour material could be achieved. Such an automated and integrated methodology has potential application in the identification of prognostic classifiers based on tumour cell heterogeneity.

An efficient block matching and spectral shift estimation algorithm with applications to ultrasound elastography.

An efficient block matching and spectral shift estimation algorithm for freehand quasi-static ultrasound elastography is described in this paper. The proposed method provides a balance between computational speed and robustness against displacement estimation error and bias; a fundamental aspect of elastography. The new algorithm was tested on an extensive set of simulated 1-D RF ultrasound signals, replicating various strain profiles. Additionally, real 2-D scans were conducted on an ultrasound phantom with prescribed elastic properties; the algorithm output was further validated with a comparison to a finite element model (FEM) of the phantom. Clinical data from a breast cancer study and histology slides were used to demonstrate the in vivo use of the new elastography technique. The algorithm showed a significant computational savings (at least 60 times faster) over existing spectral shift analysis methods. Accurate strain images were produced in as little as 2 s with the scope for further speed enhancements through parallel processing; making real-time implementation a future possibility. Moreover, it demonstrated a robustness toward displacement estimation error when compared with conventional gradient-based techniques, and was able to perform at high strain values (>5%) while showing relative insensitivity to various parameters settings, such as sample rate and block window size; a desirable performance for a practical clinical tool.

Modeling of errors in Nakagami imaging: illustration on breast mass characterization.

Nakagami imaging is an attractive tissue characterization method, as the parameter estimated at each location is related to properties of the tissues. The application to clinical ultrasound images is problematic, as the estimation of the parameters is disturbed by the presence of complex structures. We propose to consider separately the different aspects potentially affecting the value of the Nakagami parameters and quantify their effects on the estimation. This framework is applied to the classification of breast masses. Quantitative parameters are computed on two groups of ultrasound images of benign and malignant tumors. A statistical analysis of the result indicated that the previously observed difference between average values of the Nakagami parameters is explained mostly by estimation errors. In the future, new methods for reliable computation of Nakagami parameters need to be developed, and factors of error should be considered in studies using Nakagami parameters.

Evaluating lesion segmentation on breast sonography as related to lesion type.

Breast sonography currently provides a complementary diagnosis when other modalities are not conclusive. However, lesion segmentation on sonography is still a challenging problem due to the presence of artifacts. To solve these problems, Markov random fields and maximum a posteriori-based methods are used to estimate a distortion field while identifying regions of similar intensity inhomogeneity. In this study, different initialization approaches were exhaustively evaluated using a database of 212 B-mode breast sonograms and considering the lesion types. Finally, conclusions about the relationship between the segmentation results and lesions types are described.

Slip imaging: reducing ambiguity in breast lesion assessment.

Ultrasound elasticity imaging (elastography) is gaining popularity as an adjunct to B-mode ultrasound for breast cancer diagnosis. Cancerous masses are usually stiffer than normal tissue, hence, using elasticity imaging should lead to better differentiation between benign and malignant masses than using B-mode alone. Clinicians assess the mobility of masses on palpation; cancers usually being less mobile. We introduce a method to estimate mobility, called slip imaging and combine it with conventional B-mode and elasticity data. In the reported evaluation on 70 women recalled to a breast assessment clinic, images were scored by three breast radiologists independently. Diagnostic accuracy increased from 75.7% with B-mode alone, to 78.1% when including elasticity imaging, to 80.0% when further including slip imaging. Specificity increased (74.6%:75.4%:82.5% respectively), with an apparent trade-off in sensitivity (77.1%:81.3%:77.1%). We conclude that Slip imaging is potentially a useful adjunct to B-mode and elasticity imaging and should undergo further research and development.

Assisted-freehand ultrasound elasticity imaging.

Good-quality elasticity imaging requires highly controlled compressions of the breast, which are often challenging to obtain with freehand, even by an experienced radiologist. This paper presents assisted-freehand ultrasound (AFUSON): a fusion of freehand and automated ultrasound systems designed to assisted elasticity imaging acquisition while remaining as flexible as freehand. In the form of a hand-held device, this semi-automatic solution delivers both increased acquisition precision and control. Compared with freehand acquisitions, it reduces out-of-plane motion decorrelation by one-half and lateral motion by one-third, increases within-scan repeatability by 50%, and does so across operators.

Elasticity reconstruction from displacement and confidence measures of a multi-compressed ultrasound RF sequence.

Ultrasound elasticity imaging shows promise as a new way for early detection of cancers by assessing the elastic characteristics of soft tissue. So far the commonly used approach involves solving the so-called inverse elasticity problem of recovering elastic parameters from displacement measurements. We propose a finite-elementbased nonlinear scheme to estimate the elasticity distribution of soft tissue from multi-compressed ultrasound radio frequency (RF) data. An experimental ultrasound workstation has been developed to acquire multi-compressed data. A composite probe was employed as the compression plate. The contact forces and torques were acquired at the same time as imaging. Axial displacements under different static loads are estimated from the RF data before and after deformation using a cross-correlation technique. The confidence of displacement estimates is employed as a weighting factor in solving the objective function describing the inverse elasticity reconstruction problem. A novel splitand- merge strategy is employed over the image sequence in which strain images are used to provide a priori knowledge of the relative stiffness distribution of the tissue to constrain the inverse problem solution. The experimental study has allowed us to investigate the performance of our approach in the controlled environment of simulated and phantom data. For a simulated single inclusion model with 5% axial displacement estimation error, the L2-error between the target and the reconstructed Young's modulus was found to be about 1%. In vivo validation of the proposed method has been carried out and some preliminary results are presented.

Improving the contrast of breast cancer masses in ultrasound using an autoregressive model based filter.

The assessment and diagnosis of breast cancer with ultrasound is a challenging problem due to the low contrast between cancer masses and benign tissue. Due to this low contrast it has proven to be difficult to achieve reliable segmentation results on breast cancer masses. An autoregressive model has been employed to filter out of the backscattered RF-signal from a tissue harmonic image which is not degraded by harmonic leakage. Measurements on the filtered image have shown a significant (up to 45%) increase in contrast between cancer masses and benign tissue.

A novel ultrasound indentation system for measuring biomechanical properties of in vivo soft tissue.

Technologies for soft tissue analysis are advancing at a rapid place. For instance, elastography, which provides soft tissue strain images, is starting to be tried in clinical practice as a tool for diagnosing cancer. Soft tissue deformation modeling and analysis is also an active area of research that has application in surgery planning and treatment. Typically, quantitative soft tissue analysis uses nominal values of soft tissue biomechanical properties. However, in practice, soft tissue properties can vary significantly between individuals. Hence, for soft tissue methodologies to reach their full potential as patient-specific techniques, there is a need to develop ways to efficiently measure soft tissue mechanical properties in vivo. This paper describes a prototype real-time ultrasound (US) indentation test system developed to meet this need. The system is based on the integration of a force sensor and an optical tracking system with a commercial US machine integrated with a suite of analysis methodologies. In a study on a single-layer phantom, we used the system to compare various methods of estimating linear elastic properties (via a theoretical approximation, 2-D finite element analysis, 3-D finite element analysis and a standard material-testing method). In a second study on a three-layer gelatin phantom, we describe a new finite-element-based inverse solution for recovering the Young's moduli of each layer to show how the system can estimate properties of internal components of soft tissue. Finally, we show how the system can be used to derive a modified quasilinear viscoelastic (QVL) model on real breast tissue.

Velocity estimation in ultrasound images: a block matching approach.

In this paper, we focus on velocity estimation in ultrasound images sequences. Ultrasound images present many difficulties in image processing because of the typically high level of noise found in them. Recently, Cohen and Dinstein have derived a new similarity measure, according to a simplified image formation model of ultrasound images, optimal in the maximum likelihood sense. This similarity measure is better for ultrasound images than others such as the sum-of-square differences or normalised cross-correlation because it takes into account the fact that the noise in an ultrasound image is multiplicative Rayleigh noise, and that displayed ultrasound images are log-compressed. In this work we investigate the use of this similarity measure in a block matching method. The underlying framework of the method is Singh's algorithm. New improvements are made both on the similarity measure and the Singh algorithm to provide better velocity estimates. A global optimisation scheme for algorithm parameter estimation is also proposed. We show that this optimisation makes an improvement of approximately 35% in comparison to the result obtained with the worst parameter set. Results on clinically acquired cardiac and breast ultrasound sequences, demonstrate the robustness of the method.

Nonrigid registration of 3-D free-hand ultrasound images of the breast.

Three-dimensional (3-D) ultrasound imaging of the breast enables better assessment of diseases than conventional two-dimensional (2-D) imaging. Free-hand techniques are often used for generating 3-D data from a sequence of 2-D slice images. However, the breast deforms substantially during scanning because it is composed primarily of soft tissue. This often causes tissue mis-registration in spatial compounding of multiple scan sweeps. To overcome this problem, in this paper, instead of introducing additional constraints on scanning conditions, we use image processing techniques. We present a fully automatic algorithm for 3-D nonlinear registration of free-hand ultrasound data. It uses a block matching scheme and local statistics to estimate local tissue deformation. A Bayesian regularization method is applied to the sample displacement field. The final deformation field is obtained by fitting a B-spline approximating mesh to the sample displacement field. Registration accuracy is evaluated using phantom data and similar registration errors are achieved with (0.19 mm) and without (0.16 mm) gaps in the data. Experimental results show that registration is crucial in spatial compounding of different sweeps. The execution time of the method on moderate hardware is sufficiently fast for fairly large research studies.