Transformer-Based Spatio-Temporal Analysis for Classification of Aortic Stenosis Severity From Echocardiography Cine Series.

Aortic stenosis (AS) is characterized by restricted motion and calcification of the aortic valve and is the deadliest valvular cardiac disease. Assessment of AS severity is typically done by expert cardiologists using Doppler measurements of valvular flow from echocardiography. However, this limits the assessment of AS to hospitals staffed with experts to provide comprehensive echocardiography service. As accurate Doppler acquisition requires significant clinical training, in this paper, we present a deep learning framework to determine the feasibility of AS detection and severity classification based only on two-dimensional echocardiographic data. We demonstrate that our proposed spatio-temporal architecture effectively and efficiently combines both anatomical features and motion of the aortic valve for AS severity classification. Our model can process cardiac echo cine series of varying length and can identify, without explicit supervision, the frames that are most informative towards the AS diagnosis. We present an empirical study on how the model learns phases of the heart cycle without any supervision and frame-level annotations. Our architecture outperforms state-of-the-art results on a private and a public dataset, achieving 95.2% and 91.5% in AS detection, and 78.1% and 83.8% in AS severity classification on the private and public datasets, respectively. Notably, due to the lack of a large public video dataset for AS, we made slight adjustments to our architecture for the public dataset. Furthermore, our method addresses common problems in training deep networks with clinical ultrasound data, such as a low signal-to-noise ratio and frequently uninformative frames. Our source code is available at: https://github.com/neda77aa/FTC.git.

Soft Tissue Characterization with Temporal Enhanced Ultrasound through Periodic Manipulation of Point Spread Function: A Feasibility Study.

Temporal enhanced ultrasound (TeUS) is a tissue characterization approach based on analysis of a temporal series of US data. Previously we demonstrated that intrinsic or external micro-motions of scatterers in the tissue contribute towards the tissue classification properties of TeUS. This property is beneficial to detect early stage cancer, for example, where changes in nuclei configuration (scatteres) dominate tissue properties. In this study, we propose an analytical derivation and experiments to acquire TeUS through manipulation of US imaging parameters, which may be simpler to translate to clinical applications. The feasibility of the proposed method is demonstrated on tissue-mimicking phantoms. Using an autoencoder classifier, we are able to classify phantoms of varying elasticities and scattering sizes.

An End-to-End Solution for Automatic Contouring of Tumor Region in Intraoperative Images of Breast Lumpectomy.

Breast-conserving surgery, also known as lumpectomy, is an early stage breast cancer treatment that aims to spare as much healthy breast tissue as possible. A risk associated with lumpectomy is the presence of cancer positive margins post operation. Surgical navigation has been shown to reduce cancer positive margins but requires manual segmentation of the tumor intraoperatively. In this paper, we propose an end-to-end solution for automatic contouring of breast tumor from intraoperative ultrasound images using two convolutional neural network architectures, the U-Net and residual U-Net. The networks are trained on annotated intraoperative breast ultrasound images and evaluated on the quality of predicted segmentations. This work brings us one step closer to providing surgeons with an automated surgical navigation system that helps reduce cancer-positive margins during lumpectomy.

Correction to "Automatic Quality Assessment of Echocardiograms Using Convolutional Neural Networks: Feasibility on the Apical Four-Chamber View".

In the above paper [1], the first footnote should have indicated the following information: A. H. Abdi and C. Luong are joint first authors.

Joint sparse representation of brain activity patterns in multi-task fMRI data.

A single-task functional magnetic resonance imaging (fMRI) experiment may only partially highlight alterations to functional brain networks affected by a particular disorder. Multivariate analysis across multiple fMRI tasks may increase the sensitivity of fMRI-based diagnosis. Prior research using multi-task analysis in fMRI, such as those that use joint independent component analysis (jICA), has mainly assumed that brain activity patterns evoked by different tasks are independent. This may not be valid in practice. Here, we use sparsity, which is a natural characteristic of fMRI data in the spatial domain, and propose a joint sparse representation analysis (jSRA) method to identify common information across different functional subtraction (contrast) images in data from a multi-task fMRI experiment. Sparse representation methods do not require independence, or that the brain activity patterns be nonoverlapping. We use functional subtraction images within the joint sparse representation analysis to generate joint activation sources and their corresponding sparse modulation profiles. We evaluate the use of sparse representation analysis to capture individual differences with simulated fMRI data and with experimental fMRI data. The experimental fMRI data was acquired from 16 young (age: 19-26) and 16 older (age: 57-73) adults obtained from multiple speech comprehension tasks within subjects, where an independent measure (namely, age in years) can be used to differentiate between groups. Simulation results show that this method yields greater sensitivity, precision, and higher Jaccard indexes (which measures similarity and diversity of the true and estimated brain activation sources) than does the jICA method. Moreover, superiority of the jSRA method in capturing individual differences was successfully demonstrated using experimental fMRI data.

Classification of individuals based on sparse representation of brain cognitive patterns: a functional MRI study.

Many neurological disorders can change patterns of brain activity observed in functional imaging studies. These functional differences may be useful for classification of individuals into diagnostic categories. However, due to the high dimensionality of the input feature space and small set of subjects that are usually available, classification based on fMRI data is not trivial. Here, we evaluate the use of a Sparse Representation Analysis method within a Fisher Linear Discriminant (FLD) classification method, taking functional patterns characteristic of different cognitive tasks as the data input. As a test dataset, with a clear 'gold-standard' classification, we attempt to classify individuals as young, or older, based only on functional activation patterns in a speech listening task. Thirty two young (age: 19-26) and older (age: 57-73) adults (16 each) were scanned while listening to noise and to sentences degraded with noise, half of which contained meaningful context that could be used to enhance intelligibility. Different functional contrast images were used within K-SVD to generate basis activation sources and their corresponding sparse modulation profiles. Sparse modulation profiles were used in a FLD framework to classify individuals into the young and older categories. The results demonstrate the feasibility of the general approach, and confirm the potential applicability of the proposed method for real-world diagnostic problems.

Is the link between anatomical structure and function equally strong at all cognitive levels of processing?

Whereas low-level sensory processes can be linked to macroanatomy with great confidence, the degree to which high-level cognitive processes map onto anatomy is less clear. If function respects anatomy, more accurate intersubject anatomical registration should result in better functional alignment. Here, we use auditory functional magnetic resonance imaging and compare the effectiveness of affine and nonlinear registration methods for aligning anatomy and functional activation across subjects. Anatomical alignment was measured using normalized cross-correlation within functionally defined regions of interest. Functional overlap was assessed using t-statistics from the group analyses and the degree to which group statistics predict high and consistent signal change in individual data sets. In regions related to early stages of auditory processing, nonlinear registration resulted in more accurate anatomical registration and stronger functional overlap among subjects compared with affine. In frontal and temporal areas reflecting high-level processing of linguistic meaning, nonlinear registration also improved the accuracy of anatomical registration. However, functional overlap across subjects was not enhanced in these regions. Therefore, functional organization, relative to anatomy, is more variable in the frontal and temporal areas supporting meaning-based processes than in areas devoted to sensory/perceptual auditory processing. This demonstrates for the first time that functional variability increases systematically between regions supporting lower and higher cognitive processes.

Technical note: unsupervised C-arm pose tracking with radiographic fiducial.

PURPOSE: C-arm fluoroscopy reconstruction, such as that used in prostate brachytherapy, requires that the relative poses of the individual C-arm fluoroscopy images must be known prior to reconstruction. Radiographic fiducials can provide excellent C-arm pose tracking, but they need to be segmented in the image. The authors report an automated and unsupervised method that does not require prior segmentation of the fiducial. METHODS: The authors compute the individual C-arm poses relative to a stationary radiographic fiducial of known geometry. The authors register a filtered 2D fluoroscopy image of the fiducial to its 3D model by using image intensity alone without prior segmentation. To enhance the C-arm images, the authors investigated a three-step cascade filter and a line enhancement filter. The authors tested the method on a composite fiducial containing beads, straight lines, and ellipses. Ground-truth C-arm pose was provided by a clinically proven method. RESULTS: Using 111 clinical C-arm images and +/- 10 degrees and +/- 10 mm random perturbation around the ground-truth pose, a total of 2775 cases were evaluated. The average rotation and translation errors were 0.62 degrees (STD = 0.31 degrees) and 0.72 mm (STD = 0.55 mm) for the three-step filter and 0.67 degrees (STD = 0.40 degrees) and 0.87 mm (STD = 0.27 mm) using the line enhancement filter. CONCLUSIONS: The C-arm pose tracking method was sufficiently accurate and robust on human patient data for subsequent 3D implant reconstruction.

Registration between ultrasound and fluoroscopy or CT in prostate brachytherapy.

PURPOSE: In prostate brachytherapy, transrectal ultrasound (TRUS) is used to visualize the anatomy, while implanted seeds can be visualized by fluoroscopy. Intraoperative dosimetry optimization is possible using a combination of TRUS and fluoroscopy, but requires localization of the fluoroscopy-derived seed cloud, relative to the anatomy as seen on TRUS. The authors propose to develop a method of registration of TRUS images and the implants reconstructed from fluoroscopy. METHODS: A phantom was implanted with 48 seeds then imaged with TRUS and CT. Seeds were reconstructed from CT yielding a cloud of seeds. Fiducial-based ground-truth registration was established between the TRUS and CT. TRUS images are filtered, compounded, and registered to the reconstructed implants by using an intensity-based metric. The authors evaluated a volume-to-volume and point-to-volume registration scheme. In total, seven TRUS filtering techniques and three image similarity metrics were analyzed. The method was also tested on human subject data captured from a brachytherapy procedure. RESULTS: For volume-to-volume registration, noise reduction filter and normalized correlation metrics yielded the best result: An average of 0.54 +/- 0.11 mm seed localization error relative to ground truth. For point-to-volume registration, noise reduction combined with beam profile filter and mean squares metrics yielded the best result: An average of 0.38 +/- 0.19 mm seed localization error relative to the ground truth. In human patient data, C-arm fluoroscopy images showed 81 radioactive seeds implanted inside the prostate. A qualitative analysis showed clinically correct agreement between the seeds visible in TRUS and reconstructed from intraoperative fluoroscopy imaging. The measured registration error compared to the manually selected seed locations by the clinician was 2.86 +/- 1.26 mm. CONCLUSIONS: Fully automated registration between TRUS and the reconstructed seeds performed well in ground-truth phantom experiments and qualitative observation showed adequate performance on early clinical patient data.

Intra-operative Localization of Brachytherapy Implants Using Intensity-based Registration.

In prostate brachytherapy, a transrectal ultrasound (TRUS) will show the prostate boundary but not all the implanted seeds, while fluoroscopy will show all the seeds clearly but not the boundary. We propose an intensity-based registration between TRUS images and the implant reconstructed from uoroscopy as a means of achieving accurate intra-operative dosimetry. The TRUS images are first filtered and compounded, and then registered to the uoroscopy model via mutual information. A training phantom was implanted with 48 seeds and imaged. Various ultrasound filtering techniques were analyzed, and the best results were achieved with the Bayesian combination of adaptive thresholding, phase congruency, and compensation for the non-uniform ultrasound beam profile in the elevation and lateral directions. The average registration error between corresponding seeds relative to the ground truth was 0.78 mm. The effect of false positives and false negatives in ultrasound were investigated by masking true seeds in the uoroscopy volume or adding false seeds. The registration error remained below 1.01 mm when the false positive rate was 31%, and 0.96 mm when the false negative rate was 31%. This fully automated method delivers excellent registration accuracy and robustness in phantom studies, and promises to demonstrate clinically adequate performance on human data as well. Keywords: Prostate brachytherapy, Ultrasound, Fluoroscopy, Registration.

A statistical atlas-based technique for automatic segmentation of the first Heschl's gyrus in human auditory cortex from MR images.

We present an automatic method for the segmentation of the first transverse temporal gyrus of Heschl (HG), the morphological marker for primary auditory cortex in humans. The proposed technique utilizes a statistical anatomical atlas of the gyrus, generated from a set of training samples using principal component analysis. The training set consists of MRI data from 12 subjects with the corresponding Heschl's gyri manually labeled in each hemisphere (separate atlases were generated for each hemisphere). We used a leave-one-out approach to automatically segment Heschl's gyri in both hemispheres from the MR image data using generated atlases. We assessed the accuracy of this atlas-based technique by using it to segment the HG region from several test cases and finding the overlap between the segmented and labeled HG regions. Results demonstrated more than 75% and 83% accuracy in the extraction of the HG volumes in the left and right hemispheres, respectively. It is expected that the proposed tool can be adapted to extract other anatomical regions in the brain.

INTENSITY-BASED REGISTRATION OF PROSTATE BRACHYTHERAPY IMPLANTS AND ULTRASOUND.

PURPOSE: In prostate brachytherapy, determining the 3D location of the seeds relative to surrounding structures is necessary for calculating dosimetry. Ultrasound imaging provides the ability to visualize soft tissues, and implanted seeds can be reconstructed from C-arm fluoroscopy. Registration between these two complementary modalities would allow us to make immediate provisions for dosimetric deviation from the optimal implant plan. METHODS: We propose intensity-based registration between ultrasound and a reconstructed model of seeds from fluoroscopy. The ultrasound images are pre-processed with recursive thresholding and phase congruency. Then a 3D ultrasound volume is reconstructed and registered to the implant model using mutual information. RESULTS: A standard training phantom was implanted with 49 seeds. Average registration error between corresponding seeds relative to the ground truth is 0.09 mm. The effect of false positives in ultrasound was investigated by masking seeds from the fluoroscopy reconstructed model. The registration error remained below 0.5 mm at a rate of 30% false positives. CONCLUSION: Our method promises to be clinically adequate, where requirements for registration is 1.5 mm.

Validation of a new surgical procedure for percutaneous scaphoid fixation using intra-operative ultrasound.

A new technique for percutaneous fixation of non-displaced scaphoid fractures is described. The technique used pre-operative planning from computed tomography images, registration to intra-operatively acquired three-dimensional ultrasound images, and intra-operative guidance using an optical tracking system. Two stand-alone software applications were developed. The first one was used to determine the surgical plan pre-operatively and the second one was used to guide the surgeon during screw insertion. Laboratory validation of the technique included measurements of the inter-operator and intra-operator variability in the outcome of scaphoid fixation using the proposed procedure, and also included comparison of the performance of this procedure with the conventional percutaneous fixation technique using fluoroscopy. The results showed that the tight accuracy requirements of percutaneous scaphoid fixation were met and that the consistency was superior to the conventional technique.

A navigation system for shoulder arthroscopic surgery.

The general framework and experimental validation of a novel navigation system designed for shoulder arthroscopy are presented. The system was designed to improve the surgeon's perception of the three-dimensional space within the human shoulder. Prior to surgery, a surface model of the shoulder was created from computed tomography images. Intraoperatively, optically tracked arthroscopic instruments were calibrated. The surface model was then registered to the patient using tracked freehand ultrasound images taken from predefined landmark regions on the scapula. Three-dimensional models of the surgical instruments were displayed, in real time, relative to the surface model in a user interface. Laboratory experiments revealed only small registration and calibration errors, with minimal time needed to complete the intraoperative tasks.

Discrete Fourier analysis of ultrasound RF time series for detection of prostate cancer.

In this paper, we demonstrate that a set of six features extracted from the discrete Fourier transform of ultrasound Radio-Frequency (RF) time series can be used to detect prostate cancer with high sensitivity and specificity. Ultrasound RF time series refer to a series of echoes received from one spatial location of tissue while the imaging probe and the tissue are fixed in position. Our previous investigations have shown that at least one feature, fractal dimension, of these signals demonstrates strong correlation with the tissue microstructure. In the current paper, six new features that represent the frequency spectrum of the RF time series have been used, in conjunction with a neural network classification approach, to detect prostate cancer in regions of tissue as small as 0.03 cm2. Based on pathology results used as gold standard, we have acquired mean accuracy of 91%, mean sensitivity of 92% and mean specificity of 90% on seven human prostates.

An estimation/correction algorithm for detecting bone edges in CT images.

The normal direction of the bone contour in computed tomography (CT) images provides important anatomical information and can guide segmentation algorithms. Since various bones in CT images have different sizes, and the intensity values of bone pixels are generally nonuniform and noisy, estimation of the normal direction using a single scale is not reliable. We propose a multiscale approach to estimate the normal direction of bone edges. The reliability of the estimation is calculated from the estimated results and, after re-scaling, the reliability is used to further correct the normal direction. The optimal scale at each point is obtained while estimating the normal direction; this scale is then used in a simple edge detector. Our experimental results have shown that use of this estimated/corrected normal direction improves the segmentation quality by decreasing the number of unexpected edges and discontinuities (gaps) of real contours. The corrected normal direction could also be used in postprocessing to delete false edges. Our segmentation algorithm is automatic, and its performance is evaluated on CT images of the human pelvis, leg, and wrist.

An interacting multiple model probabilistic data association filter for cavity boundary extraction from ultrasound images.

This paper presents a novel segmentation technique for extracting cavity contours from ultrasound images. The problem is first discretized by projecting equispaced radii from an arbitrary seed point inside the cavity toward its boundary. The distance of the cavity boundary from the seed point is modeled by the trajectory of a moving object. The motion of this moving object is assumed to be governed by a finite set of dynamical models subject to uncertainty. Candidate edge points obtained along each radius include the measurement of the object position and some false returns. The modeling approach enables us to use the interacting multiple model estimator along with a probabilistic data association filter, for contour extraction. The convergence rate of the method is very fast because it does not employ any numerical optimization. The robustness and accuracy of the method are demonstrated by segmenting contours from a series of ultrasound images. The results are validated through comparison with manual segmentations performed by an expert. An application of the method in segmenting bone contours from computed tomography images is also presented.

A haptic-based system for medical image examination.

This paper presents a haptic-based simulator for training of radiology residents and sonographers. The system consists of a force feedback haptic device providing means to interact in real-time with volumetric images of a virtual patient, captured pre-operatively from several subjects. The training system allows trainees to develop radiology techniques and knowledge of the patient's anatomy with minimum practice on live patients, or in places or at times when radiology devices or patients with rare cases may not be available. The haptic interface guarantees position correspondence between the operator's hand and a virtual probe position that slices medical volume sets in the plane of the probe. Thus the simulated procedure becomes nearly identical to the real examinations at the hospital. Different configurations of the system are implemented and presented. Future potential applications for the system are discussed as well.