Computer-Aided Diagnosis by Tissue Image Analysis as an Optical Biopsy in Hysteroscopy.

This review of our experience in computer-assisted tissue image analysis (CATIA) research shows that significant information can be extracted and used to diagnose and distinguish normal from abnormal endometrium. CATIA enabled the evaluation and differentiation between the benign and malignant endometrium during diagnostic hysteroscopy. The efficacy of texture analysis in the endometrium image during hysteroscopy was examined in 40 women, where 209 normal and 209 abnormal regions of interest (ROIs) were extracted. There was a significant difference between normal and abnormal endometrium for the statistical features (SF) features mean, variance, median, energy and entropy; for the spatial grey-level difference matrix (SGLDM) features contrast, correlation, variance, homogeneity and entropy; and for the gray-level difference statistics (GLDS) features homogeneity, contrast, energy, entropy and mean. We further evaluated 52 hysteroscopic images of 258 normal and 258 abnormal endometrium ROIs, and tissue diagnosis was verified by histopathology after biopsy. The YCrCb color system with SF, SGLDM and GLDS color texture features based on support vector machine (SVM) modeling correctly classified 81% of the cases with a sensitivity and a specificity of 78% and 81%, respectively, for normal and hyperplastic endometrium. New technical and computational advances may improve optical biopsy accuracy and assist in the precision of lesion excision during hysteroscopy. The exchange of knowledge, collaboration, identification of tasks and CATIA method selection strategy will further improve computer-aided diagnosis implementation in the daily practice of hysteroscopy.

Deep-learning-assisted analysis of echocardiographic videos improves predictions of all-cause mortality.

Machine learning promises to assist physicians with predictions of mortality and of other future clinical events by learning complex patterns from historical data, such as longitudinal electronic health records. Here we show that a convolutional neural network trained on raw pixel data in 812,278 echocardiographic videos from 34,362 individuals provides superior predictions of one-year all-cause mortality. The model's predictions outperformed the widely used pooled cohort equations, the Seattle Heart Failure score (measured in an independent dataset of 2,404 patients with heart failure who underwent 3,384 echocardiograms), and a machine learning model involving 58 human-derived variables from echocardiograms and 100 clinical variables derived from electronic health records. We also show that cardiologists assisted by the model substantially improved the sensitivity of their predictions of one-year all-cause mortality by 13% while maintaining prediction specificity. Large unstructured datasets may enable deep learning to improve a wide range of clinical prediction models.

Medical Image Analysis Using AM-FM Models and Methods.

Medical image analysis methods require the use of effective representations for differentiating between lesions, diseased regions, and normal structure. Amplitude Modulation-Frequency Modulation (AM-FM) models provide effective representations through physically meaningful descriptors of complex non-stationary structures that can differentiate between the different lesions and normal structure. Based on AM-FM models, medical images are decomposed into AM-FM components where the instantaneous frequency provides a descriptor of local texture, the instantaneous amplitude captures slowly-varying brightness variations, while the instantaneous phase provides for a powerful descriptor of location, generalizing the traditionally important role of phase in the Fourier Analysis of images. Over the years, AM-FM representations have been used in a wide variety of medical image analysis applications based on a vastly reduced number of features that can be easily learned by simple classifiers. The paper provides an overview of AM-FM models and methods, followed by applications in medical image analysis. We also provide a summary of emerging trends and future directions.

Multidataset Independent Subspace Analysis With Application to Multimodal Fusion.

Unsupervised latent variable models-blind source separation (BSS) especially-enjoy a strong reputation for their interpretability. But they seldom combine the rich diversity of information available in multiple datasets, even though multidatasets yield insightful joint solutions otherwise unavailable in isolation. We present a direct, principled approach to multidataset combination that takes advantage of multidimensional subspace structures. In turn, we extend BSS models to capture the underlying modes of shared and unique variability across and within datasets. Our approach leverages joint information from heterogeneous datasets in a flexible and synergistic fashion. We call this method multidataset independent subspace analysis (MISA). Methodological innovations exploiting the Kotz distribution for subspace modeling, in conjunction with a novel combinatorial optimization for evasion of local minima, enable MISA to produce a robust generalization of independent component analysis (ICA), independent vector analysis (IVA), and independent subspace analysis (ISA) in a single unified model. We highlight the utility of MISA for multimodal information fusion, including sample-poor regimes ( N = 600 ) and low signal-to-noise ratio, promoting novel applications in both unimodal and multimodal brain imaging data.

Radiogenomics for Precision Medicine With a Big Data Analytics Perspective.

Precision medicine promises better healthcare delivery by improving clinical practice. Using evidence-based substratification of patients, the objective is to achieve better prognosis, diagnosis, and treatment that will transform existing clinical pathways toward optimizing care for the specific needs of each patient. The wealth of today's healthcare data, often characterized as big data, provides invaluable resources toward new knowledge discovery that has the potential to advance precision medicine. The latter requires interdisciplinary efforts that will capitalize the information, know-how, and medical data of newly formed groups fusing different backgrounds and expertise. The objective of this paper is to provide insights with respect to the state-of-the-art research in precision medicine. More specifically, our goal is to highlight the fundamental challenges in emerging fields of radiomics and radiogenomics by reviewing the case studies of Cancer and Alzheimer's disease, describe the computational challenges from a big data analytics perspective, and discuss standardization and open data initiatives that will facilitate the adoption of precision medicine methods and practices.

Real-Time Adaptation to Time-Varying Constraints for Medical Video Communications.

The wider adoption of mobile Health video communication systems in standard clinical practice requires real-time control to provide for adequate levels of clinical video quality to support reliable diagnosis. The latter can only be achieved with real-time adaptation to time-varying wireless networks' state to guarantee clinically acceptable performance throughout the streaming session, while conforming to device capabilities for supporting real-time encoding. We propose an adaptive video encoding framework based on multi-objective optimization that jointly maximizes the encoded video's quality and encoding rate (in frames per second) while minimizing bitrate demands. For this purpose, we construct a dense encoding space and use linear regression to estimate forward prediction models for quality, bitrate, and computational complexity. The prediction models are then used in an adaptive control framework that can fine-tune video encoding based on real-time constraints. We validate the system using a leave-one-out algorithm applied to ten ultrasound videos of the common carotid artery. The prediction models can estimate structural similarity quality with a median accuracy error of less than 1%, bitrate demands with deviation error of 10% or less, and encoding frame rate within a 6% margin. Real-time adaptation at a group of pictures level is demonstrated using the high efficiency video coding standard. The effectiveness of the proposed framework compared to static, nonadaptive approaches is demonstrated for different modes of operation, achieving significant quality gains, bitrate demands reductions, and performance improvements, in real-life scenarios imposing time-varying constraints. Our approach is generic and should be applicable to other medical video modalities with different applications.

Blind Source Separation for Unimodal and Multimodal Brain Networks: A Unifying Framework for Subspace Modeling.

In the past decade, numerous advances in the study of the human brain were fostered by successful applications of blind source separation (BSS) methods to a wide range of imaging modalities. The main focus has been on extracting "networks" represented as the underlying latent sources. While the broad success in learning latent representations from multiple datasets has promoted the wide presence of BSS in modern neuroscience, it also introduced a wide variety of objective functions, underlying graphical structures, and parameter constraints for each method. Such diversity, combined with a host of datatype-specific know-how, can cause a sense of disorder and confusion, hampering a practitioner's judgment and impeding further development. We organize the diverse landscape of BSS models by exposing its key features and combining them to establish a novel unifying view of the area. In the process, we unveil important connections among models according to their properties and subspace structures. Consequently, a high-level descriptive structure is exposed, ultimately helping practitioners select the right model for their applications. Equipped with that knowledge, we review the current state of BSS applications to neuroimaging. The gained insight into model connections elicits a broader sense of generalization, highlighting several directions for model development. In light of that, we discuss emerging multi-dataset multidimensional (MDM) models and summarize their benefits for the study of the healthy brain and disease-related changes.

A multiscale decomposition approach to detect abnormal vasculature in the optic disc.

This paper presents a multiscale method to detect neovascularization in the optic disc (NVD) using fundus images. Our method is applied to a manually selected region of interest (ROI) containing the optic disc. All the vessels in the ROI are segmented by adaptively combining contrast enhancement methods with a vessel segmentation technique. Textural features extracted using multiscale amplitude-modulation frequency-modulation, morphological granulometry, and fractal dimension are used. A linear SVM is used to perform the classification, which is tested by means of 10-fold cross-validation. The performance is evaluated using 300 images achieving an AUC of 0.93 with maximum accuracy of 88%.

An effective ultrasound video communication system using despeckle filtering and HEVC.

The recent emergence of the high-efficiency video coding (HEVC) standard promises to deliver significant bitrate savings over current and prior video compression standards, while also supporting higher resolutions that can meet the clinical acquisition spatiotemporal settings. The effective application of HEVC to medical ultrasound necessitates a careful evaluation of strict clinical criteria that guarantee that clinical quality will not be sacrificed in the compression process. Furthermore, the potential use of despeckle filtering prior to compression provides for the possibility of significant additional bitrate savings that have not been previously considered. This paper provides a thorough comparison of the use of MPEG-2, H.263, MPEG-4, H.264/AVC, and HEVC for compressing atherosclerotic plaque ultrasound videos. For the comparisons, we use both subjective and objective criteria based on plaque structure and motion. For comparable clinical video quality, experimental evaluation on ten videos demonstrates that HEVC reduces bitrate requirements by as much as 33.2% compared to H.264/AVC and up to 71% compared to MPEG-2. The use of despeckle filtering prior to compression is also investigated as a method that can reduce bitrate requirements through the removal of higher frequency components without sacrificing clinical quality. Based on the use of three despeckle filtering methods with both H.264/AVC and HEVC, we find that prior filtering can yield additional significant bitrate savings. The best performing despeckle filter (DsFlsmv) achieves bitrate savings of 43.6% and 39.2% compared to standard nonfiltered HEVC and H.264/AVC encoding, respectively.

Detecting volumetric changes in fMRI connectivity networks in schizophrenia patients.

There is a growing interest in identifying neuroimaging-based biomarkers for schizophrenia. Previous studies have shown both functional and structural brain abnormalities in schizophrenia patients. One main category of these findings consists of volumetric abnormalities in brain structure in different cortical and subcortical structures in patients' brain. However there has been little work investigating changes in the brain's functional volumes. Nor has there been work studying differences in brain networks as opposed to single regions. In this study, we investigated the volumes of functional networks as potential biomarkers. Independent component analysis was used to decompose fMRI images into maximally independent spatial maps and corresponding time-courses. Volume of functional networks was computed from subject-specific back reconstructed spatial maps. The results show that different nodes of the default-mode network exhibit volumetric abnormalities in schizophrenia patients. Interestingly these networks are larger in patients compared to controls.

High-resolution, low-delay, and error-resilient medical ultrasound video communication using H.264/AVC over mobile WiMAX networks.

In this study, we describe an effective video communication framework for the wireless transmission of H.264/AVC medical ultrasound video over mobile WiMAX networks. Medical ultrasound video is encoded using diagnostically-driven, error resilient encoding, where quantization levels are varied as a function of the diagnostic significance of each image region. We demonstrate how our proposed system allows for the transmission of high-resolution clinical video that is encoded at the clinical acquisition resolution and can then be decoded with low-delay. To validate performance, we perform OPNET simulations of mobile WiMAX Medium Access Control (MAC) and Physical (PHY) layers characteristics that include service prioritization classes, different modulation and coding schemes, fading channels conditions, and mobility. We encode the medical ultrasound videos at the 4CIF (704 × 576) resolution that can accommodate clinical acquisition that is typically performed at lower resolutions. Video quality assessment is based on both clinical (subjective) and objective evaluations.

Optimal compressed sensing reconstructions of fMRI using 2D deterministic and stochastic sampling geometries.

BACKGROUND: Compressive sensing can provide a promising framework for accelerating fMRI image acquisition by allowing reconstructions from a limited number of frequency-domain samples. Unfortunately, the majority of compressive sensing studies are based on stochastic sampling geometries that cannot guarantee fast acquisitions that are needed for fMRI. The purpose of this study is to provide a comprehensive optimization framework that can be used to determine the optimal 2D stochastic or deterministic sampling geometry, as well as to provide optimal reconstruction parameter values for guaranteeing image quality in the reconstructed images. METHODS: We investigate the use of frequency-space (k-space) sampling based on: (i) 2D deterministic geometries of dyadic phase encoding (DPE) and spiral low pass (SLP) geometries, and (ii) 2D stochastic geometries based on random phase encoding (RPE) and random samples on a PDF (RSP). Overall, we consider over 36 frequency-sampling geometries at different sampling rates. For each geometry, we compute optimal reconstructions of single BOLD fMRI ON & OFF images, as well as BOLD fMRI activity maps based on the difference between the ON and OFF images. We also provide an optimization framework for determining the optimal parameters and sampling geometry prior to scanning. RESULTS: For each geometry, we show that reconstruction parameter optimization converged after just a few iterations. Parameter optimization led to significant image quality improvements. For activity detection, retaining only 20.3% of the samples using SLP gave a mean PSNR value of 57.58 dB. We also validated this result with the use of the Structural Similarity Index Matrix (SSIM) image quality metric. SSIM gave an excellent mean value of 0.9747 (max = 1). This indicates that excellent reconstruction results can be achieved. Median parameter values also gave excellent reconstruction results for the ON/OFF images using the SLP sampling geometry (mean SSIM > =0.93). Here, median parameter values were obtained using mean-SSIM optimization. This approach was also validated using leave-one-out. CONCLUSIONS: We have found that compressive sensing parameter optimization can dramatically improve fMRI image reconstruction quality. Furthermore, 2D MRI scanning based on the SLP geometries consistently gave the best image reconstruction results. The implication of this result is that less complex sampling geometries will suffice over random sampling. We have also found that we can obtain stable parameter regions that can be used to achieve specific levels of image reconstruction quality when combined with specific k-space sampling geometries. Furthermore, median parameter values can be used to obtain excellent reconstruction results.

Prediction of high-risk asymptomatic carotid plaques based on ultrasonic image features.

Carotid plaques have been associated with ipsilateral neurological symptoms. High-resolution ultrasound can provide information not only on the degree of carotid artery stenosis but also on the characteristics of the arterial wall including the size and consistency of atherosclerotic plaques. The aim of this study is to determine whether the addition of ultrasonic plaque texture features to clinical features in patients with asymptomatic internal carotid artery stenosis (ACS) improves the ability to identify plaques that will produce stroke. 1121 patients with ACS have been scanned with ultrasound and followed for a mean of 4 years. It is shown that the combination of texture features based on second-order statistics spatial gray level dependence matrices (SGLDM) and clinical factors improves stroke prediction (by correctly predicting 89 out of the 108 cases that were symptomatic). Here, the best classification results of 77 ±1.8% were obtained from the use of the SGLDM texture features with support vector machine classifiers. The combination of morphological features with clinical features gave slightly worse classification results of 76 ±2.6% . These findings need to be further validated in additional prospective studies.

Detection of neovascularization in the optic disc using an AM-FM representation, granulometry, and vessel segmentation.

Neovascularization, defined as abnormal formation of blood vessels in the retina, is a sight-threatening condition indicative of late-stage diabetic retinopathy (DR). Ischemia due to leakage of blood vessels causes the body to produce new and weak vessels that can lead to complications such as vitreous hemorrhages. Neovascularization on the disc (NVD) is diagnosed when new vessels are located within one disc-diameter of the optic disc. Accurately detecting NVD is important in preventing vision loss due to DR. This paper presents a method for detecting NVD in digital fundus images. First, a region of interest (ROI) containing the optic disc is manually selected from the image. By adaptively combining contrast enhancement methods with a vessel segmentation technique, the ROI is reduced to the regions indicated by the segmented vessels. Textural features extracted by using amplitude-modulation frequency-modulation (AM-FM) techniques and granulometry are used to differentiate NVD from a normal optic disc. Partial least squares is used to perform the final classification. Leave-one-out cross-validation was used to evaluate the performance of the system with 27 NVD and 30 normal cases. We obtained an area under the receiver operator characteristic curve (AUC) of 0.85 by using all features, increasing to 0.94 with feature selection.

Automatic detection of diabetic retinopathy and age-related macular degeneration in digital fundus images.

PURPOSE: To describe and evaluate the performance of an algorithm that automatically classifies images with pathologic features commonly found in diabetic retinopathy (DR) and age-related macular degeneration (AMD). METHODS: Retinal digital photographs (N = 2247) of three fields of view (FOV) were obtained of the eyes of 822 patients at two centers: The Retina Institute of South Texas (RIST, San Antonio, TX) and The University of Texas Health Science Center San Antonio (UTHSCSA). Ground truth was provided for the presence of pathologic conditions, including microaneurysms, hemorrhages, exudates, neovascularization in the optic disc and elsewhere, drusen, abnormal pigmentation, and geographic atrophy. The algorithm was used to report on the presence or absence of disease. A detection threshold was applied to obtain different values of sensitivity and specificity with respect to ground truth and to construct a receiver operating characteristic (ROC) curve. RESULTS: The system achieved an average area under the ROC curve (AUC) of 0.89 for detection of DR and of 0.92 for detection of sight-threatening DR (STDR). With a fixed specificity of 0.50, the system's sensitivity ranged from 0.92 for all DR cases to 1.00 for clinically significant macular edema (CSME). CONCLUSIONS: A computer-aided algorithm was trained to detect different types of pathologic retinal conditions. The cases of hard exudates within 1 disc diameter (DD) of the fovea (surrogate for CSME) were detected with very high accuracy (sensitivity = 1, specificity = 0.50), whereas mild nonproliferative DR was the most challenging condition (sensitivity = 0.92, specificity = 0.50). The algorithm was also tested on images with signs of AMD, achieving a performance of AUC of 0.84 (sensitivity = 0.94, specificity = 0.50).

A 3D freehand ultrasound system for multi-view reconstructions from sparse 2D scanning planes.

BACKGROUND: A significant limitation of existing 3D ultrasound systems comes from the fact that the majority of them work with fixed acquisition geometries. As a result, the users have very limited control over the geometry of the 2D scanning planes. METHODS: We present a low-cost and flexible ultrasound imaging system that integrates several image processing components to allow for 3D reconstructions from limited numbers of 2D image planes and multiple acoustic views. Our approach is based on a 3D freehand ultrasound system that allows users to control the 2D acquisition imaging using conventional 2D probes.For reliable performance, we develop new methods for image segmentation and robust multi-view registration. We first present a new hybrid geometric level-set approach that provides reliable segmentation performance with relatively simple initializations and minimum edge leakage. Optimization of the segmentation model parameters and its effect on performance is carefully discussed. Second, using the segmented images, a new coarse to fine automatic multi-view registration method is introduced. The approach uses a 3D Hotelling transform to initialize an optimization search. Then, the fine scale feature-based registration is performed using a robust, non-linear least squares algorithm. The robustness of the multi-view registration system allows for accurate 3D reconstructions from sparse 2D image planes. RESULTS: Volume measurements from multi-view 3D reconstructions are found to be consistently and significantly more accurate than measurements from single view reconstructions. The volume error of multi-view reconstruction is measured to be less than 5% of the true volume. We show that volume reconstruction accuracy is a function of the total number of 2D image planes and the number of views for calibrated phantom. In clinical in-vivo cardiac experiments, we show that volume estimates of the left ventricle from multi-view reconstructions are found to be in better agreement with clinical measures than measures from single view reconstructions. CONCLUSIONS: Multi-view 3D reconstruction from sparse 2D freehand B-mode images leads to more accurate volume quantification compared to single view systems. The flexibility and low-cost of the proposed system allow for fine control of the image acquisition planes for optimal 3D reconstructions from multiple views.

Multiscale AM-FM demodulation and image reconstruction methods with improved accuracy.

We develop new multiscale amplitude-modulation frequency-modulation (AM-FM) demodulation methods for image processing. The approach is based on three basic ideas: (i) AM-FM demodulation using a new multiscale filterbank, (ii) new, accurate methods for instantaneous frequency (IF) estimation, and (iii) multiscale least squares AM-FM reconstructions. In particular, we introduce a variable-spacing local linear phase (VS-LLP) method for improved instantaneous frequency (IF) estimation and compare it to an extended quasilocal method and the quasi-eigen function approximation (QEA). It turns out that the new VS-LLP method is a generalization of the QEA method where we choose the best integer spacing between the samples to adapt as a function of frequency. We also introduce a new quasi-local method (QLM) for IF and IA estimation and discuss some of its advantages and limitations. The new IF estimation methods lead to significantly improved estimates. We present different multiscale decompositions to show that the proposed methods can be used to reconstruct and analyze general images.

A standardised protocol for texture feature analysis of endoscopic images in gynaecological cancer.

BACKGROUND: In the development of tissue classification methods, classifiers rely on significant differences between texture features extracted from normal and abnormal regions. Yet, significant differences can arise due to variations in the image acquisition method. For endoscopic imaging of the endometrium, we propose a standardized image acquisition protocol to eliminate significant statistical differences due to variations in: (i) the distance from the tissue (panoramic vs close up), (ii) difference in viewing angles and (iii) color correction. METHODS: We investigate texture feature variability for a variety of targets encountered in clinical endoscopy. All images were captured at clinically optimum illumination and focus using 720 x 576 pixels and 24 bits color for: (i) a variety of testing targets from a color palette with a known color distribution, (ii) different viewing angles, (iv) two different distances from a calf endometrial and from a chicken cavity. Also, human images from the endometrium were captured and analysed. For texture feature analysis, three different sets were considered: (i) Statistical Features (SF), (ii) Spatial Gray Level Dependence Matrices (SGLDM), and (iii) Gray Level Difference Statistics (GLDS). All images were gamma corrected and the extracted texture feature values were compared against the texture feature values extracted from the uncorrected images. Statistical tests were applied to compare images from different viewing conditions so as to determine any significant differences. RESULTS: For the proposed acquisition procedure, results indicate that there is no significant difference in texture features between the panoramic and close up views and between angles. For a calibrated target image, gamma correction provided an acquired image that was a significantly better approximation to the original target image. In turn, this implies that the texture features extracted from the corrected images provided for better approximations to the original images. Within the proposed protocol, for human ROIs, we have found that there is a large number of texture features that showed significant differences between normal and abnormal endometrium. CONCLUSION: This study provides a standardized protocol for avoiding any significant texture feature differences that may arise due to variability in the acquisition procedure or the lack of color correction. After applying the protocol, we have found that significant differences in texture features will only be due to the fact that the features were extracted from different types of tissue (normal vs abnormal).