Predicting Breast Cancer by Applying Deep Learning to Linked Health Records and Mammograms.

Background Computational models on the basis of deep neural networks are increasingly used to analyze health care data. However, the efficacy of traditional computational models in radiology is a matter of debate. Purpose To evaluate the accuracy and efficiency of a combined machine and deep learning approach for early breast cancer detection applied to a linked set of digital mammography images and electronic health records. Materials and Methods In this retrospective study, 52 936 images were collected in 13 234 women who underwent at least one mammogram between 2013 and 2017, and who had health records for at least 1 year before undergoing mammography. The algorithm was trained on 9611 mammograms and health records of women to make two breast cancer predictions: to predict biopsy malignancy and to differentiate normal from abnormal screening examinations. The study estimated the association of features with outcomes by using t test and Fisher exact test. The model comparisons were performed with a 95% confidence interval (CI) or by using the DeLong test. Results The resulting algorithm was validated in 1055 women and tested in 2548 women (mean age, 55 years ± 10 [standard deviation]). In the test set, the algorithm identified 34 of 71 (48%) false-negative findings on mammograms. For the malignancy prediction objective, the algorithm obtained an area under the receiver operating characteristic curve (AUC) of 0.91 (95% CI: 0.89, 0.93), with specificity of 77.3% (95% CI: 69.2%, 85.4%) at a sensitivity of 87%. When trained on clinical data alone, the model performed significantly better than the Gail model (AUC, 0.78 vs 0.54, respectively; P < .004). Conclusion The algorithm, which combined machine-learning and deep-learning approaches, can be applied to assess breast cancer at a level comparable to radiologists and has the potential to substantially reduce missed diagnoses of breast cancer. © RSNA, 2019 Online supplemental material is available for this article.

MR Imaging-derived Oxygen-Hemoglobin Dissociation Curves and Fetal-Placental Oxygen-Hemoglobin Affinities.

Purpose To generate magnetic resonance (MR) imaging-derived, oxygen-hemoglobin dissociation curves and to map fetal-placental oxygen-hemoglobin affinity in pregnant mice noninvasively by combining blood oxygen level-dependent (BOLD) T2* and oxygen-weighted T1 contrast mechanisms under different respiration challenges. Materials and Methods All procedures were approved by the Weizmann Institutional Animal Care and Use Committee. Pregnant mice were analyzed with MR imaging at 9.4 T on embryonic days 14.5 (eight dams and 58 fetuses; imprinting control region ICR strain) and 17.5 (21 dams and 158 fetuses) under respiration challenges ranging from hyperoxia to hypoxia (10 levels of oxygenation, 100%-10%; total imaging time, 100 minutes). A shorter protocol with normoxia to hyperoxia was also performed (five levels of oxygenation, 20%-100%; total imaging time, 60 minutes). Fast spin-echo anatomic images were obtained, followed by sequential acquisition of three-dimensional gradient-echo T2*- and T1-weighted images. Automated registration was applied to align regions of interest of the entire placenta, fetal liver, and maternal liver. Results were compared by using a two-tailed unpaired Student t test. R1 and R2* values were derived for each tissue. MR imaging-based oxygen-hemoglobin dissociation curves were constructed by nonlinear least square fitting of 1 minus the change in R2*divided by R2*at baseline as a function of R1 to a sigmoid-shaped curve. The apparent P50 (oxygen tension at which hemoglobin is 50% saturated) value was derived from the curves, calculated as the R1 scaled value (x) at which the change in R2* divided by R2*at baseline scaled (y) equals 0.5. Results The apparent P50 values were significantly lower in fetal liver than in maternal liver for both gestation stages (day 14.5: 21% ± 5 [P = .04] and day 17.5: 41% ± 7 [P < .0001]). The placenta showed a reduction of 18% ± 4 in mean apparent P50 values from day 14.5 to day 17.5 (P = .003). Reproduction of the MR imaging-based oxygen-hemoglobin dissociation curves with a shorter protocol that excluded the hypoxic periods was demonstrated. Conclusion MR imaging-based oxygen-hemoglobin dissociation curves and oxygen-hemoglobin affinity information were derived for pregnant mice by using 9.4-T MR imaging, which suggests a potential to overcome the need for direct sampling of fetal or maternal blood. Online supplemental material is available for this article.

Using Machine Learning to Identify Intravenous Contrast Phases on Computed Tomography.

PURPOSE: The purpose of the present work is to demonstrate the application of machine learning (ML) techniques to automatically identify the presence and physiologic phase of intravenous (IV) contrast in Computed Tomography (CT) scans of the Chest, Abdomen and Pelvis. MATERIALS AND METHODS: Training, testing and validation data were acquired from a dataset of 82,690 chest and abdomen CT examinations performed at 17 different institutions. Free text in DICOM metadata was utilized as weak labels for semi-supervised classification training. Contrast phase identification was approached as a classification task, using a 12-layer CNN and ResNet18 with four contrast-phase output. The model was reformulated to fit a regression task aimed to predict actual seconds from time of IV contrast administration to series image acquisition. Finally, transfer learning was used to optimize the model to predict contrast presence on CT Chest. RESULTS: By training based on labels inferred from noisy, free text DICOM information, contrast phase was predicted with 93.3% test accuracy (95% CI: 89.3%, 96.6%) . Regression analysis resulted in delineation of early vs late arterial phases and a nephrogenic phase in between the portal venous and delayed excretory phase. Transfer learning applied to Chest CT achieved an AUROC of 0.776 (95% CI: 0.721, 0.832) directly using the model trained for abdomen CT and 0.999 (95% CI: 0.998, 1.000) by fine-tuning. CONCLUSIONS: The presence and phase of contrast on CT examinations of the Abdomen-pelvis accurately and automatically be ascertained by a machine learning algorithm. Transfer learning applied to CT Chest achieves high precision with as little as 100 labeled samples.

Multimodal Correlative Preclinical Whole Body Imaging and Segmentation.

Segmentation of anatomical structures and particularly abdominal organs is a fundamental problem for quantitative image analysis in preclinical research. This paper presents a novel approach for whole body segmentation of small animals in a multimodal setting of MR, CT and optical imaging. The algorithm integrates multiple imaging sequences into a machine learning framework, which generates supervoxels by an efficient hierarchical agglomerative strategy and utilizes multiple SVM-kNN classifiers each constrained by a heatmap prior region to compose the segmentation. We demonstrate results showing segmentation of mice images into several structures including the heart, lungs, liver, kidneys, stomach, vena cava, bladder, tumor, and skeleton structures. Experimental validation on a large set of mice and organs, indicated that our system outperforms alternative state of the art approaches. The system proposed can be generalized to various tissues and imaging modalities to produce automatic atlas-free segmentation, thereby enabling a wide range of applications in preclinical studies of small animal imaging.

Accelerating image registration with the Johnson-Lindenstrauss lemma: application to imaging 3-D neural ultrastructure with electron microscopy.

We present a novel algorithm to accelerate feature based registration, and demonstrate the utility of the algorithm for the alignment of large transmission electron microscopy (TEM) images to create 3-D images of neural ultrastructure. In contrast to the most similar algorithms, which achieve small computation times by truncated search, our algorithm uses a novel randomized projection to accelerate feature comparison and to enable global search. Further, we demonstrate robust estimation of nonrigid transformations with a novel probabilistic correspondence framework, that enables large TEM images to be rapidly brought into alignment, removing characteristic distortions of the tissue fixation and imaging process. We analyze the impact of randomized projections upon correspondence detection, and upon transformation accuracy, and demonstrate that accuracy is maintained. We provide experimental results that demonstrate significant reduction in computation time and successful alignment of TEM images.

Small-molecule biomarkers for clinical PET imaging of apoptosis.

Apoptosis is a fundamental biologic process. Molecular imaging of apoptosis in vivo may have important implications for clinical practice, assisting in early detection of disease, monitoring of disease course, assessment of treatment efficacy, or development of new therapies. Although a PET probe for clinical imaging of apoptosis would be highly desirable, this is yet an unachieved goal, mainly because of the required challenging integration of various features, including sensitive and selective detection of the apoptotic cells, clinical aspects such as favorable biodistribution and safety profiles, and compatibility with the radiochemistry and imaging routines of clinical PET centers. Several approaches are being developed to address this challenge, all based on novel small-molecule structures targeting various steps of the apoptotic cascade. This novel concept of small-molecule PET probes for apoptosis is the focus of this review.

Accelerating feature based registration using the Johnson-Lindenstrauss Lemma.

We introduce an efficient search strategy to substantially accelerate feature based registration. Previous feature based registration algorithms often use truncated search strategies in order to achieve small computation times. Our new accelerated search strategy is based on the realization that the search for corresponding features can be dramatically accelerated by utilizing Johnson-Lindenstrauss dimension reduction. Order of magnitude calculations for the search strategy we propose here indicate that the algorithm proposed is more than a million times faster than previously utilized naive search strategies, and this advantage in speed is directly translated into an advantage in accuracy as the fast speed enables more comparisons to be made in the same amount of time. We describe the accelerated scheme together with a full complexity analysis. The registration algorithm was applied to large transmission electron microscopy (TEM) images of neural ultrastructure. Our experiments demonstrate that our algorithm enables alignment of TEM images with increased accuracy and efficiency compared to previous algorithms.

Automatic segmentation and classification of multiple sclerosis in multichannel MRI.

We introduce a multiscale approach that combines segmentation with classification to detect abnormal brain structures in medical imagery, and demonstrate its utility in automatically detecting multiple sclerosis (MS) lesions in 3-D multichannel magnetic resonance (MR) images. Our method uses segmentation to obtain a hierarchical decomposition of a multichannel, anisotropic MR scans. It then produces a rich set of features describing the segments in terms of intensity, shape, location, neighborhood relations, and anatomical context. These features are then fed into a decision forest classifier, trained with data labeled by experts, enabling the detection of lesions at all scales. Unlike common approaches that use voxel-by-voxel analysis, our system can utilize regional properties that are often important for characterizing abnormal brain structures. We provide experiments on two types of real MR images: a multichannel proton-density-, T2-, and T1-weighted dataset of 25 MS patients and a single-channel fluid attenuated inversion recovery (FLAIR) dataset of 16 MS patients. Comparing our results with lesion delineation by a human expert and with previously extensively validated results shows the promise of the approach.

Nearly automatic motion capture system for tracking octopus arm movements in 3D space.

Tracking animal movements in 3D space is an essential part of many biomechanical studies. The most popular technique for human motion capture uses markers placed on the skin which are tracked by a dedicated system. However, this technique may be inadequate for tracking animal movements, especially when it is impossible to attach markers to the animal's body either because of its size or shape or because of the environment in which the animal performs its movements. Attaching markers to an animal's body may also alter its behavior. Here we present a nearly automatic markerless motion capture system that overcomes these problems and successfully tracks octopus arm movements in 3D space. The system is based on three successive tracking and processing stages. The first stage uses a recently presented segmentation algorithm to detect the movement in a pair of video sequences recorded by two calibrated cameras. In the second stage, the results of the first stage are processed to produce 2D skeletal representations of the moving arm. Finally, the 2D skeletons are used to reconstruct the octopus arm movement as a sequence of 3D curves varying in time. Motion tracking, segmentation and reconstruction are especially difficult problems in the case of octopus arm movements because of the deformable, non-rigid structure of the octopus arm and the underwater environment in which it moves. Our successful results suggest that the motion-tracking system presented here may be used for tracking other elongated objects.

Prior knowledge driven multiscale segmentation of brain MRI.

We present a novel automatic multiscale algorithm applied to segmentation of anatomical structures in brain MRI. The algorithm which is derived from algebraic multigrid, uses a graph representation of the image and performs a coarsening process that produces a full hierarchy of segments. Our main contribution is the incorporation of prior knowledge information into the multiscale framework through a Bayesian formulation. The probabilistic information is based on an atlas prior and on a likelihood function estimated from a manually labeled training set. The significance of our new approach is that the constructed pyramid, reflects the prior knowledge formulated. This leads to an accurate and efficient methodology for detection of various anatomical structures simultaneously. Quantitative validation results on gold standard MRI show the benefit of our approach.

Atlas guided identification of brain structures by combining 3D segmentation and SVM classification.

This study presents a novel automatic approach for the identification of anatomical brain structures in magnetic resonance images (MRI). The method combines a fast multiscale multi-channel three dimensional (3D) segmentation algorithm providing a rich feature vocabulary together with a support vector machine (SVM) based classifier. The segmentation produces a full hierarchy of segments, expressed by an irregular pyramid with only linear time complexity. The pyramid provides a rich, adaptive representation of the image, enabling detection of various anatomical structures at different scales. A key aspect of the approach is the thorough set of multiscale measures employed throughout the segmentation process which are also provided at its end for clinical analysis. These features include in particular the prior probability knowledge of anatomic structures due to the use of an MRI probabilistic atlas. An SVM classifier is trained based on this set of features to identify the brain structures. We validated the approach using a gold standard real brain MRI data set. Comparison of the results with existing algorithms displays the promise of our approach.