Segmentation of cellular ultrastructures on sparsely labeled 3D electron microscopy images using deep learning.

Focused ion beam-scanning electron microscopy (FIB-SEM) images can provide a detailed view of the cellular ultrastructure of tumor cells. A deeper understanding of their organization and interactions can shed light on cancer mechanisms and progression. However, the bottleneck in the analysis is the delineation of the cellular structures to enable quantitative measurements and analysis. We mitigated this limitation using deep learning to segment cells and subcellular ultrastructure in 3D FIB-SEM images of tumor biopsies obtained from patients with metastatic breast and pancreatic cancers. The ultrastructures, such as nuclei, nucleoli, mitochondria, endosomes, and lysosomes, are relatively better defined than their surroundings and can be segmented with high accuracy using a neural network trained with sparse manual labels. Cell segmentation, on the other hand, is much more challenging due to the lack of clear boundaries separating cells in the tissue. We adopted a multi-pronged approach combining detection, boundary propagation, and tracking for cell segmentation. Specifically, a neural network was employed to detect the intracellular space; optical flow was used to propagate cell boundaries across the z-stack from the nearest ground truth image in order to facilitate the separation of individual cells; finally, the filopodium-like protrusions were tracked to the main cells by calculating the intersection over union measure for all regions detected in consecutive images along z-stack and connecting regions with maximum overlap. The proposed cell segmentation methodology resulted in an average Dice score of 0.93. For nuclei, nucleoli, and mitochondria, the segmentation achieved Dice scores of 0.99, 0.98, and 0.86, respectively. The segmentation of FIB-SEM images will enable interpretative rendering and provide quantitative image features to be associated with relevant clinical variables.

Malignancy Detection in Prostate Multi-Parametric MR Images Using U-net with Attention.

Multiparametric magnetic resonance (mpMR) images are increasingly being used for diagnosis and monitoring of prostate cancer. Detection of malignancy from prostate mpMR images requires expertise, is time consuming and prone to human error. The recent developments of U-net have demonstrated promising detection results in many medical applications. Straightforward use of U-net tends to result in over-detection in mpMR images. The recently developed attention mechanism can help retain only features relevant for malignancy detection, thus improving the detection accuracy. In this work, we propose a U-net architecture that is enhanced by the attention mechanism to detect malignancy in prostate mpMR images. This approach resulted in improved performance in terms of higher Dice score and reduced over-detection when compared to U-net in detecting malignancy.

Discrimination of Malignant and Benign Breast Lesions Using Quantitative Multiparametric MRI: A Preliminary Study.

We aimed to compare diagnostic performance in discriminating malignant and benign breast lesions between two intravoxel incoherent motion (IVIM) analysis methods for diffusion-weighted magnetic resonance imaging (DW-MRI) data and between DW- and dynamic contrast-enhanced (DCE)-MRI, and to determine if combining DW- and DCE-MRI further improves diagnostic accuracy. DW-MRI with 12 b-values and DCE-MRI were performed on 26 patients with 28 suspicious breast lesions before biopsies. The traditional biexponential fitting and a 3-b-value method were used for independent IVIM analysis of the DW-MRI data. Simulations were performed to evaluate errors in IVIM parameter estimations by the two methods across a range of signal-to-noise ratio (SNR). Pharmacokinetic modeling of DCE-MRI data was performed. Conventional radiological MRI reading yielded 86% sensitivity and 21% specificity in breast cancer diagnosis. At the same sensitivity, specificity of individual DCE- and DW-MRI markers improved to 36%-57% and that of combined DCE- or combined DW-MRI markers to 57%-71%, with DCE-MRI markers showing better diagnostic performance. The combination of DCE- and DW-MRI markers further improved specificity to 86%-93% and the improvements in diagnostic accuracy were statistically significant (P < .05) when compared with standard clinical MRI reading and most individual markers. At low breast DW-MRI SNR values (<50), like those typically seen in clinical studies, the 3-b-value approach for IVIM analysis generates markers with smaller errors and with comparable or better diagnostic performances compared with biexponential fitting. This suggests that the 3-b-value method could be an optimal IVIM-MRI method to be combined with DCE-MRI for improved diagnostic accuracy.

Early Prediction of Breast Cancer Therapy Response using Multiresolution Fractal Analysis of DCE-MRI Parametric Maps.

We aimed to determine whether multiresolution fractal analysis of voxel-based dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) parametric maps can provide early prediction of breast cancer response to neoadjuvant chemotherapy (NACT). In total, 55 patients underwent 4 DCE-MRI examinations before, during, and after NACT. The shutter-speed model was used to analyze the DCE-MRI data and generate parametric maps within the tumor region of interest. The proposed multiresolution fractal method and the more conventional methods of single-resolution fractal, gray-level co-occurrence matrix, and run-length matrix were used to extract features from the parametric maps. Only the data obtained before and after the first NACT cycle were used to evaluate early prediction of response. With a training (N = 40) and testing (N = 15) data set, support vector machine was used to assess the predictive abilities of the features in classification of pathologic complete response versus non-pathologic complete response. Generally the multiresolution fractal features from individual maps and the concatenated features from all parametric maps showed better predictive performances than conventional features, with receiver operating curve area under the curve (AUC) values of 0.91 (all parameters) and 0.80 (Ktrans), in the training and testing sets, respectively. The differences in AUC were statistically significant (P < .05) for several parametric maps. Thus, multiresolution analysis that decomposes the texture at various spatial-frequency scales may more accurately capture changes in tumor vascular heterogeneity as measured by DCE-MRI, and therefore provide better early prediction of NACT response.

Analysis of DCE-MRI for Early Prediction of Breast Cancer Therapy Response.

Positive response to neoadjuvant chemotherapy (NACT) has been correlated to better long-term outcomes in breast cancer treatment. Early prediction of response to NACT can help modify the regimen for non-responding patients, sparing them of potential toxicities of ineffective therapies. It has been observed that tumor functions such as vascularization and vascular permeability change even before noticeable changes occur in the tumor size in response to the treatment. Therefore, it is essential to have reliable imaging based features to measure these changes. Texture analysis on parametric maps from dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) has shown to be a good predictor of breast cancer response to NACT at an early stage. But hand crafted texture features might not be able to capture the rich spatio-temporal information in the parametric maps. In this work, we studied the ability of convolutional neural networks in predicting the response to NACT at an early stage.

A video/IMU hybrid system for movement estimation in infants.

Cerebral palsy is a non-progressive neurological disorder occurring in early childhood affecting body movement and muscle control. Early identification can help improve outcome through therapy-based interventions. Absence of so-called "fidgety movements" is a strong predictor of cerebral palsy. Currently, infant limb movements captured through either video cameras or accelerometers are analyzed to identify fidgety movements. However both modalities have their limitations. Video cameras do not have the high temporal resolution needed to capture subtle movements. Accelerometers have low spatial resolution and capture only relative movement. In order to overcome these limitations, we have developed a system to combine measurements from both camera and sensors to estimate the true underlying motion using extended Kalman filter. The estimated motion achieved 84% classification accuracy in identifying fidgety movements using Support Vector Machine.