Transfer learning-based approach for automated kidney segmentation on multiparametric MRI sequences.

Purpose: Multiparametric magnetic resonance imaging (mp-MRI) is being investigated for kidney cancer because of better soft tissue contrast ability. The necessity of manual labels makes the development of supervised kidney segmentation algorithms challenging for each mp-MRI protocol. Here, we developed a transfer learning-based approach to improve kidney segmentation on a small dataset of five other mp-MRI sequences. Approach: We proposed a fully automated two-dimensional (2D) attention U-Net model for kidney segmentation on T1 weighted-nephrographic phase contrast enhanced (CE)-MRI (T1W-NG) dataset ( N = 108 ). The pretrained weights of T1W-NG kidney segmentation model transferred to five other distinct mp-MRI sequences model (T2W, T1W-in-phase (T1W-IP), T1W-out-of-phase (T1W-OP), T1W precontrast (T1W-PRE), and T1W-corticomedullary-CE (T1W-CM), N = 50 ) and fine-tuned by unfreezing the layers. The individual model performances were evaluated with and without transfer-learning fivefold cross-validation on average Dice similarity coefficient (DSC), absolute volume difference, Hausdorff distance (HD), and center-of-mass distance (CD) between algorithm generated and manually segmented kidneys. Results: The developed 2D attention U-Net model for T1W-NG produced kidney segmentation DSC of 89.34 ± 5.31 % . Compared with randomly initialized weight models, the transfer learning-based models of five mp-MRI sequences showed average increase of 2.96% in DSC of kidney segmentation ( p = 0.001 to 0.006). Specifically, the transfer-learning approach increased average DSC on T2W from 87.19% to 89.90%, T1W-IP from 83.64% to 85.42%, T1W-OP from 79.35% to 83.66%, T1W-PRE from 82.05% to 85.94%, and T1W-CM from 85.65% to 87.64%. Conclusions: We demonstrate that a pretrained model for automated kidney segmentation of one mp-MRI sequence improved automated kidney segmentation on five other additional sequences.

Fully automated detection of prostate transition zone tumors on T2-weighted and apparent diffusion coefficient (ADC) map MR images using U-Net ensemble.

PURPOSE: Accurate detection of transition zone (TZ) prostate cancer (PCa) on magnetic resonance imaging (MRI) remains challenging using clinical subjective assessment due to overlap between PCa and benign prostatic hyperplasia (BPH). The objective of this paper is to describe a deep-learning-based framework for fully automated detection of PCa in the TZ using T2-weighted (T2W) and apparent diffusion coefficient (ADC) map MR images. METHOD: This was a single-center IRB-approved cross-sectional study of men undergoing 3T MRI on two systems. The dataset consisted of 196 patients (103 with and 93 without clinically significant [Grade Group 2 or higher] TZ PCa) to train and test our proposed methodology, with an additional 168 patients with peripheral zone PCa used only for training. We proposed an ensemble of classifiers in which multiple U-Net-based models are designed for prediction of TZ PCa location on ADC map MR images, with initial automated segmentation of the prostate to guide detection. We compared accuracy of ADC alone to T2W and combined ADC+T2W MRI for input images, and investigated improvements using ensembles over their constituent models with different methods of diversity in individual models by hyperparameter configuration, loss function and model architecture. RESULTS: Our developed algorithm reported sensitivity and precision of 0.829 and 0.617 in 56 test cases containing 31 instances of TZ PCa and in 25 patients without clinically significant TZ tumors. Patient-wise classification accuracy had an area under receiver operator characteristic curve (AUROC) of 0.974. Single U-Net models using ADC alone (sensitivity 0.829, precision 0.534) outperformed assessment using T2W (sensitivity 0.086, precision 0.081) and assessment using combined ADC+T2W (sensitivity 0.687, precision 0.489). While the ensemble of U-Nets with varying hyperparameters demonstrated the highest performance, all ensembles improved PCa detection compared to individual models, with sensitivities and precisions close to the collective best of constituent models. CONCLUSION: We describe a deep-learning-based method for fully automated TZ PCa detection using ADC map MR images that outperformed assessment by T2W and ADC+T2W.

Fully automated estimation of the mean linear intercept in histopathology images of mouse lung tissue.

Purpose: The mean linear intercept (MLI) score is a common metric for quantification of injury in lung histopathology images. The automated estimation of the MLI score is a challenging task because it requires accurate segmentation of different biological components of the lung tissue. Therefore, the most widely used approaches for MLI quantification are based on manual/semi-automated assessment of lung histopathology images, which can be expensive and time-consuming. We describe a fully automated pipeline for MLI estimation, which is capable of producing results comparable to human raters. Approach: We use a convolutional neural network based on U-Net architecture to segment the diagnostically relevant tissue segments in the whole slide images (WSI) of the mouse lung tissue. The proposed method extracts multiple field-of-view (FOV) images from the tissue segments and screen the FOV images, rejecting images based on presence of certain biological structures (i.e., blood vessels and bronchi). We used color slicing and region growing for segmentation of different biological structures in each FOV image. Results: The proposed method was tested on ten WSIs from mice and compared against the scores provided by three human raters. In segmenting the relevant tissue segments, our method obtained a mean accuracy, Dice coefficient, and Hausdorff distance of 98.34%, 98.22%, and 109.68 µ m , respectively. Our proposed method yields a mean precision, recall, and F 1 -score of 93.37%, 83.47%, and 87.87%, respectively, in screening of FOV images. There was substantial agreement found between the proposed method and the manual scores (Fleiss Kappa score of 0.76). The mean difference between the calculated MLI score between the automated method and average rater's score was 2.33 ± 4.13 ( 4.25 % ± 5.67 % ). Conclusion: The proposed pipeline for automated calculation of the MLI score demonstrates high consistency and accuracy with human raters and can be a potential replacement for manual/semi-automated approaches in the field.

Automated classification of solid renal masses on contrast-enhanced computed tomography images using convolutional neural network with decision fusion.

OBJECTIVES: To develop a deep learning-based method for automated classification of renal cell carcinoma (RCC) from benign solid renal masses using contrast-enhanced computed tomography (CECT) images. METHODS: This institutional review board-approved retrospective study evaluated CECT in 315 patients with 77 benign (57 oncocytomas, and 20 fat-poor angiomyolipoma) and 238 malignant (RCC: 123 clear cell, 69 papillary, and 46 chromophobe subtypes) tumors identified consecutively between 2015 and 2017. We employed a decision fusion-based model to aggregate slice level predictions determined by convolutional neural network (CNN) via a majority voting system to evaluate renal masses on CECT. The CNN-based model was trained using 7023 slices with renal masses manually extracted from CECT images of 155 patients, cropped automatically around kidneys, and augmented artificially. We also examined the fully automated approach for renal mass evaluation on CECT. Moreover, a 3D CNN was trained and tested using the same datasets and the obtained results were compared with those acquired from slice-wise algorithms. RESULTS: For differentiation of RCC versus benign solid masses, the semi-automated majority voting-based CNN algorithm achieved accuracy, precision, and recall of 83.75%, 89.05%, and 91.73% using 160 test cases, respectively. Fully automated pipeline yielded accuracy, precision, and recall of 77.36%, 85.92%, and 87.22% on the same test cases, respectively. 3D CNN reported accuracy, precision, and recall of 79.24%, 90.32%, and 84.21% using 160 test cases, respectively. CONCLUSIONS: A semi-automated majority voting CNN-based methodology enabled accurate classification of RCC from benign neoplasms among solid renal masses on CECT. KEY POINTS: • Our proposed semi-automated majority voting CNN-based algorithm achieved accuracy of 83.75% for the diagnosis of RCC from benign solid renal masses on CECT images. • A fully automated CNN-based methodology classified solid renal masses with moderate accuracy of 77.36% using the same test images. • Employing 3D CNN-based methodology yielded slightly lower accuracy for renal mass classification compared with the semi- automated 2D CNN-based algorithm (79.24%).

Fully automated localization of prostate peripheral zone tumors on apparent diffusion coefficient map MR images using an ensemble learning method.

BACKGROUND: Accurate detection and localization of prostate cancer (PCa) in men undergoing prostate MRI is a fundamental step for future targeted prostate biopsies and treatment planning. Fully automated localization of peripheral zone (PZ) PCa using the apparent diffusion coefficient (ADC) map might be clinically useful. PURPOSE/HYPOTHESIS: To describe automated localization of PCa in the PZ on ADC map MR images using an ensemble U-Net-based model. STUDY TYPE: Retrospective, case-control. POPULATION: In all, 226 patients (154 and 72 patients with and without clinically significant PZ PCa, respectively), training, and testing was performed using dataset images of 146 and 80 patients, respectively. FIELD STRENGTH: 3T, ADC maps. SEQUENCE: ADC map. ASSESSMENT: The ground truth was established by manual delineation of the prostate and prostate PZ tumors on ADC maps by dedicated radiologists using MRI-radical prostatectomy maps as a reference standard. Statistical Tests: Performance of the ensemble model was evaluated using Dice similarity coefficient (DSC), sensitivity, and specificity metrics on a per-slice basis. Receiver operating characteristic (ROC) curve and area under the curve (AUC) were employed as well. The paired t-test was used to test the differences between the performances of constituent networks of the ensemble model. RESULTS: Our developed algorithm yielded DSC, sensitivity, and specificity of 86.72% ± 9.93%, 85.76% ± 23.33%, and 76.44% ± 23.70%, respectively (mean ± standard deviation) on 80 test cases consisting of 41 and 39 instances from patients with and without clinically significant tumors including 660 extracted 2D slices. AUC was reported as 0.779. DATA CONCLUSION: An ensemble U-Net-based approach can accurately detect and segment PCa in the PZ from ADC map MR prostate images. LEVEL OF EVIDENCE: 4 Technical Efficacy: Stage 1 J. Magn. Reson. Imaging 2020;51:1223-1234.

Automated segmentation of prostate zonal anatomy on T2-weighted (T2W) and apparent diffusion coefficient (ADC) map MR images using U-Nets.

PURPOSE: Accurate regional segmentation of the prostate boundaries on magnetic resonance (MR) images is a fundamental requirement before automated prostate cancer diagnosis can be achieved. In this paper, we describe a novel methodology to segment prostate whole gland (WG), central gland (CG), and peripheral zone (PZ), where PZ + CG = WG, from T2W and apparent diffusion coefficient (ADC) map prostate MR images. METHODS: We designed two similar models each made up of two U-Nets to delineate the WG, CG, and PZ from T2W and ADC map MR images, separately. The U-Net, which is a modified version of a fully convolutional neural network, includes contracting and expanding paths with convolutional, pooling, and upsampling layers. Pooling and upsampling layers help to capture and localize image features with a high spatial consistency. We used a dataset consisting of 225 patients (combining 153 and 72 patients with and without clinically significant prostate cancer) imaged with multiparametric MRI at 3 Tesla. RESULTS AND CONCLUSION: Our proposed model for prostate zonal segmentation from T2W was trained and tested using 1154 and 1587 slices of 100 and 125 patients, respectively. Median of Dice similarity coefficient (DSC) on test dataset for prostate WG, CG, and PZ were 95.33 ± 7.77%, 93.75 ± 8.91%, and 86.78 ± 3.72%, respectively. Designed model for regional prostate delineation from ADC map images was trained and validated using 812 and 917 slices from 100 and 125 patients. This model yielded a median DSC of 92.09 ± 8.89%, 89.89 ± 10.69%, and 86.1 ± 9.56% for prostate WG, CG, and PZ on test samples, respectively. Further investigation indicated that the proposed algorithm reported high DSC for prostate WG segmentation from both T2W and ADC map MR images irrespective of WG size. In addition, segmentation accuracy in terms of DSC does not significantly vary among patients with or without significant tumors. SIGNIFICANCE: We describe a method for automated prostate zonal segmentation using T2W and ADC map MR images independent of prostate size and the presence or absence of tumor. Our results are important in terms of clinical perspective as fully automated methods for ADC map images, which are considered as one of the most important sequences for prostate cancer detection in the PZ and CG, have not been reported previously.

Rotationally resliced 3D prostate TRUS segmentation using convex optimization with shape priors.

PURPOSE: Efficient and accurate segmentations of 3D end-firing transrectal ultrasound (TRUS) images play an important role in planning of 3D TRUS guided prostate biopsy. However, poor image quality of the input 3D TRUS images, such as strong imaging artifacts and speckles, often makes it a challenging task to extract the prostate boundaries accurately and efficiently. METHODS: In this paper, the authors propose a novel convex optimization-based approach to delineate the prostate surface from a given 3D TRUS image, which reduces the original 3D segmentation problem to a sequence of simple 2D segmentation subproblems over the rotational reslices of the 3D TRUS volume. Essentially, the authors introduce a novel convex relaxation-based contour evolution approach to each 2D slicewise image segmentation with the joint optimization of shape information, where the learned 2D nonlinear statistical shape prior is incorporated to segment the initial slice, its result is propagated as a shape constraint to the segmentation of the following slices. In practice, the proposed segmentation algorithm is implemented on a GPU to achieve the high computational performance. RESULTS: Experimental results using 30 patient 3D TRUS images show that the proposed method can achieve a mean Dice similarity coefficient of 93.4% ± 2.2% in 20 s for one 3D image, outperforming the existing local-optimization-based methods, e.g., level-set and active-contour, in terms of accuracy and efficiency. In addition, inter- and intraobserver variability experiments show its good reproducibility. CONCLUSIONS: A semiautomatic segmentation approach is proposed and evaluated to extract the prostate boundary from 3D TRUS images acquired by a 3D end-firing TRUS guided prostate biopsy system. Experimental results suggest that it may be suitable for the clinical use involving the image guided prostate biopsy procedures.

3D prostate TRUS segmentation using globally optimized volume-preserving prior.

An efficient and accurate segmentation of 3D transrectal ultrasound (TRUS) images plays an important role in the planning and treatment of the practical 3D TRUS guided prostate biopsy. However, a meaningful segmentation of 3D TRUS images tends to suffer from US speckles, shadowing and missing edges etc, which make it a challenging task to delineate the correct prostate boundaries. In this paper, we propose a novel convex optimization based approach to extracting the prostate surface from the given 3D TRUS image, while preserving a new global volume-size prior. We, especially, study the proposed combinatorial optimization problem by convex relaxation and introduce its dual continuous max-flow formulation with the new bounded flow conservation constraint, which results in an efficient numerical solver implemented on GPUs. Experimental results using 12 patient 3D TRUS images show that the proposed approach while preserving the volume-size prior yielded a mean DSC of 89.5% +/- 2.4%, a MAD of 1.4 +/- 0.6 mm, a MAXD of 5.2 +/- 3.2 mm, and a VD of 7.5% +/- 6.2% in - 1 minute, deomonstrating the advantages of both accuracy and efficiency. In addition, the low standard deviation of the segmentation accuracy shows a good reliability of the proposed approach.

Prostate segmentation: an efficient convex optimization approach with axial symmetry using 3-D TRUS and MR images.

We propose a novel global optimization-based approach to segmentation of 3-D prostate transrectal ultrasound (TRUS) and T2 weighted magnetic resonance (MR) images, enforcing inherent axial symmetry of prostate shapes to simultaneously adjust a series of 2-D slice-wise segmentations in a "global" 3-D sense. We show that the introduced challenging combinatorial optimization problem can be solved globally and exactly by means of convex relaxation. In this regard, we propose a novel coherent continuous max-flow model (CCMFM), which derives a new and efficient duality-based algorithm, leading to a GPU-based implementation to achieve high computational speeds. Experiments with 25 3-D TRUS images and 30 3-D T2w MR images from our dataset, and 50 3-D T2w MR images from a public dataset, demonstrate that the proposed approach can segment a 3-D prostate TRUS/MR image within 5-6 s including 4-5 s for initialization, yielding a mean Dice similarity coefficient of 93.2%±2.0% for 3-D TRUS images and 88.5%±3.5% for 3-D MR images. The proposed method also yields relatively low intra- and inter-observer variability introduced by user manual initialization, suggesting a high reproducibility, independent of observers.

Dual optimization based prostate zonal segmentation in 3D MR images.

Efficient and accurate segmentation of the prostate and two of its clinically meaningful sub-regions: the central gland (CG) and peripheral zone (PZ), from 3D MR images, is of great interest in image-guided prostate interventions and diagnosis of prostate cancer. In this work, a novel multi-region segmentation approach is proposed to simultaneously segment the prostate and its two major sub-regions from only a single 3D T2-weighted (T2w) MR image, which makes use of the prior spatial region consistency and incorporates a customized prostate appearance model into the segmentation task. The formulated challenging combinatorial optimization problem is solved by means of convex relaxation, for which a novel spatially continuous max-flow model is introduced as the dual optimization formulation to the studied convex relaxed optimization problem with region consistency constraints. The proposed continuous max-flow model derives an efficient duality-based algorithm that enjoys numerical advantages and can be easily implemented on GPUs. The proposed approach was validated using 18 3D prostate T2w MR images with a body-coil and 25 images with an endo-rectal coil. Experimental results demonstrate that the proposed method is capable of efficiently and accurately extracting both the prostate zones: CG and PZ, and the whole prostate gland from the input 3D prostate MR images, with a mean Dice similarity coefficient (DSC) of 89.3±3.2% for the whole gland (WG), 82.2±3.0% for the CG, and 69.1±6.9% for the PZ in 3D body-coil MR images; 89.2±3.3% for the WG, 83.0±2.4% for the CG, and 70.0±6.5% for the PZ in 3D endo-rectal coil MR images. In addition, the experiments of intra- and inter-observer variability introduced by user initialization indicate a good reproducibility of the proposed approach in terms of volume difference (VD) and coefficient-of-variation (CV) of DSC.

Three-dimensional prostate segmentation using level set with shape constraint based on rotational slices for 3D end-firing TRUS guided biopsy.

PURPOSE: Prostate segmentation is an important step in the planning and treatment of 3D end-firing transrectal ultrasound (TRUS) guided prostate biopsy. In order to improve the accuracy and efficiency of prostate segmentation in 3D TRUS images, an improved level set method is incorporated into a rotational-slice-based 3D prostate segmentation to decrease the accumulated segmentation errors produced by the slice-by-slice segmentation method. METHODS: A 3D image is first resliced into 2D slices in a rotational manner in both the clockwise and counterclockwise directions. All slices intersect approximately along the rotational scanning axis and have an equal angular spacing. Six to eight boundary points are selected to initialize a level set function to extract the prostate contour within the first slice. The segmented contour is then propagated to the adjacent slice and is used as the initial contour for segmentation. This process is repeated until all slices are segmented. A modified distance regularization level set method is used to segment the prostate in all resliced 2D slices. In addition, shape-constraint and local-region-based energies are imposed to discourage the evolved level set function to leak in regions with weak edges or without edges. An anchor point based energy is used to promote the level set function to pass through the initial selected boundary points. The algorithm's performance was evaluated using distance- and volume-based metrics (sensitivity (Se), Dice similarity coefficient (DSC), mean absolute surface distance (MAD), maximum absolute surface distance (MAXD), and volume difference) by comparison with expert delineations. RESULTS: The validation results using thirty 3D patient images showed that the authors' method can obtain a DSC of 93.1% ± 1.6%, a sensitivity of 93.0% ± 2.0%, a MAD of 1.18 ± 0.36 mm, a MAXD of 3.44 ± 0.8 mm, and a volume difference of 2.6 ± 1.9 cm(3) for the entire prostate. A reproducibility experiment demonstrated that the proposed method yielded low intraobserver and interobserver variability in terms of DSC. The mean segmentation time of the authors' method for all patient 3D TRUS images was 55 ± 3.5 s, in addition to 30 ± 5 s for initialization. CONCLUSIONS: To address the challenges involved with slice-based 3D prostate segmentation, a level set based method is proposed in this paper. This method is especially developed for a 3D end-firing TRUS guided prostate biopsy system. The extensive experimental results demonstrate that the proposed method is accurate, robust, and computationally efficient.

Fast globally optimal segmentation of 3D prostate MRI with axial symmetry prior.

We propose a novel global optimization approach to segmenting a given 3D prostate T2w magnetic resonance (MR) image, which enforces the inherent axial symmetry of the prostate shape and simultaneously performs a sequence of 2D axial slice-wise segmentations with a global 3D coherence prior. We show that the proposed challenging combinatorial optimization problem can be solved globally and exactly by means of convex relaxation. With this regard, we introduce a novel coupled continuous max-flow model, which is dual to the studied convex relaxed optimization formulation and leads to an efficient multiplier augmented algorithm based on the modern convex optimization theory. Moreover, the new continuous max-flow based algorithm was implemented on GPUs to achieve a substantial improvement in computation. Experimental results using public and in-house datasets demonstrate great advantages of the proposed method in terms of both accuracy and efficiency.

Efficient 3D multi-region prostate MRI segmentation using dual optimization.

Efficient and accurate extraction of the prostate, in particular its clinically meaningful sub-regions from 3D MR images, is of great interest in image-guided prostate interventions and diagnosis of prostate cancer. In this work, we propose a novel multi-region segmentation approach to simultaneously locating the boundaries of the prostate and its two major sub-regions: the central gland and the peripheral zone. The proposed method utilizes the prior knowledge of the spatial region consistency and employs a customized prostate appearance model to simultaneously segment multiple clinically meaningful regions. We solve the resulted challenging combinatorial optimization problem by means of convex relaxation, for which we introduce a novel spatially continuous flow-maximization model and demonstrate its duality to the investigated convex relaxed optimization problem with the region consistency constraint. Moreover, the proposed continuous max-flow model naturally leads to a new and efficient continuous max-flow based algorithm, which enjoys great advantages in numerics and can be readily implemented on GPUs. Experiments using 15 T2-weighted 3D prostate MR images, by inter- and intra-operator variability, demonstrate the promising performance of the proposed approach.

Rotational-slice-Based prostate segmentation using level set with shape constraint for 3D end-firing TRUS guided biopsy.

Prostate segmentation in 3D ultrasound images is an important step in the planning and treatment of 3D end-firing transrectal ultrasound (TRUS) guided prostate biopsy. A semi-automatic prostate segmentation method is presented in this paper, which integrates a modified distance regularization level set formulation with shape constraint to a rotational-slice-based 3D prostate segmentation method. Its performance, using different metrics, has been evaluated on a set of twenty 3D patient prostate images by comparison with expert delineations. The volume overlap ratio of 93.39 +/- 1.26% and the mean absolute surface distance of 1.16 +/- 0.34 mm were found in the quantitative validation result.