Validation of a surface-based deformable MRI-3D ultrasound image registration algorithm toward clinical implementation for interstitial prostate brachytherapy.

PURPOSE: The purpose of this study was to evaluate and clinically implement a deformable surface-based magnetic resonance imaging (MRI) to three-dimensional ultrasound (US) image registration algorithm for prostate brachytherapy (BT) with the aim to reduce operator dependence and facilitate dose escalation to an MRI-defined target. METHODS AND MATERIALS: Our surface-based deformable image registration (DIR) algorithm first translates and scales to align the US- and MR-defined prostate surfaces, followed by deformation of the MR-defined prostate surface to match the US-defined prostate surface. The algorithm performance was assessed in a phantom using three deformation levels, followed by validation in three retrospective high-dose-rate BT clinical cases. For comparison, manual rigid registration and cognitive fusion by physician were also employed. Registration accuracy was assessed using the Dice similarity coefficient (DSC) and target registration error (TRE) for embedded spherical landmarks. The algorithm was then implemented intraoperatively in a prospective clinical case. RESULTS: In the phantom, our DIR algorithm demonstrated a mean DSC and TRE of 0.74 ± 0.08 and 0.94 ± 0.49 mm, respectively, significantly improving the performance compared to manual rigid registration with 0.64 ± 0.16 and 1.88 ± 1.24 mm, respectively. Clinical results demonstrated reduced variability compared to the current standard of cognitive fusion by physicians. CONCLUSIONS: We successfully validated a DIR algorithm allowing for translation of MR-defined target and organ-at-risk contours into the intraoperative environment. Prospective clinical implementation demonstrated the intraoperative feasibility of our algorithm, facilitating targeted biopsies and dose escalation to the MR-defined lesion. This method provides the potential to standardize the registration procedure between physicians, reducing operator dependence.

Automatic femoral articular cartilage segmentation using deep learning in three-dimensional ultrasound images of the knee.

Objective: This study aimed to develop a deep learning-based approach to automatically segment the femoral articular cartilage (FAC) in 3D ultrasound (US) images of the knee to increase time efficiency and decrease rater variability. Design: Our method involved deep learning predictions on 2DUS slices sampled in the transverse plane to view the cartilage of the femoral trochlea, followed by reconstruction into a 3D surface. A 2D U-Net was modified and trained using a dataset of 200 2DUS images resliced from 20 3DUS images. Segmentation accuracy was evaluated using a holdout dataset of 50 2DUS images resliced from 5 3DUS images. Absolute and signed error metrics were computed and FAC segmentation performance was compared between rater 1 and 2 manual segmentations. Results: Our U-Net-based algorithm performed with mean 3D DSC, recall, precision, VPD, MSD, and HD of 73.1 â€‹± â€‹3.9%, 74.8 â€‹± â€‹6.1%, 72.0 â€‹± â€‹6.3%, 10.4 â€‹± â€‹6.0%, 0.3 â€‹± â€‹0.1 â€‹mm, and 1.6 â€‹± â€‹0.7 â€‹mm, respectively. Compared to the individual 2D predictions, our algorithm demonstrated a decrease in performance after 3D reconstruction, but these differences were not found to be statistically significant. The percent difference between the manually segmented volumes of the 2 raters was 3.4%, and rater 2 demonstrated the largest VPD with 14.2 â€‹± â€‹11.4 â€‹mm3 compared to 10.4 â€‹± â€‹6.0 â€‹mm3 for rater 1. Conclusion: This study investigated the use of a modified U-Net algorithm to automatically segment the FAC in 3DUS knee images of healthy volunteers, demonstrating that this segmentation method would increase the efficiency of anterior femoral cartilage volume estimation and expedite the post-acquisition processing for 3D US images of the knee.

Effect of dataset size, image quality, and image type on deep learning-based automatic prostate segmentation in 3D ultrasound.

Three-dimensional (3D) transrectal ultrasound (TRUS) is utilized in prostate cancer diagnosis and treatment, necessitating time-consuming manual prostate segmentation. We have previously developed an automatic 3D prostate segmentation algorithm involving deep learning prediction on radially sampled 2D images followed by 3D reconstruction, trained on a large, clinically diverse dataset with variable image quality. As large clinical datasets are rare, widespread adoption of automatic segmentation could be facilitated with efficient 2D-based approaches and the development of an image quality grading method. The complete training dataset of 6761 2D images, resliced from 206 3D TRUS volumes acquired using end-fire and side-fire acquisition methods, was split to train two separate networks using either end-fire or side-fire images. Split datasets were reduced to 1000, 500, 250, and 100 2D images. For deep learning prediction, modified U-Net and U-Net++ architectures were implemented and compared using an unseen test dataset of 40 3D TRUS volumes. A 3D TRUS image quality grading scale with three factors (acquisition quality, artifact severity, and boundary visibility) was developed to assess the impact on segmentation performance. For the complete training dataset, U-Net and U-Net++ networks demonstrated equivalent performance, but when trained using split end-fire/side-fire datasets, U-Net++ significantly outperformed the U-Net. Compared to the complete training datasets, U-Net++ trained using reduced-size end-fire and side-fire datasets demonstrated equivalent performance down to 500 training images. For this dataset, image quality had no impact on segmentation performance for end-fire images but did have a significant effect for side-fire images, with boundary visibility having the largest impact. Our algorithm provided fast (<1.5 s) and accurate 3D segmentations across clinically diverse images, demonstrating generalizability and efficiency when employed on smaller datasets, supporting the potential for widespread use, even when data is scarce. The development of an image quality grading scale provides a quantitative tool for assessing segmentation performance.

Automatic prostate segmentation using deep learning on clinically diverse 3D transrectal ultrasound images.

PURPOSE: Needle-based procedures for diagnosing and treating prostate cancer, such as biopsy and brachytherapy, have incorporated three-dimensional (3D) transrectal ultrasound (TRUS) imaging to improve needle guidance. Using these images effectively typically requires the physician to manually segment the prostate to define the margins used for accurate registration, targeting, and other guidance techniques. However, manual prostate segmentation is a time-consuming and difficult intraoperative process, often occurring while the patient is under sedation (biopsy) or anesthetic (brachytherapy). Minimizing procedure time with a 3D TRUS prostate segmentation method could provide physicians with a quick and accurate prostate segmentation, and allow for an efficient workflow with improved patient throughput to enable faster patient access to care. The purpose of this study was to develop a supervised deep learning-based method to segment the prostate in 3D TRUS images from different facilities, generated using multiple acquisition methods and commercial ultrasound machine models to create a generalizable algorithm for needle-based prostate cancer procedures. METHODS: Our proposed method for 3D segmentation involved prediction on two-dimensional (2D) slices sampled radially around the approximate central axis of the prostate, followed by reconstruction into a 3D surface. A 2D U-Net was modified, trained, and validated using images from 84 end-fire and 122 side-fire 3D TRUS images acquired during clinical biopsies and brachytherapy procedures. Modifications to the expansion section of the standard U-Net included the addition of 50% dropouts and the use of transpose convolutions instead of standard upsampling followed by convolution to reduce overfitting and improve performance, respectively. Manual contours provided the annotations needed for the training, validation, and testing datasets, with the testing dataset consisting of 20 end-fire and 20 side-fire unseen 3D TRUS images. Since predicting with 2D images has the potential to lose spatial and structural information, comparisons to 3D reconstruction and optimized 3D networks including 3D V-Net, Dense V-Net, and High-resolution 3D-Net were performed following an investigation into different loss functions. An extended selection of absolute and signed error metrics were computed, including pixel map comparisons [dice similarity coefficient (DSC), recall, and precision], volume percent differences (VPD), mean surface distance (MSD), and Hausdorff distance (HD), to assess 3D segmentation accuracy. RESULTS: Overall, our proposed reconstructed modified U-Net performed with a median [first quartile, third quartile] absolute DSC, recall, precision, VPD, MSD, and HD of 94.1 [92.6, 94.9]%, 96.0 [93.1, 98.5]%, 93.2 [88.8, 95.4]%, 5.78 [2.49, 11.50]%, 0.89 [0.73, 1.09] mm, and 2.89 [2.37, 4.35] mm, respectively. When compared to the best-performing optimized 3D network (i.e., 3D V-Net with a Dice plus cross-entropy loss function), our proposed method performed with a significant improvement across nearly all metrics. A computation time <0.7 s per prostate was observed, which is a sufficiently short segmentation time for intraoperative implementation. CONCLUSIONS: Our proposed algorithm was able to provide a fast and accurate 3D segmentation across variable 3D TRUS prostate images, enabling a generalizable intraoperative solution for needle-based prostate cancer procedures. This method has the potential to decrease procedure times, supporting the increasing interest in needle-based 3D TRUS approaches.

Deep learning segmentation of general interventional tools in two-dimensional ultrasound images.

PURPOSE: Many interventional procedures require the precise placement of needles or therapy applicators (tools) to correctly achieve planned targets for optimal diagnosis or treatment of cancer, typically leveraging the temporal resolution of ultrasound (US) to provide real-time feedback. Identifying tools in two-dimensional (2D) images can often be time-consuming with the precise position difficult to distinguish. We have developed and implemented a deep learning method to segment tools in 2D US images in near real-time for multiple anatomical sites, despite the widely varying appearances across interventional applications. METHODS: A U-Net architecture with a Dice similarity coefficient (DSC) loss function was used to perform segmentation on input images resized to 256 × 256 pixels. The U-Net was modified by adding 50% dropouts and the use of transpose convolutions in the decoder section of the network. The proposed approach was trained with 917 images and manual segmentations from prostate/gynecologic brachytherapy, liver ablation, and kidney biopsy/ablation procedures, as well as phantom experiments. Real-time data augmentation was applied to improve generalizability and doubled the dataset for each epoch. Postprocessing to identify the tool tip and trajectory was performed using two different approaches, comparing the largest island with a linear fit to random sample consensus (RANSAC) fitting. RESULTS: Comparing predictions from 315 unseen test images to manual segmentations, the overall median [first quartile, third quartile] tip error, angular error, and DSC were 3.5 [1.3, 13.5] mm, 0.8 [0.3, 1.7]°, and 73.3 [56.2, 82.3]%, respectively, following RANSAC postprocessing. The predictions with the lowest median tip and angular errors were observed in the gynecologic images (median tip error: 0.3 mm; median angular error: 0.4°) with the highest errors in the kidney images (median tip error: 10.1 mm; median angular error: 2.9°). The performance on the kidney images was likely due to a reduction in acoustic signal associated with oblique insertions relative to the US probe and the increased number of anatomical interfaces with similar echogenicity. Unprocessed segmentations were performed with a mean time of approximately 50 ms per image. CONCLUSIONS: We have demonstrated that our proposed approach can accurately segment tools in 2D US images from multiple anatomical locations and a variety of clinical interventional procedures in near real-time, providing the potential to improve image guidance during a broad range of diagnostic and therapeutic cancer interventions.

Fast prostate segmentation in 3D TRUS images based on continuity constraint using an autoregressive model.

In this article a new slice-based 3D prostate segmentation method based on a continuity constraint, implemented as an autoregressive (AR) model is described. In order to decrease the propagated segmentation error produced by the slice-based 3D segmentation method, a continuity constraint was imposed in the prostate segmentation algorithm. A 3D ultrasound image was segmented using the slice-based segmentation method. Then, a cross-sectional profile of the resulting contours was obtained by intersecting the 2D segmented contours with a coronal plane passing through the midpoint of the manually identified rotational axis, which is considered to be the approximate center of the prostate. On the coronal cross-sectional plane, these intersections form a set of radial lines directed from the center of the prostate. The lengths of these radial lines were smoothed using an AR model. Slice-based 3D segmentations were performed in the clockwise and in the anticlockwise directions, where clockwise and anticlockwise are defined with respect to the propagation directions on the coronal view. This resulted in two different segmentations for each 2D slice. For each pair of unmatched segments, in which the distance between the contour generated clockwise and that generated anticlockwise was greater than 4 mm, a method was used to select the optimal contour. Experiments performed using 3D prostate ultrasound images of nine patients demonstrated that the proposed method produced accurate 3D prostate boundaries without manual editing. The average distance between the proposed method and manual segmentation was 1.29 mm. The average intraobserver coefficient of variation (i.e., the standard deviation divided by the average volume) of the boundaries segmented by the proposed method was 1.6%. The average segmentation time of a 352 x 379 x 704 image on a Pentium IV 2.8 GHz PC was 10 s.

Evaluation of inter-session 3D-TRUS to 3D-TRUS image registration for repeat prostate biopsies.

To ensure accurate targeting and repeatability, 3D TRUS-guided biopsies require registration to determine coordinate transformations to (1) incorporate pre-procedure biopsy plans and (2) compensate for inter-session prostate motion and deformation between repeat biopsy sessions. We evaluated prostate surface- and image-based 3D-to-3D TRUS registration by measuring the TRE of manually marked, corresponding, intrinsic fiducials in the whole gland and peripheral zone, and also evaluated the error anisotropy. The image-based rigid and non-rigid methods yielded the best results with mean TREs of 2.26 mm and 1.96 mm, respectively. These results compare favorably with a clinical need for an error of less than 2.5 mm.

3D ultrasound analysis of carotid plaque volume and surface morphology.

Morphological characterization of carotid plaques has been used for risk stratification and evaluation of response to therapy, evaluation of new risk factors, genetic research, and for quantifying effects of new anti-atherosclerotic therapies. We developed a 3D US system that allows detailed studies of carotid plaques in 3D. Our software includes 3D reconstruction, viewing, manual and semi-automated segmentation of carotid plaques, and surface morphology analysis to be used for quantitative tracking of plaque changes. We evaluated our plaque quantification software by examining plaque volume measurement accuracy, variability, and plaque surface morphology. We used vascular test phantoms to study segmentation accuracy, and used 48 3D US carotid plaques of patients ranging in size from 13.2 mm(3) to 544.0 mm(3) to study plaque volume measurement variability. We compared results from the semi-automated plaque measurements to the results obtained from manual measurements, which were used as the "gold" standard. We developed a surface plaque morphology quantification technique based on the segmented plaque surface curvature and used it to analyze plaques. Accuracy of plaque volume measurements for the simulated plaques ranged from 4.2% to 1.5% for volumes ranging from 68.5 mm(3) to 286 mm(3). The variability study showed that coefficients of variation in the measurement of plaque volume decreased with increasing plaque size for both inter- (90.8-3.9%) and intra-observer (70.2-3.1%) measurements over the plaque sizes measured. Surface morphology analysis showed that 1 mm ulceration could be quantified and monitored for changes over time. The automated plaque quantification approach showed a little higher intra-observer variability than the manual technique, and its performance was better for segmenting the wall than the lumen. Our results indicate that our approach is sensitive tool and can be used in studies of plaque progression and regression as it relates to atherosclerosis treatment effects and can be used effectively in longitudinal studies for direct measurement carotid plaque volume.

Slice-Based Prostate Segmentation in 3D US Images Using Continuity Constraint.

In the diagnosis and therapy of prostate cancer, it is critical to measure the volume of the prostate and locate its boundary. Three-dimensional transrectal ultrasound (3D TRUS) imaging has been demonstrated to be a useful technique to perform such a task. Due to image speckle as well as low contrast in ultrasound images, segmentation of the prostate in 3D US images is challenging. In this paper, we report on the development of an improved slice-based 3D prostate segmentation method. First, we imposed a continuity constraint for the end points of the prostate boundaries in a cross-sectional plane so that a smooth prostate boundary in 2D is obtained. Then, in each 2D slice, we inserted the end points into the vertex list of the initial contour to obtain a new contour, which forces the evolving contour to be driven to the boundary of the prostate. Evaluation demonstrated that our method could segment the prostate in 3D TRUS images more quickly and accurately.