Deep learning for automated contouring of neurovascular structures on magnetic resonance imaging for prostate cancer patients.

Background and purpose: Manual contouring of neurovascular structures on prostate magnetic resonance imaging (MRI) is labor-intensive and prone to considerable interrater disagreement. Our aim is to contour neurovascular structures automatically on prostate MRI by deep learning (DL) to improve workflow and interrater agreement. Materials and methods: Segmentation of neurovascular structures was performed on pre-treatment 3.0 T MRI data of 131 prostate cancer patients (training [n = 105] and testing [n = 26]). The neurovascular structures include the penile bulb (PB), corpora cavernosa (CCs), internal pudendal arteries (IPAs), and neurovascular bundles (NVBs). Two DL networks, nnU-Net and DeepMedic, were trained for auto-contouring on prostate MRI and evaluated using volumetric Dice similarity coefficient (DSC), mean surface distances (MSD), Hausdorff distances, and surface DSC. Three radiation oncologists evaluated the DL-generated contours and performed corrections when necessary. Interrater agreement was assessed and the time required for manual correction was recorded. Results: nnU-Net achieved a median DSC of 0.92 (IQR: 0.90-0.93) for the PB, 0.90 (IQR: 0.86-0.92) for the CCs, 0.79 (IQR: 0.77-0.83) for the IPAs, and 0.77 (IQR: 0.72-0.81) for the NVBs, which outperformed DeepMedic for each structure (p < 0.03). nnU-Net showed a median MSD of 0.24 mm for the IPAs and 0.71 mm for the NVBs. The median interrater DSC ranged from 0.93 to 1.00, with the majority of cases (68.9%) requiring manual correction times under two minutes. Conclusions: DL enables reliable auto-contouring of neurovascular structures on pre-treatment MRI data, easing the clinical workflow in neurovascular-sparing MR-guided radiotherapy.

Clinical utility of convolutional neural networks for treatment planning in radiotherapy for spinal metastases.

BACKGROUND AND PURPOSE: Spine delineation is essential for high quality radiotherapy treatment planning of spinal metastases. However, manual delineation is time-consuming and prone to interobserver variability. Automatic spine delineation, especially using deep learning, has shown promising results in healthy subjects. We aimed to evaluate the clinical utility of deep learning-based vertebral body delineations for radiotherapy planning purposes. MATERIALS AND METHODS: A multi-scale convolutional neural network (CNN) was used for automatic segmentation and labeling. Two approaches were tested: the combined approach using one CNN for both segmentation and labeling, and the sequential approach using separate CNN's for these tasks. Training and internal validation data included 580 vertebrae, external validation data included 202 vertebrae. For quantitative assessment, Dice similarity coefficient (DSC) and Hausdorff distance (HD) were used. Axial slices from external images were presented to radiation oncologists for subjective evaluation. RESULTS: Both approaches performed comparably during the internal validation (DSC: 96.7%, HD: 3.6 mm), but the sequential approach proved more robust during the external validation (DSC: 94.5% vs 94.4%, p < 0.001, HD: 4.5 vs 7.1 mm, p < 0.001). Subsequently, subjective evaluation of this sequential approach showed that experienced radiation oncologists could distinguish automatic from human-made contours in 63% of cases. They rated automatic contours clinically acceptable in 77% of cases, compared to 88% of human-made contours. CONCLUSION: We present a feasible approach for automatic vertebral body delineation using two variants of a multi-scale CNN. This approach generates high quality automatic delineations, which can save time in a clinical radiotherapy workflow.

Deep learning-based synthetic CT generation for paediatric brain MR-only photon and proton radiotherapy.

BACKGROUND AND PURPOSE: To enable accurate magnetic resonance imaging (MRI)-based dose calculations, synthetic computed tomography (sCT) images need to be generated. We aim at assessing the feasibility of dose calculations from MRI acquired with a heterogeneous set of imaging protocol for paediatric patients affected by brain tumours. MATERIALS AND METHODS: Sixty paediatric patients undergoing brain radiotherapy were included. MR imaging protocols varied among patients, and data heterogeneity was maintained in train/validation/test sets. Three 2D conditional generative adversarial networks (cGANs) were trained to generate sCT from T1-weighted MRI, considering the three orthogonal planes and its combination (multi-plane sCT). For each patient, median and standard deviation (σ) of the three views were calculated, obtaining a combined sCT and a proxy for uncertainty map, respectively. The sCTs were evaluated against the planning CT in terms of image similarity and accuracy for photon and proton dose calculations. RESULTS: A mean absolute error of 61 ± 14 HU (mean±1σ) was obtained in the intersection of the body contours between CT and sCT. The combined multi-plane sCTs performed better than sCTs from any single plane. Uncertainty maps highlighted that multi-plane sCTs differed at the body contours and air cavities. A dose difference of -0.1 ± 0.3% and 0.1 ± 0.4% was obtained on the D > 90% of the prescribed dose and mean γ2%,2mm pass-rate of 99.5 ± 0.8% and 99.2 ± 1.1% for photon and proton planning, respectively. CONCLUSION: Accurate MR-based dose calculation using a combination of three orthogonal planes for sCT generation is feasible for paediatric brain cancer patients, even when training on a heterogeneous dataset.

Clinical implementation of MRI-based organs-at-risk auto-segmentation with convolutional networks for prostate radiotherapy.

BACKGROUND: Structure delineation is a necessary, yet time-consuming manual procedure in radiotherapy. Recently, convolutional neural networks have been proposed to speed-up and automatise this procedure, obtaining promising results. With the advent of magnetic resonance imaging (MRI)-guided radiotherapy, MR-based segmentation is becoming increasingly relevant. However, the majority of the studies investigated automatic contouring based on computed tomography (CT). PURPOSE: In this study, we investigate the feasibility of clinical use of deep learning-based automatic OARs delineation on MRI. MATERIALS AND METHODS: We included 150 patients diagnosed with prostate cancer who underwent MR-only radiotherapy. A three-dimensional (3D) T1-weighted dual spoiled gradient-recalled echo sequence was acquired with 3T MRI for the generation of the synthetic-CT. The first 48 patients were included in a feasibility study training two 3D convolutional networks called DeepMedic and dense V-net (dV-net) to segment bladder, rectum and femurs. A research version of an atlas-based software was considered for comparison. Dice similarity coefficient, 95% Hausdorff distances (HD95), and mean distances were calculated against clinical delineations. For eight patients, an expert RTT scored the quality of the contouring for all the three methods. A choice among the three approaches was made, and the chosen approach was retrained on 97 patients and implemented for automatic use in the clinical workflow. For the successive 53 patients, Dice, HD95 and mean distances were calculated against the clinically used delineations. RESULTS: DeepMedic, dV-net and the atlas-based software generated contours in 60 s, 4 s and 10-15 min, respectively. Performances were higher for both the networks compared to the atlas-based software. The qualitative analysis demonstrated that delineation from DeepMedic required fewer adaptations, followed by dV-net and the atlas-based software. DeepMedic was clinically implemented. After retraining DeepMedic and testing on the successive patients, the performances slightly improved. CONCLUSION: High conformality for OARs delineation was achieved with two in-house trained networks, obtaining a significant speed-up of the delineation procedure. Comparison of different approaches has been performed leading to the succesful adoption of one of the neural networks, DeepMedic, in the clinical workflow. DeepMedic maintained in a clinical setting the accuracy obtained in the feasibility study.

Deep learning-based reconstruction of in vivo pelvis conductivity with a 3D patch-based convolutional neural network trained on simulated MR data.

PURPOSE: To demonstrate that mapping pelvis conductivity at 3T with deep learning (DL) is feasible. METHODS: 210 dielectric pelvic models were generated based on CT scans of 42 cervical cancer patients. For all dielectric models, electromagnetic and MR simulations with realistic accuracy and precision were performed to obtain B1+ and transceive phase (ϕ± ). Simulated B1+ and ϕ± served as input to a 3D patch-based convolutional neural network, which was trained in a supervised fashion to retrieve the conductivity. The same network architecture was retrained using only ϕ± in input. Both network configurations were tested on simulated MR data and their conductivity reconstruction accuracy and precision were assessed. Furthermore, both network configurations were used to reconstruct conductivity maps from a healthy volunteer and two cervical cancer patients. DL-based conductivity was compared in vivo and in silico to Helmholtz-based (H-EPT) conductivity. RESULTS: Conductivity maps obtained from both network configurations were comparable. Accuracy was assessed by mean error (ME) with respect to ground truth conductivity. On average, ME < 0.1 Sm-1 for all tissues. Maximum MEs were 0.2 Sm-1 for muscle and tumour, and 0.4 Sm-1 for bladder. Precision was indicated with the difference between 90th and 10th conductivity percentiles, and was below 0.1 Sm-1 for fat, bone and muscle, 0.2 Sm-1 for tumour and 0.3 Sm-1 for bladder. In vivo, DL-based conductivity had median values in agreement with H-EPT values, but a higher precision. CONCLUSION: Anatomically detailed, noise-robust 3D conductivity maps with good sensitivity to tissue conductivity variations were reconstructed in the pelvis with DL.

A single neural network for cone-beam computed tomography-based radiotherapy of head-and-neck, lung and breast cancer.

Background and purpose Adaptive radiotherapy based on cone-beam computed tomography (CBCT) requires high CT number accuracy to ensure accurate dose calculations. Recently, deep learning has been proposed for fast CBCT artefact corrections on single anatomical sites. This study investigated the feasibility of applying a single convolutional network to facilitate dose calculation based on CBCT for head-and-neck, lung and breast cancer patients. Materials and Methods Ninety-nine patients diagnosed with head-and-neck, lung or breast cancer undergoing radiotherapy with CBCT-based position verification were included in this study. The CBCTs were registered to planning CT according to clinical procedures. Three cycle-consistent generative adversarial networks (cycle-GANs) were trained in an unpaired manner on 15 patients per anatomical site generating synthetic-CTs (sCTs). Another network was trained with all the anatomical sites together. Performances of all four networks were compared and evaluated for image similarity against rescan CT (rCT). Clinical plans were recalculated on rCT and sCT and analysed through voxel-based dose differences and γ -analysis. Results A sCT was generated in 10 s. Image similarity was comparable between models trained on different anatomical sites and a single model for all sites. Mean dose differences < 0.5 % were obtained in high-dose regions. Mean gamma (3%, 3 mm) pass-rates > 95 % were achieved for all sites. Conclusion Cycle-GAN reduced CBCT artefacts and increased similarity to CT, enabling sCT-based dose calculations. A single network achieved CBCT-based dose calculation generating synthetic CT for head-and-neck, lung, and breast cancer patients with similar performance to a network specifically trained for each anatomical site.

CBCT correction using a cycle-consistent generative adversarial network and unpaired training to enable photon and proton dose calculation.

In presence of inter-fractional anatomical changes, clinical benefits are anticipated from image-guided adaptive radiotherapy. Nowadays, cone-beam CT (CBCT) imaging is mostly utilized during pre-treatment imaging for position verification. Due to various artifacts, image quality is typically not sufficient for photon or proton dose calculation, thus demanding accurate CBCT correction, as potentially provided by deep learning techniques. This work aimed at investigating the feasibility of utilizing a cycle-consistent generative adversarial network (cycleGAN) for prostate CBCT correction using unpaired training. Thirty-three patients were included. The network was trained to translate uncorrected, original CBCT images (CBCTorg) into planning CT equivalent images (CBCTcycleGAN). HU accuracy was determined by comparison to a previously validated CBCT correction technique (CBCTcor). Dosimetric accuracy was inferred for volumetric-modulated arc photon therapy (VMAT) and opposing single-field uniform dose (OSFUD) proton plans, optimized on CBCTcor and recalculated on CBCTcycleGAN. Single-sided SFUD proton plans were utilized to assess proton range accuracy. The mean HU error of CBCTcycleGAN with respect to CBCTcor decreased from 24 HU for CBCTorg to -6 HU. Dose calculation accuracy was high for VMAT, with average pass-rates of 100%/89% for a 2%/1% dose difference criterion. For proton OSFUD plans, the average pass-rate for a 2% dose difference criterion was 80%. Using a (2%, 2 mm) gamma criterion, the pass-rate was 96%. 93% of all analyzed SFUD profiles had a range agreement better than 3 mm. CBCT correction time was reduced from 6-10 min for CBCTcor to 10 s for CBCTcycleGAN. Our study demonstrated the feasibility of utilizing a cycleGAN for CBCT correction, achieving high dose calculation accuracy for VMAT. For proton therapy, further improvements may be required. Due to unpaired training, the approach does not rely on anatomically consistent training data or potentially inaccurate deformable image registration. The substantial speed-up for CBCT correction renders the method particularly interesting for adaptive radiotherapy.

Dosimetric evaluation of synthetic CT for head and neck radiotherapy generated by a patch-based three-dimensional convolutional neural network.

PURPOSE: To develop and evaluate a patch-based convolutional neural network (CNN) to generate synthetic computed tomography (sCT) images for magnetic resonance (MR)-only workflow for radiotherapy of head and neck tumors. A patch-based deep learning method was chosen to improve robustness to abnormal anatomies caused by large tumors, surgical excisions, or dental artifacts. In this study, we evaluate whether the generated sCT images enable accurate MR-based dose calculations in the head and neck region. METHODS: We conducted a retrospective study on 34 patients with head and neck cancer who underwent both CT and MR imaging for radiotherapy treatment planning. To generate the sCTs, a large field-of-view T2-weighted Turbo Spin Echo MR sequence was used from the clinical protocol for multiple types of head and neck tumors. To align images as well as possible on a voxel-wise level, CT scans were nonrigidly registered to the MR (CTreg ). The CNN was based on a U-net architecture and consisted of 14 layers with 3 × 3 × 3 filters. Patches of 48 × 48 × 48 were randomly extracted and fed into the training. sCTs were created for all patients using threefold cross validation. For each patient, the clinical CT-based treatment plan was recalculated on sCT using Monaco TPS (Elekta). We evaluated mean absolute error (MAE) and mean error (ME) within the body contours and dice scores in air and bone mask. Also, dose differences and gamma pass rates between CT- and sCT-based plans inside the body contours were calculated. RESULTS: sCT generation took 4 min per patient. The MAE over the patient population of the sCT within the intersection of body contours was 75 ± 9 Hounsfield Units (HU) (±1 SD), and the ME was 9 ± 11 HU. Dice scores of the air and bone masks (CTreg vs sCT) were 0.79 ± 0.08 and 0.70 ± 0.07, respectively. Dosimetric analysis showed mean deviations of -0.03% ± 0.05% for dose within the body contours and -0.07% ± 0.22% inside the >90% dose volume. Dental artifacts obscuring the CT could be circumvented in the sCT by the CNN-based approach in combination with Turbo Spin Echo (TSE) magnetic resonance imaging (MRI) sequence that typically is less prone to susceptibility artifacts. CONCLUSIONS: The presented CNN generated sCTs from conventional MR images without adding scan time to the acquisition. Dosimetric evaluation suggests that dose calculations performed on the sCTs are accurate, and can therefore be used for MR-only radiotherapy treatment planning of the head and neck.

MR-Only Brain Radiation Therapy: Dosimetric Evaluation of Synthetic CTs Generated by a Dilated Convolutional Neural Network.

PURPOSE: This work aims to facilitate a fast magnetic resonance (MR)-only workflow for radiation therapy of intracranial tumors. Here, we evaluate whether synthetic computed tomography (sCT) images generated with a dilated convolutional neural network (CNN) enable accurate MR-based dose calculations in the brain. METHODS AND MATERIALS: We conducted a retrospective study of 52 patients with brain tumors who underwent both computed tomography (CT) and MR imaging for radiation therapy treatment planning. To generate the sCTs, a T1-weighted gradient echo MR sequence was selected from the clinical protocol for multiple types of brain tumors. sCTs were created for all 52 patients with a dilated CNN using 2-fold cross validation; in each fold, 26 patients were used for training and the remaining 26 patients were used for evaluation. For each patient, the clinical CT-based treatment plan was recalculated on sCT. We calculated dose differences and gamma pass rates between CT- and sCT-based plans inside body and planning target volume. Geometric fidelity of the sCT and differences in beam depth and equivalent path length were assessed between both treatment plans. RESULTS: sCT generation took 1 minute per patient. Over the patient population, the mean absolute error of the sCT within the intersection of body contours was 67 ± 11 HU (±1 standard deviation [SD], range: 51-117 HU), and the mean error was 13 ± 9 HU (±1 SD, range: -2 to 38 HU). Dosimetric analysis showed mean deviations of 0.00% ± 0.02% (±1 SD, range: -0.05 to 0.03) for dose within the body contours and -0.13% ± 0.39% (±1 SD, range: -1.43 to 0.80) inside the planning target volume. Mean γ1mm/1% was 98.8% ± 2.2% for doses >50% of the prescribed dose. CONCLUSIONS: The presented dilated CNN generated sCTs from conventional MR images without adding scan time to the acquisition. Dosimetric evaluation suggests that dose calculations performed on the sCTs are accurate and can therefore be used for MR-only intracranial radiation therapy treatment planning.

Dose evaluation of fast synthetic-CT generation using a generative adversarial network for general pelvis MR-only radiotherapy.

To enable magnetic resonance (MR)-only radiotherapy and facilitate modelling of radiation attenuation in humans, synthetic CT (sCT) images need to be generated. Considering the application of MR-guided radiotherapy and online adaptive replanning, sCT generation should occur within minutes. This work aims at assessing whether an existing deep learning network can rapidly generate sCT images for accurate MR-based dose calculations in the entire pelvis. A study was conducted on data of 91 patients with prostate (59), rectal (18) and cervical (14) cancer who underwent external beam radiotherapy acquiring both CT and MRI for patients' simulation. Dixon reconstructed water, fat and in-phase images obtained from a conventional dual gradient-recalled echo sequence were used to generate sCT images. A conditional generative adversarial network (cGAN) was trained in a paired fashion on 2D transverse slices of 32 prostate cancer patients. The trained network was tested on the remaining patients to generate sCT images. For 30 patients in the test set, dose recalculations of the clinical plan were performed on sCT images. Dose distributions were evaluated comparing voxel-based dose differences, gamma and dose-volume histogram (DVH) analysis. The sCT generation required 5.6 s and 21 s for a single patient volume on a GPU and CPU, respectively. On average, sCT images resulted in a higher dose to the target of maximum 0.3%. The average gamma pass rates using the 3%, 3 mm and 2%, 2 mm criteria were above 97 and 91%, respectively, for all volumes of interests considered. All DVH points calculated on sCT differed less than ±2.5% from the corresponding points on CT. Results suggest that accurate MR-based dose calculation using sCT images generated with a cGAN trained on prostate cancer patients is feasible for the entire pelvis. The sCT generation was sufficiently fast for integration in an MR-guided radiotherapy workflow.