Geometric evaluation and quantifying dosimetric impact of diverse deformable image registration algorithms on abdomen images with biomechanically modeled deformations.

PURPOSE: Deformable image registration (DIR) has been increasingly used in radiation therapy (RT). The accuracy of DIR algorithms and how it impacts on the RT plan dosimetrically were examined in our study for abdominal sites using biomechanically modeled deformations. METHODS: Five pancreatic cancer patients were enrolled in this study. Following the guidelines of AAPM TG-132, a patient-specific quality assurance (QA) workflow was developed to evaluate DIR for the abdomen using the TG-132 recommended virtual simulation software ImSimQA (Shrewsbury, UK). First, the planning CT was deformed to simulate respiratory motion using the embedded biomechanical model in ImSimQA. Additionally, 5 mm translational motion was added to the stomach, duodenum, and small bowel. The original planning CT and the deformed CT were then imported into Eclipse and MIM to perform DIR. The output displacement vector fields (DVFs) were compared with the ground truth from ImSimQA. Furthermore, the original treatment plan was recalculated on the ground-truth deformed CT and the deformed CT (with Eclipse and MIM DVF). The dose errors were calculated on a voxel-to-voxel basis. RESULTS: Data analysis comparing DVF from Eclipse versus MIM show the average mean DVF magnitude errors of 2.8 ± 1.0  versus 1.1 ± 0.7 mm for stomach and duodenum, 5.2 ± 4.0  versus 2.5 ± 1.0 mm for small bowel, and 4.8 ± 4.1  versus 2.7 ± 1.1 mm for the gross tumor volume (GTV), respectively, across all patients. The mean dose error on stomach+duodenum and small bowel were 2.3 ± 0.6% for Eclipse, and 1.0 ± 0.3% for MIM. As the DIR magnitude error increases, the dose error range increase, for both Eclipse and MIM. CONCLUSION: In our study, an initial assessment was conducted to evaluate the accuracy of DIR and its dosimetric impact on radiotherapy. A patient-specific DIR QA workflow was developed for pancreatic cancer patients. This workflow exhibits promising potential for future implementation as a clinical workflow.

Multi-modality imaging parameters that predict rapid tumor regression in head and neck radiotherapy.

Background and purpose: Volume regression during radiotherapy can indicate patient-specific treatment response. We aimed to identify pre-treatment multimodality imaging (MMI) metrics from positron emission tomography (PET), magnetic resonance imaging (MRI), and computed tomography (CT) that predict rapid tumor regression during radiotherapy in human papilloma virus (HPV) associated oropharyngeal carcinoma. Materials and methods: Pre-treatment FDG PET-CT, diffusion-weighted MRI (DW-MRI), and intra-treatment (at 1, 2, and 3 weeks) MRI were acquired in 72 patients undergoing chemoradiation therapy for HPV+ oropharyngeal carcinoma. Nodal gross tumor volumes were delineated on longitudinal images to measure intra-treatment volume changes. Pre-treatment PET standardized uptake value (SUV), CT Hounsfield Unit (HU), and non-gaussian intravoxel incoherent motion DW-MRI metrics were computed and correlated with volume changes. Intercorrelations between MMI metrics were also assessed using network analysis. Validation was carried out on a separate cohort (N = 64) for FDG PET-CT. Results: Significant correlations with volume loss were observed for baseline FDG SUVmean (Spearman ρ = 0.46, p < 0.001), CT HUmean (ρ = 0.38, p = 0.001), and DW-MRI diffusion coefficient, Dmean (ρ = -0.39, p < 0.001). Network analysis revealed 41 intercorrelations between MMI and volume loss metrics, but SUVmean remained a statistically significant predictor of volume loss in multivariate linear regression (p = 0.01). Significant correlations were also observed for SUVmean in the validation cohort in both primary (ρ = 0.30, p = 0.02) and nodal (ρ = 0.31, p = 0.02) tumors. Conclusions: Multiple pre-treatment imaging metrics were correlated with rapid nodal gross tumor volume loss during radiotherapy. FDG-PET SUV in particular exhibited significant correlations with volume regression across the two cohorts and in multivariate analysis.

Automated planning of stereotactic spine re-irradiation using cumulative dose limits.

Background and Purpose: The lack of dedicated tools in commercial planning systems currently restricts efficient review and planning for re-irradiation. The aim of this study was to develop an automated re-irradiation planning framework based on cumulative doses. Materials and Methods: We performed a retrospective study of 14 patients who received spine SBRT re-irradiation near a previously irradiated treatment site. A fully-automated workflow, DART (Dose Accumulation-based Re-irradiation Tool), was implemented within Eclipse by leveraging a combination of a dose accumulation script and a proprietary automated optimization algorithm. First, we converted the prior treatment dose into equivalent dose in 2 Gy fractions (EQD2) and mapped it to the current anatomy, utilizing deformable image registration. Subsequently, the intersection of EQD2 isodose lines with relevant organs at risk defines a series of optimization structures. During plan optimization, the residual allowable dose at a specified tissue tolerance was treated as a hard constraint. Results: All DART plans met institutional physical and cumulative constraints and passed plan checks by qualified medical physicists. DART demonstrated significant improvements in target coverage over clinical plans, with an average increase in PTV D99% and V100% of 2.3 Gy [range -0.3-7.7 Gy] and 3.4 % [range -0.4 %-7.6 %] (p < 0.01, paired t-test), respectively. Moreover, high-dose spillage (>105 %) outside the PTV was reduced by up to 7 cm3. The homogeneity index for DART plans was improved by 19 % (p < 0.001). Conclusions: DART provides a powerful framework to achieve more tailored re-irradiation plans by accounting for dose distributions from the previous treatments. The superior plan quality could improve the therapeutic ratio for re-irradiation patients.

Clinical Experience With an Offline Adaptive Radiation Therapy Head and Neck Program: Dosimetric Benefits and Opportunities for Patient Selection.

PURPOSE: The objective of this study was to develop a linear accelerator (LINAC)-based adaptive radiation therapy (ART) workflow for the head and neck that is informed by automated image tracking to identify major anatomic changes warranting adaptation. In this study, we report our initial clinical experience with the program and an investigation into potential trigger signals for ART. METHODS AND MATERIALS: Offline ART was systematically performed on patients receiving radiation therapy for head and neck cancer on C-arm LINACs. Adaptations were performed at a single time point during treatment with resimulation approximately 3 weeks into treatment. Throughout treatment, all patients were tracked using an automated image tracking system called the Automated Watchdog for Adaptive Radiotherapy Environment (AWARE). AWARE measures volumetric changes in gross tumor volumes (GTVs) and selected normal tissues via cone beam computed tomography scans and deformable registration. The benefit of ART was determined by comparing adaptive plan dosimetry and normal tissue complication probabilities against the initial plans recalculated on resimulation computed tomography scans. Dosimetric differences were then correlated with AWARE-measured volume changes to identify patient-specific triggers for ART. Candidate trigger variables were evaluated using receiver operator characteristic analysis. RESULTS: In total, 46 patients received ART in this study. Among these patients, we observed a significant decrease in dose to the submandibular glands (mean ± standard deviation: -219.2 ± 291.2 cGy, P < 10-5), parotids (-68.2 ± 197.7 cGy, P = .001), and oral cavity (-238.7 ± 206.7 cGy, P < 10-5) with the adaptive plan. Normal tissue complication probabilities for xerostomia computed from mean parotid doses also decreased significantly with the adaptive plans (P = .008). We also observed systematic intratreatment volume reductions (ΔV) for GTVs and normal tissues. Candidate triggers were identified that predicted significant improvement with ART, including parotid ΔV = 7%, neck ΔV = 2%, and nodal GTV ΔV = 29%. CONCLUSIONS: Systematic offline head and neck ART was successfully deployed on conventional LINACs and reduced doses to critical salivary structures and the oral cavity. Automated cone beam computed tomography tracking provided information regarding anatomic changes that may aid patient-specific triggering for ART.

Creating patient-specific digital phantoms with a longitudinal atlas for evaluating deformable CT-CBCT registration in adaptive lung radiotherapy.

BACKGROUND: Quality assurance of deformable image registration (DIR) is challenging because the ground truth is often unavailable. In addition, current approaches that rely on artificial transformations do not adequately resemble clinical scenarios encountered in adaptive radiotherapy. PURPOSE: We developed an atlas-based method to create a variety of patient-specific serial digital phantoms with CBCT-like image quality to assess the DIR performance for longitudinal CBCT imaging data in adaptive lung radiotherapy. METHODS: A library of deformations was created by extracting the longitudinal changes observed between a planning CT and weekly CBCT from an atlas of lung radiotherapy patients. The planning CT of an inquiry patient was first deformed by mapping the deformation pattern from a matched atlas patient, and subsequently appended with CBCT artifacts to imitate a weekly CBCT. Finally, a group of digital phantoms around an inquiry patient was produced to simulate a series of possible evolutions of tumor and adjacent normal structures. We validated the generated deformation vector fields (DVFs) to ensure numerically and physiologically realistic transformations. The proposed framework was applied to evaluate the performance of the DIR algorithm implemented in the commercial Eclipse treatment planning system in a retrospective study of eight inquiry patients. RESULTS: The generated DVFs were inverse consistent within less than 3 mm and did not exhibit unrealistic folding. The deformation patterns adequately mimicked the observed longitudinal anatomical changes of the matched atlas patients. Worse Eclipse DVF accuracy was observed in regions of low image contrast or artifacts. The structure volumes exhibiting a DVF error magnitude of equal or more than 2 mm ranged from 24.5% (spinal cord) to 69.2% (heart) and the maximum DVF error exceeded 5 mm for all structures except the spinal cord. Contour-based evaluations showed a high degree of alignment with dice similarity coefficients above 0.8 in all cases, which underestimated the overall DVF accuracy within the structures. CONCLUSIONS: It is feasible to create and augment digital phantoms based on a particular patient of interest using multiple series of deformation patterns from matched patients in an atlas. This can provide a semi-automated procedure to complement the quality assurance of CT-CBCT DIR and facilitate the clinical implementation of image-guided and adaptive radiotherapy that involve longitudinal CBCT imaging studies.

Automated tracking of morphologic changes in weekly magnetic resonance imaging during head and neck radiotherapy.

BACKGROUND AND PURPOSE: Anatomic changes during head and neck radiotherapy can impact dose delivery, necessitate adaptive replanning, and indicate patient-specific response to treatment. We have developed an automated system to track these changes through longitudinal MRI scans to aid identification and clinical intervention. The purpose of this article is to describe this tracking system and present results from an initial cohort of patients. MATERIALS AND METHODS: The Automated Watchdog in Adaptive Radiotherapy Environment (AWARE) was developed to process longitudinal MRI data for radiotherapy patients. AWARE automatically identifies and collects weekly scans, propagates radiotherapy planning structures, computes structure changes over time, and reports important trends to the clinical team. AWARE also incorporates manual structure review and revision from clinical experts and dynamically updates tracking statistics when necessary. AWARE was applied to patients receiving weekly T2-weighted MRI scans during head and neck radiotherapy. Changes in nodal gross tumor volume (GTV) and parotid gland delineations were tracked over time to assess changes during treatment and identify early indicators of treatment response. RESULTS: N = 91 patients were tracked and analyzed in this study. Nodal GTVs and parotids both shrunk considerably throughout treatment (-9.7 ± 7.7% and -3.7 ± 3.3% per week, respectively). Ipsilateral parotids shrunk significantly faster than contralateral (-4.3 ± 3.1% vs. -2.9 ± 3.3% per week, p = 0.005) and increased in distance from GTVs over time (+2.7 ± 7.2% per week, p < 1 × 10-5 ). Automatic structure propagations agreed well with manual revisions (Dice = 0.88 ± 0.09 for parotids and 0.80 ± 0.15 for GTVs), but for GTVs the agreement degraded 4-5 weeks after the start of treatment. Changes in GTV volume observed by AWARE as early as one week into treatment were predictive of large changes later in the course (AUC = 0.79). CONCLUSION: AWARE automatically identified longitudinal changes in GTV and parotid volumes during radiotherapy. Results suggest that this system may be useful for identifying rapidly responding patients as early as one week into treatment.

RMSim: controlled respiratory motion simulation on static patient scans.

Objective.This work aims to generate realistic anatomical deformations from static patient scans. Specifically, we present a method to generate these deformations/augmentations via deep learning driven respiratory motion simulation that provides the ground truth for validating deformable image registration (DIR) algorithms and driving more accurate deep learning based DIR.Approach.We present a novel 3D Seq2Seq deep learning respiratory motion simulator (RMSim) that learns from 4D-CT images and predicts future breathing phases given a static CT image. The predicted respiratory patterns, represented by time-varying displacement vector fields (DVFs) at different breathing phases, are modulated through auxiliary inputs of 1D breathing traces so that a larger amplitude in the trace results in more significant predicted deformation. Stacked 3D-ConvLSTMs are used to capture the spatial-temporal respiration patterns. Training loss includes a smoothness loss in the DVF and mean-squared error between the predicted and ground truth phase images. A spatial transformer deforms the static CT with the predicted DVF to generate the predicted phase image. 10-phase 4D-CTs of 140 internal patients were used to train and test RMSim. The trained RMSim was then used to augment a public DIR challenge dataset for training VoxelMorph to show the effectiveness of RMSim-generated deformation augmentation.Main results.We validated our RMSim output with both private and public benchmark datasets (healthy and cancer patients). The structure similarity index measure (SSIM) for predicted breathing phases and ground truth 4D CT images was 0.92 ± 0.04, demonstrating RMSim's potential to generate realistic respiratory motion. Moreover, the landmark registration error in a public DIR dataset was improved from 8.12 ± 5.78 mm to 6.58mm ± 6.38 mm using RMSim-augmented training data.Significance.The proposed approach can be used for validating DIR algorithms as well as for patient-specific augmentations to improve deep learning DIR algorithms. The code, pretrained models, and augmented DIR validation datasets will be released athttps://github.com/nadeemlab/SeqX2Y.

Seq2Morph: A deep learning deformable image registration algorithm for longitudinal imaging studies and adaptive radiotherapy.

PURPOSE: To simultaneously register all the longitudinal images acquired in a radiotherapy course for analyzing patients' anatomy changes for adaptive radiotherapy (ART). METHODS: To address the unique needs of ART, we designed Seq2Morph, a novel deep learning-based deformable image registration (DIR) network. Seq2Morph was built upon VoxelMorph which is a general-purpose framework for learning-based image registration. The major upgrades are (1) expansion of inputs to all weekly cone-beam computed tomography (CBCTs) acquired for monitoring treatment responses throughout a radiotherapy course, for registration to their planning CT; (2) incorporation of 3D convolutional long short-term memory between the encoder and decoder of VoxelMorph, to parse the temporal patterns of anatomical changes; and (3) addition of bidirectional pathways to calculate and minimize inverse consistency errors (ICEs). Longitudinal image sets from 50 patients, including a planning CT and 6 weekly CBCTs per patient, were utilized for network training and cross-validation. The outputs were deformation vector fields for all the registration pairs. The loss function was composed of a normalized cross-correlation for image intensity similarity, a DICE for contour similarity, an ICE, and a deformation regularization term. For performance evaluation, DICE and Hausdorff distance (HD) for the manual versus predicted contours of tumor and esophagus on weekly basis were quantified and further compared with other state-of-the-art algorithms, including conventional VoxelMorph and large deformation diffeomorphic metric mapping (LDDMM). RESULTS: Visualization of the hidden states of Seq2Morph revealed distinct spatiotemporal anatomy change patterns. Quantitatively, Seq2Morph performed similarly to LDDMM, but significantly outperformed VoxelMorph as measured by GTV DICE: (0.799±0.078, 0.798±0.081, and 0.773±0.078), and 50% HD (mm): (0.80±0.57, 0.88±0.66, and 0.95±0.60). The per-patient inference of Seq2Morph took 22 s, much less than LDDMM (∼30 min). CONCLUSIONS: Seq2Morph can provide accurate and fast DIR for longitudinal image studies by exploiting spatial-temporal patterns. It closely matches the clinical workflow and has the potential to serve both online and offline ART.

Deep learning driven predictive treatment planning for adaptive radiotherapy of lung cancer.

BACKGROUND AND PURPOSE: To develop a novel deep learning algorithm of sequential analysis, Seq2Seq, for predicting weekly anatomical changes of lung tumor and esophagus during definitive radiotherapy, incorporate the potential tumor shrinkage into a predictive treatment planning paradigm, and improve the therapeutic ratio. METHODS AND MATERIALS: Seq2Seq starts with the primary tumor and esophagus observed on the planning CT to predict their geometric evolution during radiotherapy on a weekly basis, and subsequently updates the predictions with new snapshots acquired via weekly CBCTs. Seq2Seq is equipped with convolutional long short term memory to analyze the spatial-temporal changes of longitudinal images, trained and validated using a dataset including sixty patients. Predictive plans were optimized according to each weekly prediction and made ready for weekly deployment to mitigate the clinical burden of online weekly replanning. RESULTS: Seq2Seq tracks structural changes well: DICE between predicted and actual weekly tumor and esophagus were (0.83 ± 0.10, 0.79 ± 0.14, 0.78 ± 0.12, 0.77 ± 0.12, 0.75 ± 0.12, 0.71 ± 0.17), and (0.72 ± 0.16, 0.73 ± 0.11, 0.75 ± 0.08, 0.74 ± 0.09, 0.72 ± 0.14, 0.71 ± 0.14), respectively, while the average Hausdorff distances were within 2 mm. Evaluating dose to the actual weekly tumor and esophagus, a 4.2 Gy reduction in esophagus mean dose while maintaining 60 Gy tumor coverage was achieved with the predictive weekly plans, compared to the plan optimized using the initial tumor and esophagus alone, primarily due to noticeable tumor shrinkage during radiotherapy. CONCLUSION: It is feasible to predict the longitudinal changes of tumor and esophagus with the Seq2Seq, which could lead to improving the efficiency and effectiveness of lung adaptive radiotherapy.

Deformation driven Seq2Seq longitudinal tumor and organs-at-risk prediction for radiotherapy.

PURPOSE: Radiotherapy presents unique challenges and clinical requirements for longitudinal tumor and organ-at-risk (OAR) prediction during treatment. The challenges include tumor inflammation/edema and radiation-induced changes in organ geometry, whereas the clinical requirements demand flexibility in input/output sequence timepoints to update the predictions on rolling basis and the grounding of all predictions in relationship to the pre-treatment imaging information for response and toxicity assessment in adaptive radiotherapy. METHODS: To deal with the aforementioned challenges and to comply with the clinical requirements, we present a novel 3D sequence-to-sequence model based on Convolution Long Short-Term Memory (ConvLSTM) that makes use of series of deformation vector fields (DVFs) between individual timepoints and reference pre-treatment/planning CTs to predict future anatomical deformations and changes in gross tumor volume as well as critical OARs. High-quality DVF training data are created by employing hyper-parameter optimization on the subset of the training data with DICE coefficient and mutual information metric. We validated our model on two radiotherapy datasets: a publicly available head-and-neck dataset (28 patients with manually contoured pre-, mid-, and post-treatment CTs), and an internal non-small cell lung cancer dataset (63 patients with manually contoured planning CT and 6 weekly CBCTs). RESULTS: The use of DVF representation and skip connections overcomes the blurring issue of ConvLSTM prediction with the traditional image representation. The mean and standard deviation of DICE for predictions of lung GTV at weeks 4, 5, and 6 were 0.83 ± 0.09, 0.82 ± 0.08, and 0.81 ± 0.10, respectively, and for post-treatment ipsilateral and contralateral parotids, were 0.81 ± 0.06 and 0.85 ± 0.02. CONCLUSION: We presented a novel DVF-based Seq2Seq model for medical images, leveraging the complete 3D imaging information of a relatively large longitudinal clinical dataset, to carry out longitudinal GTV/OAR predictions for anatomical changes in HN and lung radiotherapy patients, which has potential to improve RT outcomes.

Multi-class medical image segmentation using one-vs-rest graph cuts and majority voting.

Purpose: Semi-automatic image segmentation is still a valuable tool in clinical applications since it retains the expert oversights legally required. However, semi-automatic methods for simultaneous multi-class segmentation are difficult to be clinically implemented due to the complexity of underlining algorithms. We purpose an efficient one-vs-rest graph cut approach of which the complexity only grows linearly as the number of classes increases. Approach: Given an image slice, we construct multiple one-vs-rest graphs, each for a tissue class, for inference of a conditional random field (CRF). The one-vs-rest graph cut is to minimize the CRF energy derived from regional and boundary class probabilities estimated from random forests to obtain a one-vs-rest segmentation. The final segmentation is obtained by fusing from those one-vs-rest segmentations based on majority voting. We compare our method to a well-used multi-class graph cut method, alpha-beta swap, and a fully connected CRF (FCCRF) method, in brain tumor segmentation of 20 high-grade tumor cases in 2013 MICCAI dataset. Results: Our method achieved mean Dice score of 0.83 for whole tumor, compared to 0.80 by alpha-beta swap and 0.79 by FCCRF. There was a performance improvement over alpha-beta swap by a factor of five. Conclusions: Our method utilizes the probabilistic-based CRF which can be estimated from any machine learning technique. Comparing to traditional multi-class graph cut, the purposed one-vs-rest approach has complexity that grows only linearly as the number of classes increases, therefore, our method can be applicable for both online semi-automatic and offline automatic segmentation in clinical applications.

Early Prediction of Acute Esophagitis for Adaptive Radiation Therapy.

PURPOSE: Acute esophagitis (AE) is a common dose-limiting toxicity in radiation therapy of locally advanced non-small cell lung cancer (LA-NSCLC). We developed an early AE prediction model from weekly accumulated esophagus dose and its associated local volumetric change. METHODS AND MATERIALS: Fifty-one patients with LA-NSCLC underwent treatment with intensity modulated radiation therapy to 60 Gy in 2-Gy fractions with concurrent chemotherapy and weekly cone beam computed tomography (CBCT). Twenty-eight patients (55%) developed grade ≥2 AE (≥AE2) at a median of 4 weeks after the start of radiation therapy. For early ≥AE2 prediction, the esophagus on CBCT of the first 2 weeks was deformably registered to the planning computed tomography images, and weekly esophagus dose was accumulated. Week 1-to-week 2 (w1→w2) esophagus volume changes including maximum esophagus expansion (MEex%) and volumes with ≥x% local expansions (VEx%; x = 5, 10, 15) were calculated from the Jacobian map of deformation vector field gradients. Logistic regression model with 5-fold cross-validation was built using combinations of the accumulated mean esophagus doses (MED) and the esophagus change parameters with the lowest P value in univariate analysis. The model was validated on an additional 18 and 11 patients with weekly CBCT and magnetic resonance imaging (MRI), respectively, and compared with models using only planned mean dose (MEDPlan). Performance was assessed using area under the curve (AUC) and Hosmer-Lemeshow test (PHL). RESULTS: Univariately, w1→w2 VE10% (P = .004), VE5% (P = .01) and MEex% (P = .02) significantly predicted ≥AE2. A model combining MEDW2 and w1→w2 VE10% had the best performance (AUC = 0.80; PHL = 0.43), whereas the MEDPlan model had a lower accuracy (AUC = 0.67; PHL = 0.26). The combined model also showed high accuracy in the CBCT (AUC = 0.78) and MRI validations (AUC = 0.75). CONCLUSIONS: A CBCT-based, cross-validated, and internally validated model on MRI with a combination of accumulated esophagus dose and local volume change from the first 2 weeks of chemotherapy significantly improved AE prediction compared with conventional models using only the planned dose. This model could inform plan adaptation early to lower the risk of esophagitis.

Predicting spatial esophageal changes in a multimodal longitudinal imaging study via a convolutional recurrent neural network.

Acute esophagitis (AE) occurs among a significant number of patients with locally advanced lung cancer treated with radiotherapy. Early prediction of AE, indicated by esophageal wall expansion, is critical, as it can facilitate the redesign of treatment plans to reduce radiation-induced esophageal toxicity in an adaptive radiotherapy (ART) workflow. We have developed a novel machine learning framework to predict the patient-specific spatial presentation of the esophagus in the weeks following treatment, using magnetic resonance imaging (MRI)/ cone-beam CT (CBCT) scans acquired earlier in the 6 week radiotherapy course. Our algorithm captures the response patterns of the esophagus to radiation on a patch level, using a convolutional neural network. A recurrence neural network then parses the evolutionary patterns of the selected features in the time series, and produces a predicted esophagus-or-not label for each individual patch over future weeks. Finally, the esophagus is reconstructed, using all the predicted labels. The algorithm is trained and validated by means of ∼ 250 000 patches taken from MRI scans acquired weekly from a variety of patients, and tested using both weekly MRI and CBCT scans under a leave-one-patient-out scheme. In addition, our approach is externally validated using a publicly available dataset (Hugo 2017). Using the first three weekly scans, the algorithm can predict the condition of the esophagus over the succeeding 3 weeks with a Dice coefficient of 0.83 ± 0.04, estimate esophagus volume highly (0.98), correlated with the actual volume, using our institutional MRI/CBCT data. When evaluated using the external weekly CBCT data, the averaged Dice coefficient is 0.89 ± 0.03. Our novel algorithm may prove useful in enabling radiation oncologists to monitor and detect AE in its early stages, and could potentially play an important role in the ART decision-making process.

Self-derived organ attention for unpaired CT-MRI deep domain adaptation based MRI segmentation.

To develop and evaluate a deep learning method to segment parotid glands from MRI using unannotated MRI and unpaired expert-segmented CT datasets. We introduced a new self-derived organ attention deep learning network for combined CT to MRI image-to-image translation (I2I) and MRI segmentation, all trained as an end-to-end network. The expert segmentations available on CT scans were combined with the I2I translated pseudo MR images to train the MRI segmentation network. Once trained, the MRI segmentation network alone is required. We introduced an organ attention discriminator that constrains the CT to MR generator to synthesize pseudo MR images that preserve organ geometry and appearance statistics as in real MRI. The I2I translation network training was regularized using the organ attention discriminator, global image-matching discriminator, and cycle consistency losses. MRI segmentation training was regularized by using cross-entropy loss. Segmentation performance was compared against multiple domain adaptation-based segmentation methods using the Dice similarity coefficient (DSC) and Hausdorff distance at the 95th percentile (HD95). All networks were trained using 85 unlabeled T2-weighted fat suppressed (T2wFS) MRIs and 96 expert-segmented CT scans. Performance upper-limit was based on fully supervised MRI training done using the 85 T2wFS MRI with expert segmentations. Independent evaluation was performed on 77 MRIs never used in training. The proposed approach achieved the highest accuracy (left parotid: DSC 0.82 ± 0.03, HD95 2.98 ± 1.01 mm; right parotid: 0.81 ± 0.05, HD95 3.14 ± 1.17 mm) compared to other methods. This accuracy was close to the reference fully supervised MRI segmentation (DSC of 0.84 ± 0.04, a HD95 of 2.24 ± 0.77 mm for the left parotid, and a DSC of 0.84 ± 0.06 and HD95 of 2.32 ± 1.37 mm for the right parotid glands).

Predictive dose accumulation for HN adaptive radiotherapy.

During radiation therapy (RT) of head and neck (HN) cancer, the shape and volume of the parotid glands (PG) may change significantly, resulting in clinically relevant deviations of delivered dose from the planning dose. Early and accurate longitudinal prediction of PG anatomical changes during the RT can be valuable to inform decisions on plan adaptation. We developed a deep neural network for longitudinal predictions using the displacement fields (DFs) between the planning computed tomography (pCT) and weekly cone beam computed tomography (CBCT). Sixty-three HN patients treated with volumetric modulated arc were retrospectively studied. We calculated DFs between pCT and week 1-3 CBCT by B-spline and Demon deformable image registration (DIR). The resultant DFs were subsequently used as input to our novel network to predict the week 4 to 6 DFs for generating predicted weekly PG contours and weekly dose distributions. For evaluation, we measured dice similarity (DICE), and the uncertainty of accumulated dose. Moreover, we compared the detection accuracies of candidates for adaptive radiotherapy (ART) when the trigger criteria were mean dose difference more than 10%, 7.5%, and 5%, respectively. The DICE of ipsilateral/contralateral PG at week 4 to 6 using the prediction model trained with B-spline were 0.81 [Formula: see text] 0.07/0.81 [Formula: see text] 0.04 (week 4), 0.79 [Formula: see text] 0.06/0.81 [Formula: see text] 0.05 (week 5) and 0.78 [Formula: see text] 0.06/0.82 [Formula: see text] (week 6). The DICE with the Demons model were 0.78 [Formula: see text] 0.08/0.82 [Formula: see text] 0.03 (week 4), 0.77 [Formula: see text] 0.07/0.82 [Formula: see text] 0.04 (week 5) and 0.75 [Formula: see text] 0.07/0.82 [Formula: see text] 0.02 (week 6). The dose volume histogram (DVH) analysis with the predicted accumulated dose showed the feasibility of predicting dose uncertainty due to the PG anatomical changes. The AUC of ART candidate detection with our predictive model was over 0.90. In conclusion, the proposed network was able to predict future anatomical changes and dose uncertainty of PGs with clinically acceptable accuracy, and hence can be readily integrated into the ART workflow.

PSIGAN: Joint Probabilistic Segmentation and Image Distribution Matching for Unpaired Cross-Modality Adaptation-Based MRI Segmentation.

We developed a new joint probabilistic segmentation and image distribution matching generative adversarial network (PSIGAN) for unsupervised domain adaptation (UDA) and multi-organ segmentation from magnetic resonance (MRI) images. Our UDA approach models the co-dependency between images and their segmentation as a joint probability distribution using a new structure discriminator. The structure discriminator computes structure of interest focused adversarial loss by combining the generated pseudo MRI with probabilistic segmentations produced by a simultaneously trained segmentation sub-network. The segmentation sub-network is trained using the pseudo MRI produced by the generator sub-network. This leads to a cyclical optimization of both the generator and segmentation sub-networks that are jointly trained as part of an end-to-end network. Extensive experiments and comparisons against multiple state-of-the-art methods were done on four different MRI sequences totalling 257 scans for generating multi-organ and tumor segmentation. The experiments included, (a) 20 T1-weighted (T1w) in-phase mdixon and (b) 20 T2-weighted (T2w) abdominal MRI for segmenting liver, spleen, left and right kidneys, (c) 162 T2-weighted fat suppressed head and neck MRI (T2wFS) for parotid gland segmentation, and (d) 75 T2w MRI for lung tumor segmentation. Our method achieved an overall average DSC of 0.87 on T1w and 0.90 on T2w for the abdominal organs, 0.82 on T2wFS for the parotid glands, and 0.77 on T2w MRI for lung tumors.

Library of deep-learning image segmentation and outcomes model-implementations.

An open-source library of implementations for deep-learning-based image segmentation and outcomes models based on radiotherapy and radiomics is presented. As oncology treatment planning becomes increasingly driven by automation, such a library of model implementations is crucial to (i) validate existing models on datasets collected at different institutions, (ii) automate segmentation, (iii) create ensembles for improving performance and (iv) incorporate validated models in the clinical workflow. Inclusion of deep-learning-based image segmentation and outcomes models in the same library provides a fully automated and reproduceable pipeline to estimate prognosis. The library was developed with the Computational Environment for Radiological Research (CERR) software platform. Centralizing model implementations in CERR builds upon its rich set of radiotherapy and radiomics tools and caters to the world-wide user base. CERR provides well-validated feature extraction pipelines for radiotherapy dosimetry and radiomics with fine control over the calculation settings, allowing users to select appropriate parameters used in model derivation. Models for automatic image segmentation are distributed via containers, allowing them to be deployed with a variety of scientific computing architectures. The library includes implementations of popular DVH-based models outlined in the Quantitative Analysis of Normal Tissue Effects in the Clinic effort and recently published literature. Radiomics models include features from the Image Biomarker Standardization Initiative and application-specific features found to be relevant across multiple sites and image modalities. The library is distributed as a module within CERR at https://www.github.com/cerr/CERR under the GNU-GPL copyleft with additional restrictions on clinical and commercial use and provision to dual license in future.

Quantification of accumulated dose and associated anatomical changes of esophagus using weekly Magnetic Resonance Imaging acquired during radiotherapy of locally advanced lung cancer.

BACKGROUND AND PURPOSE: Minimizing acute esophagitis (AE) in locally advanced non-small cell lung cancer (LA-NSCLC) is critical given the proximity between the esophagus and the tumor. In this pilot study, we developed a clinical platform for quantification of accumulated doses and volumetric changes of esophagus via weekly Magnetic Resonance Imaging (MRI) for adaptive radiotherapy (RT). MATERIAL AND METHODS: Eleven patients treated via intensity-modulated RT to 60-70 Gy in 2-3 Gy-fractions with concurrent chemotherapy underwent weekly MRIs. Eight patients developed AE grade 2 (AE2), 3-6 weeks after RT started. First, weekly MRI esophagus contours were rigidly propagated to planning CT and the distances between the medial esophageal axes were calculated as positional uncertainties. Then, the weekly MRI were deformably registered to the planning CT and the total dose delivered to esophagus was accumulated. Weekly Maximum Esophagus Expansion (MEex) was calculated using the Jacobian map. Eventually, esophageal dose parameters (Mean Esophagus Dose (MED), V90% and D5cc) between the planned and accumulated dose were compared. RESULTS: Positional esophagus uncertainties were 6.8 ± 1.8 mm across patients. For the entire cohort at the end of RT: the median accumulated MED was significantly higher than the planned dose (24 Gy vs. 21 Gy p = 0.006). The median V90% and D5cc were 12.5 cm3 vs. 11.5 cm3 (p = 0.05) and 61 Gy vs. 60 Gy (p = 0.01), for accumulated and planned dose, respectively. The median MEex was 24% and was significantly associated with AE2 (p = 0.008). CONCLUSIONS: MRI is well suited for tracking esophagus volumetric changes and accumulating doses. Longitudinal esophagus expansion could reflect radiation-induced inflammation that may link to AE.

Toward predicting the evolution of lung tumors during radiotherapy observed on a longitudinal MR imaging study via a deep learning algorithm.

PURPOSE: To predict the spatial and temporal trajectories of lung tumor during radiotherapy monitored under a longitudinal magnetic resonance imaging (MRI) study via a deep learning algorithm for facilitating adaptive radiotherapy (ART). METHODS: We monitored 10 lung cancer patients by acquiring weekly MRI-T2w scans over a course of radiotherapy. Under an ART workflow, we developed a predictive neural network (P-net) to predict the spatial distributions of tumors in the coming weeks utilizing images acquired earlier in the course. The three-step P-net consisted of a convolutional neural network to extract relevant features of the tumor and its environment, followed by a recurrence neural network constructed with gated recurrent units to analyze trajectories of tumor evolution in response to radiotherapy, and finally an attention model to weight the importance of weekly observations and produce the predictions. The performance of P-net was measured with Dice and root mean square surface distance (RMSSD) between the algorithm-predicted and experts-contoured tumors under a leave-one-out scheme. RESULTS: Tumor shrinkage was 60% ± 27% (mean ± standard deviation) by the end of radiotherapy across nine patients. Using images from the first three weeks, P-net predicted tumors on future weeks (4, 5, 6) with a Dice and RMSSD of (0.78 ± 0.22, 0.69 ± 0.24, 0.69 ± 0.26), and (2.1 ± 1.1 mm, 2.3 ± 0.8 mm, 2.6 ± 1.4 mm), respectively. CONCLUSION: The proposed deep learning algorithm can capture and predict spatial and temporal patterns of tumor regression in a longitudinal imaging study. It closely follows the clinical workflow, and could facilitate the decision-making of ART. A prospective study including more patients is warranted.