Automated tracking of morphologic changes in weekly magnetic resonance imaging during head and neck radiotherapy.

BACKGROUND AND PURPOSE: Anatomic changes during head and neck radiotherapy can impact dose delivery, necessitate adaptive replanning, and indicate patient-specific response to treatment. We have developed an automated system to track these changes through longitudinal MRI scans to aid identification and clinical intervention. The purpose of this article is to describe this tracking system and present results from an initial cohort of patients. MATERIALS AND METHODS: The Automated Watchdog in Adaptive Radiotherapy Environment (AWARE) was developed to process longitudinal MRI data for radiotherapy patients. AWARE automatically identifies and collects weekly scans, propagates radiotherapy planning structures, computes structure changes over time, and reports important trends to the clinical team. AWARE also incorporates manual structure review and revision from clinical experts and dynamically updates tracking statistics when necessary. AWARE was applied to patients receiving weekly T2-weighted MRI scans during head and neck radiotherapy. Changes in nodal gross tumor volume (GTV) and parotid gland delineations were tracked over time to assess changes during treatment and identify early indicators of treatment response. RESULTS: N = 91 patients were tracked and analyzed in this study. Nodal GTVs and parotids both shrunk considerably throughout treatment (-9.7 ± 7.7% and -3.7 ± 3.3% per week, respectively). Ipsilateral parotids shrunk significantly faster than contralateral (-4.3 ± 3.1% vs. -2.9 ± 3.3% per week, p = 0.005) and increased in distance from GTVs over time (+2.7 ± 7.2% per week, p < 1 × 10-5 ). Automatic structure propagations agreed well with manual revisions (Dice = 0.88 ± 0.09 for parotids and 0.80 ± 0.15 for GTVs), but for GTVs the agreement degraded 4-5 weeks after the start of treatment. Changes in GTV volume observed by AWARE as early as one week into treatment were predictive of large changes later in the course (AUC = 0.79). CONCLUSION: AWARE automatically identified longitudinal changes in GTV and parotid volumes during radiotherapy. Results suggest that this system may be useful for identifying rapidly responding patients as early as one week into treatment.

Evaluation of respiratory motion-corrected cone-beam CT at end expiration in abdominal radiotherapy sites: a prospective study.

BACKGROUND: Cone beam computed tomography (CBCT) for radiotherapy image guidance suffers from respiratory motion artifacts. This limits soft tissue visualization and localization accuracy, particularly in abdominal sites. We report on a prospective study of respiratory motion-corrected (RMC)-CBCT to evaluate its efficacy in localizing abdominal organs and improving soft tissue visibility at end expiration. MATERIAL AND METHODS: In an IRB approved study, 11 patients with gastroesophageal junction (GEJ) cancer and five with pancreatic cancer underwent a respiration-correlated CT (4DCT), a respiration-gated CBCT (G-CBCT) near end expiration and a one-minute free-breathing CBCT scan on a single treatment day. Respiration was recorded with an external monitor. An RMC-CBCT and an uncorrected CBCT (NC-CBCT) were computed from the free-breathing scan, based on a respiratory model of deformations derived from the 4DCT. Localization discrepancy was computed as the 3D displacement of the GEJ region (GEJ patients), or gross tumor volume (GTV) and kidneys (pancreas patients) in the NC-CBCT and RMC-CBCT relative to their positions in the G-CBCT. Similarity of soft-tissue features was measured using a normalized cross correlation (NCC) function. RESULTS: Localization discrepancy from the end-expiration G-CBCT was reduced for RMC-CBCT compared to NC-CBCT in eight of eleven GEJ cases (mean ± standard deviation, respectively, 0.21 ± 0.11 and 0.43 ± 0.28 cm), in all five pancreatic GTVs (0.26 ± 0.21 and 0.42 ± 0.29 cm) and all ten kidneys (0.19 ± 0.13 and 0.51 ± 0.25 cm). Soft-tissue feature similarity around GEJ was higher with RMC-CBCT in nine of eleven cases (NCC =0.48 ± 0.20 and 0.43 ± 0.21), and eight of ten kidneys (0.44 ± 0.16 and 0.40 ± 0.17). CONCLUSIONS: In a prospective study of motion-corrected CBCT in GEJ and pancreas, RMC-CBCT yielded improved organ visibility and localization accuracy for gated treatment at end expiration in the majority of cases.

Creating patient-specific digital phantoms with a longitudinal atlas for evaluating deformable CT-CBCT registration in adaptive lung radiotherapy.

BACKGROUND: Quality assurance of deformable image registration (DIR) is challenging because the ground truth is often unavailable. In addition, current approaches that rely on artificial transformations do not adequately resemble clinical scenarios encountered in adaptive radiotherapy. PURPOSE: We developed an atlas-based method to create a variety of patient-specific serial digital phantoms with CBCT-like image quality to assess the DIR performance for longitudinal CBCT imaging data in adaptive lung radiotherapy. METHODS: A library of deformations was created by extracting the longitudinal changes observed between a planning CT and weekly CBCT from an atlas of lung radiotherapy patients. The planning CT of an inquiry patient was first deformed by mapping the deformation pattern from a matched atlas patient, and subsequently appended with CBCT artifacts to imitate a weekly CBCT. Finally, a group of digital phantoms around an inquiry patient was produced to simulate a series of possible evolutions of tumor and adjacent normal structures. We validated the generated deformation vector fields (DVFs) to ensure numerically and physiologically realistic transformations. The proposed framework was applied to evaluate the performance of the DIR algorithm implemented in the commercial Eclipse treatment planning system in a retrospective study of eight inquiry patients. RESULTS: The generated DVFs were inverse consistent within less than 3 mm and did not exhibit unrealistic folding. The deformation patterns adequately mimicked the observed longitudinal anatomical changes of the matched atlas patients. Worse Eclipse DVF accuracy was observed in regions of low image contrast or artifacts. The structure volumes exhibiting a DVF error magnitude of equal or more than 2 mm ranged from 24.5% (spinal cord) to 69.2% (heart) and the maximum DVF error exceeded 5 mm for all structures except the spinal cord. Contour-based evaluations showed a high degree of alignment with dice similarity coefficients above 0.8 in all cases, which underestimated the overall DVF accuracy within the structures. CONCLUSIONS: It is feasible to create and augment digital phantoms based on a particular patient of interest using multiple series of deformation patterns from matched patients in an atlas. This can provide a semi-automated procedure to complement the quality assurance of CT-CBCT DIR and facilitate the clinical implementation of image-guided and adaptive radiotherapy that involve longitudinal CBCT imaging studies.

Automated planning of stereotactic spine re-irradiation using cumulative dose limits.

Background and Purpose: The lack of dedicated tools in commercial planning systems currently restricts efficient review and planning for re-irradiation. The aim of this study was to develop an automated re-irradiation planning framework based on cumulative doses. Materials and Methods: We performed a retrospective study of 14 patients who received spine SBRT re-irradiation near a previously irradiated treatment site. A fully-automated workflow, DART (Dose Accumulation-based Re-irradiation Tool), was implemented within Eclipse by leveraging a combination of a dose accumulation script and a proprietary automated optimization algorithm. First, we converted the prior treatment dose into equivalent dose in 2 Gy fractions (EQD2) and mapped it to the current anatomy, utilizing deformable image registration. Subsequently, the intersection of EQD2 isodose lines with relevant organs at risk defines a series of optimization structures. During plan optimization, the residual allowable dose at a specified tissue tolerance was treated as a hard constraint. Results: All DART plans met institutional physical and cumulative constraints and passed plan checks by qualified medical physicists. DART demonstrated significant improvements in target coverage over clinical plans, with an average increase in PTV D99% and V100% of 2.3 Gy [range -0.3-7.7 Gy] and 3.4 % [range -0.4 %-7.6 %] (p < 0.01, paired t-test), respectively. Moreover, high-dose spillage (>105 %) outside the PTV was reduced by up to 7 cm3. The homogeneity index for DART plans was improved by 19 % (p < 0.001). Conclusions: DART provides a powerful framework to achieve more tailored re-irradiation plans by accounting for dose distributions from the previous treatments. The superior plan quality could improve the therapeutic ratio for re-irradiation patients.

Clinical Experience With an Offline Adaptive Radiation Therapy Head and Neck Program: Dosimetric Benefits and Opportunities for Patient Selection.

PURPOSE: The objective of this study was to develop a linear accelerator (LINAC)-based adaptive radiation therapy (ART) workflow for the head and neck that is informed by automated image tracking to identify major anatomic changes warranting adaptation. In this study, we report our initial clinical experience with the program and an investigation into potential trigger signals for ART. METHODS AND MATERIALS: Offline ART was systematically performed on patients receiving radiation therapy for head and neck cancer on C-arm LINACs. Adaptations were performed at a single time point during treatment with resimulation approximately 3 weeks into treatment. Throughout treatment, all patients were tracked using an automated image tracking system called the Automated Watchdog for Adaptive Radiotherapy Environment (AWARE). AWARE measures volumetric changes in gross tumor volumes (GTVs) and selected normal tissues via cone beam computed tomography scans and deformable registration. The benefit of ART was determined by comparing adaptive plan dosimetry and normal tissue complication probabilities against the initial plans recalculated on resimulation computed tomography scans. Dosimetric differences were then correlated with AWARE-measured volume changes to identify patient-specific triggers for ART. Candidate trigger variables were evaluated using receiver operator characteristic analysis. RESULTS: In total, 46 patients received ART in this study. Among these patients, we observed a significant decrease in dose to the submandibular glands (mean ± standard deviation: -219.2 ± 291.2 cGy, P < 10-5), parotids (-68.2 ± 197.7 cGy, P = .001), and oral cavity (-238.7 ± 206.7 cGy, P < 10-5) with the adaptive plan. Normal tissue complication probabilities for xerostomia computed from mean parotid doses also decreased significantly with the adaptive plans (P = .008). We also observed systematic intratreatment volume reductions (ΔV) for GTVs and normal tissues. Candidate triggers were identified that predicted significant improvement with ART, including parotid ΔV = 7%, neck ΔV = 2%, and nodal GTV ΔV = 29%. CONCLUSIONS: Systematic offline head and neck ART was successfully deployed on conventional LINACs and reduced doses to critical salivary structures and the oral cavity. Automated cone beam computed tomography tracking provided information regarding anatomic changes that may aid patient-specific triggering for ART.

RMSim: controlled respiratory motion simulation on static patient scans.

Objective.This work aims to generate realistic anatomical deformations from static patient scans. Specifically, we present a method to generate these deformations/augmentations via deep learning driven respiratory motion simulation that provides the ground truth for validating deformable image registration (DIR) algorithms and driving more accurate deep learning based DIR.Approach.We present a novel 3D Seq2Seq deep learning respiratory motion simulator (RMSim) that learns from 4D-CT images and predicts future breathing phases given a static CT image. The predicted respiratory patterns, represented by time-varying displacement vector fields (DVFs) at different breathing phases, are modulated through auxiliary inputs of 1D breathing traces so that a larger amplitude in the trace results in more significant predicted deformation. Stacked 3D-ConvLSTMs are used to capture the spatial-temporal respiration patterns. Training loss includes a smoothness loss in the DVF and mean-squared error between the predicted and ground truth phase images. A spatial transformer deforms the static CT with the predicted DVF to generate the predicted phase image. 10-phase 4D-CTs of 140 internal patients were used to train and test RMSim. The trained RMSim was then used to augment a public DIR challenge dataset for training VoxelMorph to show the effectiveness of RMSim-generated deformation augmentation.Main results.We validated our RMSim output with both private and public benchmark datasets (healthy and cancer patients). The structure similarity index measure (SSIM) for predicted breathing phases and ground truth 4D CT images was 0.92 ± 0.04, demonstrating RMSim's potential to generate realistic respiratory motion. Moreover, the landmark registration error in a public DIR dataset was improved from 8.12 ± 5.78 mm to 6.58mm ± 6.38 mm using RMSim-augmented training data.Significance.The proposed approach can be used for validating DIR algorithms as well as for patient-specific augmentations to improve deep learning DIR algorithms. The code, pretrained models, and augmented DIR validation datasets will be released athttps://github.com/nadeemlab/SeqX2Y.

Seq2Morph: A deep learning deformable image registration algorithm for longitudinal imaging studies and adaptive radiotherapy.

PURPOSE: To simultaneously register all the longitudinal images acquired in a radiotherapy course for analyzing patients' anatomy changes for adaptive radiotherapy (ART). METHODS: To address the unique needs of ART, we designed Seq2Morph, a novel deep learning-based deformable image registration (DIR) network. Seq2Morph was built upon VoxelMorph which is a general-purpose framework for learning-based image registration. The major upgrades are (1) expansion of inputs to all weekly cone-beam computed tomography (CBCTs) acquired for monitoring treatment responses throughout a radiotherapy course, for registration to their planning CT; (2) incorporation of 3D convolutional long short-term memory between the encoder and decoder of VoxelMorph, to parse the temporal patterns of anatomical changes; and (3) addition of bidirectional pathways to calculate and minimize inverse consistency errors (ICEs). Longitudinal image sets from 50 patients, including a planning CT and 6 weekly CBCTs per patient, were utilized for network training and cross-validation. The outputs were deformation vector fields for all the registration pairs. The loss function was composed of a normalized cross-correlation for image intensity similarity, a DICE for contour similarity, an ICE, and a deformation regularization term. For performance evaluation, DICE and Hausdorff distance (HD) for the manual versus predicted contours of tumor and esophagus on weekly basis were quantified and further compared with other state-of-the-art algorithms, including conventional VoxelMorph and large deformation diffeomorphic metric mapping (LDDMM). RESULTS: Visualization of the hidden states of Seq2Morph revealed distinct spatiotemporal anatomy change patterns. Quantitatively, Seq2Morph performed similarly to LDDMM, but significantly outperformed VoxelMorph as measured by GTV DICE: (0.799±0.078, 0.798±0.081, and 0.773±0.078), and 50% HD (mm): (0.80±0.57, 0.88±0.66, and 0.95±0.60). The per-patient inference of Seq2Morph took 22 s, much less than LDDMM (∼30 min). CONCLUSIONS: Seq2Morph can provide accurate and fast DIR for longitudinal image studies by exploiting spatial-temporal patterns. It closely matches the clinical workflow and has the potential to serve both online and offline ART.

Interactive semiautomatic contour delineation using statistical conditional random fields framework.

PURPOSE: Contouring a normal anatomical structure during radiation treatment planning requires significant time and effort. The authors present a fast and accurate semiautomatic contour delineation method to reduce the time and effort required of expert users. METHODS: Following an initial segmentation on one CT slice, the user marks the target organ and nontarget pixels with a few simple brush strokes. The algorithm calculates statistics from this information that, in turn, determines the parameters of an energy function containing both boundary and regional components. The method uses a conditional random field graphical model to define the energy function to be minimized for obtaining an estimated optimal segmentation, and a graph partition algorithm to efficiently solve the energy function minimization. Organ boundary statistics are estimated from the segmentation and propagated to subsequent images; regional statistics are estimated from the simple brush strokes that are either propagated or redrawn as needed on subsequent images. This greatly reduces the user input needed and speeds up segmentations. The proposed method can be further accelerated with graph-based interpolation of alternating slices in place of user-guided segmentation. CT images from phantom and patients were used to evaluate this method. The authors determined the sensitivity and specificity of organ segmentations using physician-drawn contours as ground truth, as well as the predicted-to-ground truth surface distances. Finally, three physicians evaluated the contours for subjective acceptability. Interobserver and intraobserver analysis was also performed and Bland-Altman plots were used to evaluate agreement. RESULTS: Liver and kidney segmentations in patient volumetric CT images show that boundary samples provided on a single CT slice can be reused through the entire 3D stack of images to obtain accurate segmentation. In liver, our method has better sensitivity and specificity (0.925 and 0.995) than region growing (0.897 and 0.995) and level set methods (0.912 and 0.985) as well as shorter mean predicted-to-ground truth distance (2.13 mm) compared to regional growing (4.58 mm) and level set methods (8.55 mm and 4.74 mm). Similar results are observed in kidney segmentation. Physician evaluation of ten liver cases showed that 83% of contours did not need any modification, while 6% of contours needed modifications as assessed by two or more evaluators. In interobserver and intraobserver analysis, Bland-Altman plots showed our method to have better repeatability than the manual method while the delineation time was 15% faster on average. CONCLUSIONS: Our method achieves high accuracy in liver and kidney segmentation and considerably reduces the time and labor required for contour delineation. Since it extracts purely statistical information from the samples interactively specified by expert users, the method avoids heuristic assumptions commonly used by other methods. In addition, the method can be expanded to 3D directly without modification because the underlying graphical framework and graph partition optimization method fit naturally with the image grid structure.

Reduction of irregular breathing artifacts in respiration-correlated CT images using a respiratory motion model.

PURPOSE: Respiration-correlated CT (RCCT) images produced with commonly used phase-based sorting of CT slices often exhibit discontinuity artifacts between CT slices, caused by cycle-to-cycle amplitude variations in respiration. Sorting based on the displacement of the respiratory signal yields slices at more consistent respiratory motion states and hence reduces artifacts, but missing image data (gaps) may occur. The authors report on the application of a respiratory motion model to produce an RCCT image set with reduced artifacts and without missing data. METHODS: Input data consist of CT slices from a cine CT scan acquired while recording respiration by monitoring abdominal displacement. The model-based generation of RCCT images consists of four processing steps: (1) displacement-based sorting of CT slices to form volume images at 10 motion states over the cycle; (2) selection of a reference image without gaps and deformable registration between the reference image and each of the remaining images; (3) generation of the motion model by applying a principal component analysis to establish a relationship between displacement field and respiration signal at each motion state; (4) application of the motion model to deform the reference image into images at the 9 other motion states. Deformable image registration uses a modified fast free-form algorithm that excludes zero-intensity voxels, caused by missing data, from the image similarity term in the minimization function. In each iteration of the minimization, the displacement field in the gap regions is linearly interpolated from nearest neighbor nonzero intensity slices. Evaluation of the model-based RCCT examines three types of image sets: cine scans of a physical phantom programmed to move according to a patient respiratory signal, NURBS-based cardiac torso (NCAT) software phantom, and patient thoracic scans. RESULTS: Comparison in physical motion phantom shows that object distortion caused by variable motion amplitude in phase-based sorting is visibly reduced with model-based RCCT. Comparison of model-based RCCT to original NCAT images as ground truth shows best agreement at motion states whose displacement-sorted images have no missing slices, with mean and maximum discrepancies in lung of 1 and 3 mm, respectively. Larger discrepancies correlate with motion states having a larger number of missing slices in the displacement-sorted images. Artifacts in patient images at different motion states are also reduced. Comparison with displacement-sorted patient images as a ground truth shows that the model-based images closely reproduce the ground truth geometry at different motion states. CONCLUSIONS: Results in phantom and patient images indicate that the proposed method can produce RCCT image sets with reduced artifacts relative to phase-sorted images, without the gaps inherent in displacement-sorted images. The method requires a reference image at one motion state that has no missing data. Highly irregular breathing patterns can affect the method's performance, by introducing artifacts in the reference image (although reduced relative to phase-sorted images), or in decreased accuracy in the image prediction of motion states containing large regions of missing data.

Deep learning driven predictive treatment planning for adaptive radiotherapy of lung cancer.

BACKGROUND AND PURPOSE: To develop a novel deep learning algorithm of sequential analysis, Seq2Seq, for predicting weekly anatomical changes of lung tumor and esophagus during definitive radiotherapy, incorporate the potential tumor shrinkage into a predictive treatment planning paradigm, and improve the therapeutic ratio. METHODS AND MATERIALS: Seq2Seq starts with the primary tumor and esophagus observed on the planning CT to predict their geometric evolution during radiotherapy on a weekly basis, and subsequently updates the predictions with new snapshots acquired via weekly CBCTs. Seq2Seq is equipped with convolutional long short term memory to analyze the spatial-temporal changes of longitudinal images, trained and validated using a dataset including sixty patients. Predictive plans were optimized according to each weekly prediction and made ready for weekly deployment to mitigate the clinical burden of online weekly replanning. RESULTS: Seq2Seq tracks structural changes well: DICE between predicted and actual weekly tumor and esophagus were (0.83 ± 0.10, 0.79 ± 0.14, 0.78 ± 0.12, 0.77 ± 0.12, 0.75 ± 0.12, 0.71 ± 0.17), and (0.72 ± 0.16, 0.73 ± 0.11, 0.75 ± 0.08, 0.74 ± 0.09, 0.72 ± 0.14, 0.71 ± 0.14), respectively, while the average Hausdorff distances were within 2 mm. Evaluating dose to the actual weekly tumor and esophagus, a 4.2 Gy reduction in esophagus mean dose while maintaining 60 Gy tumor coverage was achieved with the predictive weekly plans, compared to the plan optimized using the initial tumor and esophagus alone, primarily due to noticeable tumor shrinkage during radiotherapy. CONCLUSION: It is feasible to predict the longitudinal changes of tumor and esophagus with the Seq2Seq, which could lead to improving the efficiency and effectiveness of lung adaptive radiotherapy.

Multi-class medical image segmentation using one-vs-rest graph cuts and majority voting.

Purpose: Semi-automatic image segmentation is still a valuable tool in clinical applications since it retains the expert oversights legally required. However, semi-automatic methods for simultaneous multi-class segmentation are difficult to be clinically implemented due to the complexity of underlining algorithms. We purpose an efficient one-vs-rest graph cut approach of which the complexity only grows linearly as the number of classes increases. Approach: Given an image slice, we construct multiple one-vs-rest graphs, each for a tissue class, for inference of a conditional random field (CRF). The one-vs-rest graph cut is to minimize the CRF energy derived from regional and boundary class probabilities estimated from random forests to obtain a one-vs-rest segmentation. The final segmentation is obtained by fusing from those one-vs-rest segmentations based on majority voting. We compare our method to a well-used multi-class graph cut method, alpha-beta swap, and a fully connected CRF (FCCRF) method, in brain tumor segmentation of 20 high-grade tumor cases in 2013 MICCAI dataset. Results: Our method achieved mean Dice score of 0.83 for whole tumor, compared to 0.80 by alpha-beta swap and 0.79 by FCCRF. There was a performance improvement over alpha-beta swap by a factor of five. Conclusions: Our method utilizes the probabilistic-based CRF which can be estimated from any machine learning technique. Comparing to traditional multi-class graph cut, the purposed one-vs-rest approach has complexity that grows only linearly as the number of classes increases, therefore, our method can be applicable for both online semi-automatic and offline automatic segmentation in clinical applications.

Deformation driven Seq2Seq longitudinal tumor and organs-at-risk prediction for radiotherapy.

PURPOSE: Radiotherapy presents unique challenges and clinical requirements for longitudinal tumor and organ-at-risk (OAR) prediction during treatment. The challenges include tumor inflammation/edema and radiation-induced changes in organ geometry, whereas the clinical requirements demand flexibility in input/output sequence timepoints to update the predictions on rolling basis and the grounding of all predictions in relationship to the pre-treatment imaging information for response and toxicity assessment in adaptive radiotherapy. METHODS: To deal with the aforementioned challenges and to comply with the clinical requirements, we present a novel 3D sequence-to-sequence model based on Convolution Long Short-Term Memory (ConvLSTM) that makes use of series of deformation vector fields (DVFs) between individual timepoints and reference pre-treatment/planning CTs to predict future anatomical deformations and changes in gross tumor volume as well as critical OARs. High-quality DVF training data are created by employing hyper-parameter optimization on the subset of the training data with DICE coefficient and mutual information metric. We validated our model on two radiotherapy datasets: a publicly available head-and-neck dataset (28 patients with manually contoured pre-, mid-, and post-treatment CTs), and an internal non-small cell lung cancer dataset (63 patients with manually contoured planning CT and 6 weekly CBCTs). RESULTS: The use of DVF representation and skip connections overcomes the blurring issue of ConvLSTM prediction with the traditional image representation. The mean and standard deviation of DICE for predictions of lung GTV at weeks 4, 5, and 6 were 0.83 ± 0.09, 0.82 ± 0.08, and 0.81 ± 0.10, respectively, and for post-treatment ipsilateral and contralateral parotids, were 0.81 ± 0.06 and 0.85 ± 0.02. CONCLUSION: We presented a novel DVF-based Seq2Seq model for medical images, leveraging the complete 3D imaging information of a relatively large longitudinal clinical dataset, to carry out longitudinal GTV/OAR predictions for anatomical changes in HN and lung radiotherapy patients, which has potential to improve RT outcomes.

Early Prediction of Acute Esophagitis for Adaptive Radiation Therapy.

PURPOSE: Acute esophagitis (AE) is a common dose-limiting toxicity in radiation therapy of locally advanced non-small cell lung cancer (LA-NSCLC). We developed an early AE prediction model from weekly accumulated esophagus dose and its associated local volumetric change. METHODS AND MATERIALS: Fifty-one patients with LA-NSCLC underwent treatment with intensity modulated radiation therapy to 60 Gy in 2-Gy fractions with concurrent chemotherapy and weekly cone beam computed tomography (CBCT). Twenty-eight patients (55%) developed grade ≥2 AE (≥AE2) at a median of 4 weeks after the start of radiation therapy. For early ≥AE2 prediction, the esophagus on CBCT of the first 2 weeks was deformably registered to the planning computed tomography images, and weekly esophagus dose was accumulated. Week 1-to-week 2 (w1→w2) esophagus volume changes including maximum esophagus expansion (MEex%) and volumes with ≥x% local expansions (VEx%; x = 5, 10, 15) were calculated from the Jacobian map of deformation vector field gradients. Logistic regression model with 5-fold cross-validation was built using combinations of the accumulated mean esophagus doses (MED) and the esophagus change parameters with the lowest P value in univariate analysis. The model was validated on an additional 18 and 11 patients with weekly CBCT and magnetic resonance imaging (MRI), respectively, and compared with models using only planned mean dose (MEDPlan). Performance was assessed using area under the curve (AUC) and Hosmer-Lemeshow test (PHL). RESULTS: Univariately, w1→w2 VE10% (P = .004), VE5% (P = .01) and MEex% (P = .02) significantly predicted ≥AE2. A model combining MEDW2 and w1→w2 VE10% had the best performance (AUC = 0.80; PHL = 0.43), whereas the MEDPlan model had a lower accuracy (AUC = 0.67; PHL = 0.26). The combined model also showed high accuracy in the CBCT (AUC = 0.78) and MRI validations (AUC = 0.75). CONCLUSIONS: A CBCT-based, cross-validated, and internally validated model on MRI with a combination of accumulated esophagus dose and local volume change from the first 2 weeks of chemotherapy significantly improved AE prediction compared with conventional models using only the planned dose. This model could inform plan adaptation early to lower the risk of esophagitis.

Correction of motion artifacts in cone-beam CT using a patient-specific respiratory motion model.

PURPOSE: Respiratory motion adversely affects CBCT image quality and limits its localization accuracy for image-guided radiation treatment. Motion correction methods in CBCT have focused on the thorax because of its higher soft tissue contrast, whereas low-contrast tissue in abdomen remains a challenge. The authors report on a method to correct respiration-induced motion artifacts in 1 min CBCT scans that is applicable in both thorax and abdomen, using a motion model adapted to the patient from a respiration-correlated image set. METHODS: Model adaptation consists of nonrigid image registration that maps each image to a reference image in the respiration-correlated set, followed by a principal component analysis to reduce errors in the nonrigid registration. The model parametrizes the deformation field in terms of observed surrogate (diaphragm or implanted marker) position and motion (inhalation or exhalation) between the images. In the thorax, the model is obtained from the same CBCT images that are to be motion-corrected, whereas in the abdomen, the model uses respiration-correlated CT (RCCT) images acquired prior to the treatment session. The CBCT acquisition is a single 360 degrees rotation lasting 1 min, while simultaneously recording patient breathing. The approximately 600 projection images are sorted into six (in thorax) or ten (in abdomen) subsets and reconstructed to obtain a set of low-quality respiration-correlated RC-CBCT images. Application of the motion model deforms each of the RC-CBCT images to a chosen reference image in the set; combining all images yields a single high-quality CBCT image with reduced blurring and motion artifacts. Repeated application of the model with different reference images produces a series of motion-corrected CBCT images over the respiration cycle, for determining the motion extent of the tumor and nearby organs at risk. The authors also investigate a simpler correction method, which does not use PCA and correlates motion state with respiration phase, thus assuming repeatable breathing patterns. Comparison of contrast-to-noise ratios of pixel intensities within anatomical structures relative to surrounding background tissue provides a quantitative assessment of relative organ visibility. RESULTS: Evaluation in lung phantom, two patient cases in thorax and two in upper abdomen, shows that blurring and streaking artifacts are visibly reduced with motion correction. The boundaries of tumors in the thorax, liver, and kidneys are sharper and more discernible. Repeat application of the method in one thorax case, with reference images chosen at end expiration and end inspiration, indicates its feasibility for observing tumor motion extent. Phase-based motion correction without PCA reduces blurring less effectively; in addition, implanted markers appear broken up, indicating inconsistencies in the phase-based correction. In structures showing 1 cm or more motion excursion, PCA-based motion correction shows the highest contrast-to-noise ratios in the cases examined. CONCLUSIONS: Motion correction of CBCT is feasible and yields observable improvement in the thorax and abdomen. The PCA-based model is an important component: First, by reducing deformation errors caused by the nonrigid registration and second, by relating deformation to surrogate position rather than phase, thus accommodating breathing pattern changes between imaging sessions. The accuracy of the method requires confirmation in further patient studies.

Predictive dose accumulation for HN adaptive radiotherapy.

During radiation therapy (RT) of head and neck (HN) cancer, the shape and volume of the parotid glands (PG) may change significantly, resulting in clinically relevant deviations of delivered dose from the planning dose. Early and accurate longitudinal prediction of PG anatomical changes during the RT can be valuable to inform decisions on plan adaptation. We developed a deep neural network for longitudinal predictions using the displacement fields (DFs) between the planning computed tomography (pCT) and weekly cone beam computed tomography (CBCT). Sixty-three HN patients treated with volumetric modulated arc were retrospectively studied. We calculated DFs between pCT and week 1-3 CBCT by B-spline and Demon deformable image registration (DIR). The resultant DFs were subsequently used as input to our novel network to predict the week 4 to 6 DFs for generating predicted weekly PG contours and weekly dose distributions. For evaluation, we measured dice similarity (DICE), and the uncertainty of accumulated dose. Moreover, we compared the detection accuracies of candidates for adaptive radiotherapy (ART) when the trigger criteria were mean dose difference more than 10%, 7.5%, and 5%, respectively. The DICE of ipsilateral/contralateral PG at week 4 to 6 using the prediction model trained with B-spline were 0.81 [Formula: see text] 0.07/0.81 [Formula: see text] 0.04 (week 4), 0.79 [Formula: see text] 0.06/0.81 [Formula: see text] 0.05 (week 5) and 0.78 [Formula: see text] 0.06/0.82 [Formula: see text] (week 6). The DICE with the Demons model were 0.78 [Formula: see text] 0.08/0.82 [Formula: see text] 0.03 (week 4), 0.77 [Formula: see text] 0.07/0.82 [Formula: see text] 0.04 (week 5) and 0.75 [Formula: see text] 0.07/0.82 [Formula: see text] 0.02 (week 6). The dose volume histogram (DVH) analysis with the predicted accumulated dose showed the feasibility of predicting dose uncertainty due to the PG anatomical changes. The AUC of ART candidate detection with our predictive model was over 0.90. In conclusion, the proposed network was able to predict future anatomical changes and dose uncertainty of PGs with clinically acceptable accuracy, and hence can be readily integrated into the ART workflow.

PSIGAN: Joint Probabilistic Segmentation and Image Distribution Matching for Unpaired Cross-Modality Adaptation-Based MRI Segmentation.

We developed a new joint probabilistic segmentation and image distribution matching generative adversarial network (PSIGAN) for unsupervised domain adaptation (UDA) and multi-organ segmentation from magnetic resonance (MRI) images. Our UDA approach models the co-dependency between images and their segmentation as a joint probability distribution using a new structure discriminator. The structure discriminator computes structure of interest focused adversarial loss by combining the generated pseudo MRI with probabilistic segmentations produced by a simultaneously trained segmentation sub-network. The segmentation sub-network is trained using the pseudo MRI produced by the generator sub-network. This leads to a cyclical optimization of both the generator and segmentation sub-networks that are jointly trained as part of an end-to-end network. Extensive experiments and comparisons against multiple state-of-the-art methods were done on four different MRI sequences totalling 257 scans for generating multi-organ and tumor segmentation. The experiments included, (a) 20 T1-weighted (T1w) in-phase mdixon and (b) 20 T2-weighted (T2w) abdominal MRI for segmenting liver, spleen, left and right kidneys, (c) 162 T2-weighted fat suppressed head and neck MRI (T2wFS) for parotid gland segmentation, and (d) 75 T2w MRI for lung tumor segmentation. Our method achieved an overall average DSC of 0.87 on T1w and 0.90 on T2w for the abdominal organs, 0.82 on T2wFS for the parotid glands, and 0.77 on T2w MRI for lung tumors.

Self-derived organ attention for unpaired CT-MRI deep domain adaptation based MRI segmentation.

To develop and evaluate a deep learning method to segment parotid glands from MRI using unannotated MRI and unpaired expert-segmented CT datasets. We introduced a new self-derived organ attention deep learning network for combined CT to MRI image-to-image translation (I2I) and MRI segmentation, all trained as an end-to-end network. The expert segmentations available on CT scans were combined with the I2I translated pseudo MR images to train the MRI segmentation network. Once trained, the MRI segmentation network alone is required. We introduced an organ attention discriminator that constrains the CT to MR generator to synthesize pseudo MR images that preserve organ geometry and appearance statistics as in real MRI. The I2I translation network training was regularized using the organ attention discriminator, global image-matching discriminator, and cycle consistency losses. MRI segmentation training was regularized by using cross-entropy loss. Segmentation performance was compared against multiple domain adaptation-based segmentation methods using the Dice similarity coefficient (DSC) and Hausdorff distance at the 95th percentile (HD95). All networks were trained using 85 unlabeled T2-weighted fat suppressed (T2wFS) MRIs and 96 expert-segmented CT scans. Performance upper-limit was based on fully supervised MRI training done using the 85 T2wFS MRI with expert segmentations. Independent evaluation was performed on 77 MRIs never used in training. The proposed approach achieved the highest accuracy (left parotid: DSC 0.82 ± 0.03, HD95 2.98 ± 1.01 mm; right parotid: 0.81 ± 0.05, HD95 3.14 ± 1.17 mm) compared to other methods. This accuracy was close to the reference fully supervised MRI segmentation (DSC of 0.84 ± 0.04, a HD95 of 2.24 ± 0.77 mm for the left parotid, and a DSC of 0.84 ± 0.06 and HD95 of 2.32 ± 1.37 mm for the right parotid glands).

Predicting spatial esophageal changes in a multimodal longitudinal imaging study via a convolutional recurrent neural network.

Acute esophagitis (AE) occurs among a significant number of patients with locally advanced lung cancer treated with radiotherapy. Early prediction of AE, indicated by esophageal wall expansion, is critical, as it can facilitate the redesign of treatment plans to reduce radiation-induced esophageal toxicity in an adaptive radiotherapy (ART) workflow. We have developed a novel machine learning framework to predict the patient-specific spatial presentation of the esophagus in the weeks following treatment, using magnetic resonance imaging (MRI)/ cone-beam CT (CBCT) scans acquired earlier in the 6 week radiotherapy course. Our algorithm captures the response patterns of the esophagus to radiation on a patch level, using a convolutional neural network. A recurrence neural network then parses the evolutionary patterns of the selected features in the time series, and produces a predicted esophagus-or-not label for each individual patch over future weeks. Finally, the esophagus is reconstructed, using all the predicted labels. The algorithm is trained and validated by means of ∼ 250 000 patches taken from MRI scans acquired weekly from a variety of patients, and tested using both weekly MRI and CBCT scans under a leave-one-patient-out scheme. In addition, our approach is externally validated using a publicly available dataset (Hugo 2017). Using the first three weekly scans, the algorithm can predict the condition of the esophagus over the succeeding 3 weeks with a Dice coefficient of 0.83 ± 0.04, estimate esophagus volume highly (0.98), correlated with the actual volume, using our institutional MRI/CBCT data. When evaluated using the external weekly CBCT data, the averaged Dice coefficient is 0.89 ± 0.03. Our novel algorithm may prove useful in enabling radiation oncologists to monitor and detect AE in its early stages, and could potentially play an important role in the ART decision-making process.

Quantification of accumulated dose and associated anatomical changes of esophagus using weekly Magnetic Resonance Imaging acquired during radiotherapy of locally advanced lung cancer.

BACKGROUND AND PURPOSE: Minimizing acute esophagitis (AE) in locally advanced non-small cell lung cancer (LA-NSCLC) is critical given the proximity between the esophagus and the tumor. In this pilot study, we developed a clinical platform for quantification of accumulated doses and volumetric changes of esophagus via weekly Magnetic Resonance Imaging (MRI) for adaptive radiotherapy (RT). MATERIAL AND METHODS: Eleven patients treated via intensity-modulated RT to 60-70 Gy in 2-3 Gy-fractions with concurrent chemotherapy underwent weekly MRIs. Eight patients developed AE grade 2 (AE2), 3-6 weeks after RT started. First, weekly MRI esophagus contours were rigidly propagated to planning CT and the distances between the medial esophageal axes were calculated as positional uncertainties. Then, the weekly MRI were deformably registered to the planning CT and the total dose delivered to esophagus was accumulated. Weekly Maximum Esophagus Expansion (MEex) was calculated using the Jacobian map. Eventually, esophageal dose parameters (Mean Esophagus Dose (MED), V90% and D5cc) between the planned and accumulated dose were compared. RESULTS: Positional esophagus uncertainties were 6.8 ± 1.8 mm across patients. For the entire cohort at the end of RT: the median accumulated MED was significantly higher than the planned dose (24 Gy vs. 21 Gy p = 0.006). The median V90% and D5cc were 12.5 cm3 vs. 11.5 cm3 (p = 0.05) and 61 Gy vs. 60 Gy (p = 0.01), for accumulated and planned dose, respectively. The median MEex was 24% and was significantly associated with AE2 (p = 0.008). CONCLUSIONS: MRI is well suited for tracking esophagus volumetric changes and accumulating doses. Longitudinal esophagus expansion could reflect radiation-induced inflammation that may link to AE.

Library of deep-learning image segmentation and outcomes model-implementations.

An open-source library of implementations for deep-learning-based image segmentation and outcomes models based on radiotherapy and radiomics is presented. As oncology treatment planning becomes increasingly driven by automation, such a library of model implementations is crucial to (i) validate existing models on datasets collected at different institutions, (ii) automate segmentation, (iii) create ensembles for improving performance and (iv) incorporate validated models in the clinical workflow. Inclusion of deep-learning-based image segmentation and outcomes models in the same library provides a fully automated and reproduceable pipeline to estimate prognosis. The library was developed with the Computational Environment for Radiological Research (CERR) software platform. Centralizing model implementations in CERR builds upon its rich set of radiotherapy and radiomics tools and caters to the world-wide user base. CERR provides well-validated feature extraction pipelines for radiotherapy dosimetry and radiomics with fine control over the calculation settings, allowing users to select appropriate parameters used in model derivation. Models for automatic image segmentation are distributed via containers, allowing them to be deployed with a variety of scientific computing architectures. The library includes implementations of popular DVH-based models outlined in the Quantitative Analysis of Normal Tissue Effects in the Clinic effort and recently published literature. Radiomics models include features from the Image Biomarker Standardization Initiative and application-specific features found to be relevant across multiple sites and image modalities. The library is distributed as a module within CERR at https://www.github.com/cerr/CERR under the GNU-GPL copyleft with additional restrictions on clinical and commercial use and provision to dual license in future.