Machine learning-based automatic proton therapy planning: Impact of post-processing and dose-mimicking in plan robustness.

PURPOSE: Automated treatment planning strategies are being widely implemented in clinical routines to reduce inter-planner variability, speed up the optimization process, and improve plan quality. This study aims to evaluate the feasibility and quality of intensity-modulated proton therapy (IMPT) plans generated with four different knowledge-based planning (KBP) pipelines fully integrated into a commercial treatment planning system (TPS). MATERIALS/METHODS: A data set containing 60 oropharyngeal cancer patients was split into 11 folds, each containing 47 patients for training, five patients for validation, and five patients for testing. A dose prediction model was trained on each of the folds, resulting in a total of 11 models. Three patients were left out in order to assess if the differences introduced between models were significant. From voxel-based dose predictions, we analyze the two steps that follow the dose prediction: post-processing of the predicted dose and dose mimicking (DM). We focused on the effect of post-processing (PP) or no post-processing (NPP) combined with two different DM algorithms for optimization: the one available in the commercial TPS RayStation (RSM) and a simpler isodose-based mimicking (IBM). Using 55 test patients (five test patients for each model), we evaluated the quality and robustness of the plans generated by the four proposed KBP pipelines (PP-RSM, PP-IBM, NPP-RSM, NPP-IBM). After robust evaluation, dose-volume histogram (DVH) metrics in nominal and worst-case scenarios were compared to those of the manually generated plans. RESULTS: Nominal doses from the four KBP pipelines showed promising results achieving comparable target coverage and improved dose to organs at risk (OARs) compared to the manual plans. However, too optimistic post-processing applied to the dose prediction (i.e. important decrease of the dose to the organs) compromised the robustness of the plans. Even though RSM seemed to partially compensate for the lack of robustness in the PP plans, still 65% of the patients did not achieve the expected robustness levels. NPP-RSM plans seemed to achieve the best trade-off between robustness and OAR sparing. DISCUSSION/CONCLUSIONS: PP and DM strategies are crucial steps to generate acceptable robust and deliverable IMPT plans from ML-predicted doses. Before the clinical implementation of any KBP pipeline, the PP and DM parameters predefined by the commercial TPS need to be modified accordingly with a comprehensive feedback loop in which the robustness of the final dose calculations is evaluated. With the right choice of PP and DM parameters, KBP strategies have the potential to generate IMPT plans within clinically acceptable levels comparable to plans manually generated by dosimetrists.

OpenKBP-Opt: an international and reproducible evaluation of 76 knowledge-based planning pipelines.

Objective.To establish an open framework for developing plan optimization models for knowledge-based planning (KBP).Approach.Our framework includes radiotherapy treatment data (i.e. reference plans) for 100 patients with head-and-neck cancer who were treated with intensity-modulated radiotherapy. That data also includes high-quality dose predictions from 19 KBP models that were developed by different research groups using out-of-sample data during the OpenKBP Grand Challenge. The dose predictions were input to four fluence-based dose mimicking models to form 76 unique KBP pipelines that generated 7600 plans (76 pipelines × 100 patients). The predictions and KBP-generated plans were compared to the reference plans via: the dose score, which is the average mean absolute voxel-by-voxel difference in dose; the deviation in dose-volume histogram (DVH) points; and the frequency of clinical planning criteria satisfaction. We also performed a theoretical investigation to justify our dose mimicking models.Main results.The range in rank order correlation of the dose score between predictions and their KBP pipelines was 0.50-0.62, which indicates that the quality of the predictions was generally positively correlated with the quality of the plans. Additionally, compared to the input predictions, the KBP-generated plans performed significantly better (P< 0.05; one-sided Wilcoxon test) on 18 of 23 DVH points. Similarly, each optimization model generated plans that satisfied a higher percentage of criteria than the reference plans, which satisfied 3.5% more criteria than the set of all dose predictions. Lastly, our theoretical investigation demonstrated that the dose mimicking models generated plans that are also optimal for an inverse planning model.Significance.This was the largest international effort to date for evaluating the combination of KBP prediction and optimization models. We found that the best performing models significantly outperformed the reference dose and dose predictions. In the interest of reproducibility, our data and code is freely available.

Three-dimensional dose prediction for lung IMRT patients with deep neural networks: robust learning from heterogeneous beam configurations.

PURPOSE: The use of neural networks to directly predict three-dimensional dose distributions for automatic planning is becoming popular. However, the existing methods use only patient anatomy as input and assume consistent beam configuration for all patients in the training database. The purpose of this work was to develop a more general model that considers variable beam configurations in addition to patient anatomy to achieve more comprehensive automatic planning with a potentially easier clinical implementation, without the need to train specific models for different beam settings. METHODS: The proposed anatomy and beam (AB) model is based on our newly developed deep learning architecture, and hierarchically densely connected U-Net (HD U-Net), which combines U-Net and DenseNet. The AB model contains 10 input channels: one for beam setup and the other 9 for anatomical information (PTV and organs). The beam setup information is represented by a 3D matrix of the non-modulated beam's eye view ray-tracing dose distribution. We used a set of images from 129 patients with lung cancer treated with IMRT with heterogeneous beam configurations (4-9 beams of various orientations) for training/validation (100 patients) and testing (29 patients). Mean squared error was used as the loss function. We evaluated the model's accuracy by comparing the mean dose, maximum dose, and other relevant dose-volume metrics for the predicted dose distribution against those of the clinically delivered dose distribution. Dice similarity coefficients were computed to address the spatial correspondence of the isodose volumes between the predicted and clinically delivered doses. The model was also compared with our previous work, the anatomy only (AO) model, which does not consider beam setup information and uses only 9 channels for anatomical information. RESULTS: The AB model outperformed the AO model, especially in the low and medium dose regions. In terms of dose-volume metrics, AB outperformed AO by about 1-2%. The largest improvement was found to be about 5% in lung volume receiving a dose of 5Gy or more (V5 ). The improvement for spinal cord maximum dose was also important, that is, 3.6% for cross-validation and 2.6% for testing. The AB model achieved Dice scores for isodose volumes as much as 10% higher than the AO model in low and medium dose regions and about 2-5% higher in high dose regions. CONCLUSIONS: The AO model, which does not use beam configuration as input, can still predict dose distributions with reasonable accuracy in high dose regions but introduces large errors in low and medium dose regions for IMRT cases with variable beam numbers and orientations. The proposed AB model outperforms the AO model substantially in low and medium dose regions, and slightly in high dose regions, by considering beam setup information through a cumulative non-modulated beam's eye view ray-tracing dose distribution. This new model represents a major step forward towards predicting 3D dose distributions in real clinical practices, where beam configuration could vary from patient to patient, from planner to planner, and from institution to institution.

Performance of a hybrid Monte Carlo-Pencil Beam dose algorithm for proton therapy inverse planning.

PURPOSE: Analytical algorithms have a limited accuracy when modeling very heterogeneous tumor sites. This work addresses the performance of a hybrid dose optimizer that combines both Monte Carlo (MC) and pencil beam (PB) dose engines to get the best trade-off between speed and accuracy for proton therapy plans. METHODS: The hybrid algorithm calculates the optimal spot weights (w) by means of an iterative optimization process where the dose at each iteration is computed by using a precomputed dose influence matrix based on the conventional PB plus a correction term c obtained from a MC simulation. Updates of c can be triggered as often as necessary by calling the MC dose engine with the last corrected values of w as input. In order to analyze the performance of the hybrid algorithm against dose calculation errors, it was applied to a simplistic water phantom for which several test cases with different errors were simulated, including proton range uncertainties. Afterwards, the algorithm was used in three clinical cases (prostate, lung, and brain) and benchmarked against full MC-based optimization. The influence of different stopping criteria in the final results was also investigated. RESULTS: The hybrid algorithm achieved excellent results provided that the estimated range in a homogeneous material is the same for the two dose engines involved, i.e., PB and MC. For the three patient cases, the hybrid plans were clinically equivalent to those obtained with full MC-based optimization. Only a single update of c was needed in the hybrid algorithm to fulfill the clinical dose constraints, which represents an extra computation time to obtain c that ranged from 1 (brain) to 4 min (lung) with respect to the conventional PB-based optimization, and an estimated average gain factor of 14 with respect to full MC-based optimization. CONCLUSION: The hybrid algorithm provides an improved trade-off between accuracy and speed. This algorithm can be immediately considered as an option for improving dose calculation accuracy of commercial analytical treatment planning systems, without a significant increase in the computation time (≪5 min) with respect to current PB-based optimization.

Validation and application of a fast Monte Carlo algorithm for assessing the clinical impact of approximations in analytical dose calculations for pencil beam scanning proton therapy.

PURPOSE: Monte Carlo (MC) dose calculation is generally superior to analytical dose calculation (ADC) used in commercial TPS to model the dose distribution especially for heterogeneous sites, such as lung and head/neck patients. The purpose of this study was to provide a validated, fast, and open-source MC code, MCsquare, to assess the impact of approximations in ADC on clinical pencil beam scanning (PBS) plans covering various sites. METHODS: First, MCsquare was validated using tissue-mimicking IROC lung phantom measurements as well as benchmarked with the general purpose Monte Carlo TOPAS for patient dose calculation. Then a comparative analysis between MCsquare and ADC was performed for a total of 50 patients with 10 patients per site (including liver, pelvis, brain, head-and-neck, and lung). Differences among TOPAS, MCsquare, and ADC were evaluated using four dosimetric indices based on the dose-volume histogram (target Dmean, D95, homogeneity index, V95), a 3D gamma index analysis (using 3%/3 mm criteria), and estimations of tumor control probability (TCP). RESULTS: Comparison between MCsquare and TOPAS showed less than 1.8% difference for all of the dosimetric indices/TCP values and resulted in a 3D gamma index passing rate for voxels within the target in excess of 99%. When comparing ADC and MCsquare, the variances of all the indices were found to increase as the degree of tissue heterogeneity increased. In the case of lung, the D95s for ADC were found to differ by as much as 6.5% from the corresponding MCsquare statistic. The median gamma index passing rate for voxels within the target volume decreased from 99.3% for liver to 75.8% for lung. Resulting TCP differences can be large for lung (≤10.5%) and head-and-neck (≤6.2%), while smaller for brain, pelvis and liver (≤1.5%). CONCLUSIONS: Given the differences found in the analysis, accurate dose calculation algorithms such as Monte Carlo simulations are needed for proton therapy, especially for disease sites with high heterogeneity, such as head-and-neck and lung. The establishment of MCsquare can facilitate patient plan reviews at any institution and can potentially provide unbiased comparison in clinical trials given its accuracy, speed and open-source availability.