A radiotherapy community data-driven approach to determine which complexity metrics best predict the impact of atypical TPS beam modeling on clinical dose calculation accuracy.

PURPOSE: To quantify the impact of treatment planning system beam model parameters, based on the actual spread in radiotherapy community data, on clinical treatment plans and determine which complexity metrics best describe the impact beam modeling errors have on dose accuracy. METHODS: Ten beam modeling parameters for a Varian accelerator were modified in RayStation to match radiotherapy community data at the 2.5, 25, 50, 75, and 97.5 percentile levels. These modifications were evaluated on 25 patient cases, including prostate, non-small cell lung, H&N, brain, and mesothelioma, generating 1,000 plan perturbations. Differences in the mean planned dose to clinical target volumes (CTV) and organs at risk (OAR) were evaluated with respect to the planned dose using the reference (50th-percentile) parameter values. Correlation between CTV dose differences, and 18 different complexity metrics were evaluated using linear regression; R-squared values were used to determine the best metric. RESULTS: Perturbations to MLC offset and transmission parameters demonstrated the greatest changes in dose: up to 5.7% in CTVs and 16.7% for OARs. More complex clinical plans showed greater dose perturbation with atypical beam model parameters. The mean MLC Gap and Tongue & Groove index (TGi) complexity metrics best described the impact of TPS beam modeling variations on clinical dose delivery across all anatomical sites; similar, though not identical, trends between complexity and dose perturbation were observed among all sites. CONCLUSION: Extreme values for MLC offset and MLC transmission beam modeling parameters were found to most substantially impact the dose distribution of clinical plans and careful attention should be given to these beam modeling parameters. The mean MLC Gap and TGi complexity metrics were best suited to identifying clinical plans most sensitive to beam modeling errors; this could help provide focus for clinical QA in identifying unacceptable plans.

Predicted Inferior Outcomes for Lung SBRT With Treatment Planning Systems That Fail Independent Phantom-Based Audits.

PURPOSE: More than 15% of radiation therapy clinics fail external audits with anthropomorphic phantoms conducted by Imaging and Radiation Oncology Core-Houston (IROC-H) while passing other industry-standard quality assurance (QA) tests. We seek to evaluate the predicted effect of such failed plans on outcomes for patients treated with stereotactic body radiation therapy (SBRT) for lung tumors. METHODS AND MATERIALS: We conducted a retrospective study of 55 patients treated with SBRT for lung tumors with a prescription biologically equivalent dose (BED)10 ≥100 Gy using a treatment planning system (TPS) that passed IROC-H phantom audits. Sample linear accelerator beam models with introduced errors were commissioned by varying the multileaf collimator leaf-tip offset parameter (ie, dosimetric leaf gap) over the range ±1.0 mm relative to the validated model. These models mimic TPS that pass internal QA measures but fail IROC-H tests. Patient plans were recalculated on sample beam models. The predicted tumor control probability (TCP) and normal tissue complication probability (NTCP) were calculated based on published dose-response models. RESULTS: A leaf-tip offset value of -1.0 mm decreased the fraction of plans receiving a planning treatment volume of BED10 ≥100 Gy from 95% to 27%. This translated to a significant decrease in 2-year TCP of 4.8% (95% CI: 2.0%-5.5%) with a decrease in TCP up to 21%. Conversely, a leaf-tip offset of +1.0 mm resulted in 36% of patients exceeding previously met organs at risk (OAR) constraints, including 2 instances of spinal cord and brachial plexus overdoses and a small increase in chest wall NTCP of 0.7%, (95% CI: 0.5%-0.8%). CONCLUSIONS: Simulated treatment plans with modest MLC leaf offsets result in lung SBRT plans that significantly underdose tumor or exceed OAR constraints. These dosimetric endpoints translate to significant detriments in TCP. These simulated plans mimic planning systems that pass internal QA measures but fail independent phantom-based tests, underscoring the need for enhanced quality assurance including external audits of TPS.

The current status and shortcomings of stereotactic radiosurgery.

Background: Stereotactic radiosurgery (SRS) is a common treatment for intracranial lesions. This work explores the state of SRS treatment delivery to characterize current treatment accuracy based on treatment parameters. Methods: NCI clinical trials involving SRS rely on an end-to-end treatment delivery on a patient surrogate (credentialing phantom) from the Imaging and Radiation Oncology Core (IROC) to test their treatment accuracy. The results of 1072 SRS phantom irradiations between 2012 and 2020 were retrospectively analyzed. Univariate analysis and random forest models were used to associate irradiation conditions with phantom performance. The following categories were evaluated in terms of how they predicted outcomes: year of irradiation, TPS algorithm, machine model, energy, and delivered field size. Results: Overall, only 84.6% of irradiations have met the IROC/NCI acceptability criteria. Pass rate has remained constant over time, while dose calculation accuracy has slightly improved. Dose calculation algorithm (P < .001), collimator (P = .024), and field size (P < .001) were statistically significant predictors of pass/fail. Specifically, pencil beam algorithms and cone collimators were more likely to be associated with failing phantom results. Random forest modeling identified the size of the field as the most important factor for passing or failing followed by algorithm. Conclusion: Constant throughout this retrospective study, approximately 15% of institutions fail to meet IROC/NCI standards for SRS treatment. In current clinical practice, this is particularly associated with smaller fields that yielded less accurate results. There is ongoing need to improve small field dosimetry, beam modeling, and QA to ensure high treatment quality, patient safety, and optimal clinical trials.

Impact of lung block shape on cardiac dose for total body irradiation.

Evaluating cardiac dose during total body irradiation (TBI) is of increasing interest. A three-dimensional beam model for TBI was commissioned and lung shielding was simulated in a treatment planning system with the cardiac silhouette partially blocked and unblocked. When blocked, the median heart dose decreased by 6% (IQR = 6%) and the median cardiac V12Gy decreased by 27% (IQR = 17%). The median left anterior descending artery dose decreased 20% (IQR = 12%) for blocked cases. Because using partial heart shielding may result in considerable changes in dose to cardiac structures, TBI protocols should explicitly consider lung block design parameters and their potential effects.

Effect of total body irradiation lung block parameters on lung doses using three-dimensional dosimetry.

PURPOSE: Total body irradiation (TBI) is an integral part of stem cell transplant. However, patients are at risk of treatment-related toxicities, including radiation pneumonitis. While lung dose is one of the most crucial aspects of TBI dosimetry, currently available data are based on point doses. As volumetric dose distribution could be substantially altered by lung block parameters, we used 3D dosimetry in our treatment planning system to estimate volumetric lung dose and measure the impact of various lung block designs. MATERIALS AND METHODS: We commissioned a TBI beam model in RayStation that matches the measured tissue-phantom ratio under our clinical TBI setup. Cerrobend blocks were automatically generated in RayStation on thoracic Computed Tomography (CT) scans from three anonymized patients using the lung, clavicle, spine, and diaphragmatic contours. The margin for block edge was varied to 0, 1, or 2 cm from the superior, lateral, and inferior thoracic borders, with a uniform margin 2.5 cm lateral to the vertebral bodies. The lung dose was calculated and compared with a prescription dose of 1200 cGy in six fractions (three with blocks and three without). RESULT: The point dose at midplane under the block and the average lung dose are at the range of 73%-76% and 80%-88% of prescription dose respectively regardless of the block margins. In contrast, the percent lung volume receiving 10 Gy increased by nearly two-fold, from 31% to 60% over the margins from 0 to 2 cm. CONCLUSIONS: The TPS-derived 3D lung dose is substantially different from the nominal dose assumed with HVL lung blocks. Point doses under the block are insufficient to accurately gauge the relationship between dose and pneumonitis, and TBI dosimetry could be highly variable between patients and institutions as more descriptive parameters are not included in protocols. Much progress remains to be made to optimize and standardize technical aspects of TBI, and better dosimetry could provide more precise dosimetric predictors for pneumonitis risk.

Photon beam modeling variations predict errors in IMRT dosimetry audits.

BACKGROUND & PURPOSE: To evaluate treatment planning system (TPS) beam modeling parameters as contributing factors to IMRT audit performance. MATERIALS & METHODS: We retrospectively analyzed IROC Houston phantom audit performance and concurrent beam modeling survey responses from 337 irradiations performed between August 2017 and November 2019. Irradiation results were grouped based on the reporting of typical or atypical beam modeling parameter survey responses (<10th or >90th percentile values), and compared for passing versus failing (>7% error) or "poor" (>5% error) irradiation status. Additionally, we assessed the impact on the planned dose distribution from variations in modeling parameter value. Finally, we estimated the overall impact of beam modeling parameter variance on dose calculations, based on reported community variations. RESULTS: Use of atypical modeling parameters were more frequently seen with failing phantom audit results (p = 0.01). Most pronounced was for Eclipse AAA users, where phantom irradiations with atypical values of dosimetric leaf gap (DLG) showed a greater incidence of both poor-performing (p = 0.048) and failing phantom audits (p = 0.014); and in general, DLG value was correlated with dose calculation accuracy (r = 0.397, p < 0.001). Manipulating TPS parameters induced systematic changes in planned dose distributions which were consistent with prior observations of how failures manifest. Dose change estimations based on these dose calculations agreed well with true dosimetric errors identified. CONCLUSION: Atypical TPS beam modeling parameters are associated with failing phantom audits. This is identified as an important factor contributing to the observed failing phantom results, and highlights the need for accurate beam modeling.

Sensitivity of IROC phantom performance to radiotherapy treatment planning system beam modeling parameters based on community-driven data.

PURPOSE: Treatment planning system (TPS) dose calculations have previously been shown to be sensitive to modeling errors, especially when treating with complex strategies like intensity-modulated radiation therapy (IMRT). This work investigates the dosimetric impact of several dosimetric and nondosimetric beam modeling parameters, based on their distribution in the radiotherapy community, in two commercial TPSs in order to understand the realistic potential for dose deviations and their clinical effects. METHODS AND MATERIALS: Beam models representing standard 120-leaf Varian Clinac-type machines were developed in Eclipse 13.5 (AAA algorithm) and RayStation 9A (v8.99, collapsed-cone algorithm) based upon median values of dosimetric measurements from Imaging and Radiation Oncology Core (IROC) Houston site visit data and community beam modeling parameter survey data in order to represent a baseline linear accelerator. Five clinically acceptable treatment plans (three IMRT, two VMAT) were developed for the IROC head and neck phantom. Dose distributions for each plan were recalculated after individually modifying parameters of interest (e.g., MLC transmission, percent depth doses [PDDs], and output factors) according to the 2.5th to 97.5th percentiles of community survey and machine performance data to encompass the realistic extent of variance in the radiotherapy community. The resultant dose distributions were evaluated by examining relative changes in average dose for thermoluminescent dosimeter (TLD) locations across the two target volumes and organ at risk (OAR). Interplay was also examined for parameters generating changes in target dose greater than 1%. RESULTS: For Eclipse, dose calculations were sensitive to changes in the dosimetric leaf gap (DLG), which resulted in differences from -5% to +3% to the targets relative to the baseline beam model. Modifying the MLC transmission factor introduced differences up to ± 1%. For RayStation, parameters determining MLC behaviors likewise contributed substantially; the MLC offset introduced changes in dose from -4% to +7%, and the MLC transmission caused changes of -4% to +2%. Among the dosimetric qualities examined, changes in PDD implementation resulted in the most substantial changes, but these were only up to ±1%. Other dosimetric factors had <1% impact on dose accuracy. Interplay between impactful parameters was found to be minimal. CONCLUSION: Factors related to the modeling of the MLC, particularly relating to the leaf offset, can cause clinically significant changes in the calculated dose for IMRT and VMAT plans. This should be of concern to the radiotherapy community because the clinical effects of poor TPS commissioning were based on reported data from clinically implemented beam models. These results further reinforce that dose errors caused by poor TPS calculations are often involved in IROC phantom failures.

Dose calculation errors as a component of failing IROC lung and spine phantom irradiations.

PURPOSE: Between July 2013 and August 2019, 22% of the imaging and radiation oncology core (IROC) spine, and 15% of the moving lung phantom irradiations have failed to meet established acceptability criteria. The spine phantom simulates a highly modulated stereotactic body radiation therapy (SBRT) case, whereas the lung phantom represents a low-to-none modulation moving target case. In this study, we assessed the contribution of dose calculation errors to these phantom results and evaluated their effects on failure rates. METHODS: We evaluated dose calculation errors by comparing the calculation accuracy of various institutions' treatment planning systems (TPSs) vs IROC-Houston's previously established independent dose recalculation system (DRS). Each calculation was compared with the measured dose actually delivered to the phantom; cases in which the recalculation was more accurate were interpreted as a deficiency in the institution's TPS. A total of 258 phantom irradiation plans (172 lung and 86 spine) were recomputed. RESULTS: Overall, the DRS performed better than the TPSs in 47% of the spine phantom cases. However, the DRS was more accurate in 93% of failing spine phantom cases (with an average improvement of 2.35%), indicating a deficiency in the institution's treatment planning system. Deficiencies in dose calculation accounted for 60% of the overall discrepancy between measured and planned doses among spine phantoms. In contrast, lung phantom DRS calculations were more accurate in only 35% and 42% of all and failing lung phantom cases respectively, indicating that dose calculation errors were not substantially present. These errors accounted for only 30% of the overall discrepancy between measured and planned doses. CONCLUSIONS: Dose calculation errors are common and substantial in IROC spine phantom irradiations, highlighting a major failure mode in this phantom and in clinical treatment management of these cases. In contrast, dose calculation accuracy had only a minimal contribution to failing lung phantom results, indicating that other failure modes drive problems with this phantom and similar clinical treatments.

Reference dataset of users' photon beam modeling parameters for the Eclipse, Pinnacle, and RayStation treatment planning systems.

PURPOSE: The aim of this work was to provide a novel description of how the radiotherapy community configures treatment planning system (TPS) radiation beam models for clinically used treatment machines. Here we describe the results of a survey of self-reported TPS beam modeling parameter values across different C-arm linear accelerators, beam energies, and multileaf collimator (MLC) configurations. ACQUISITION AND VALIDATION METHODS: Beam modeling data were acquired via electronic survey implemented through the Imaging and Radiation Oncology Core (IROC) Houston Quality Assurance Center's online facility questionnaire. The survey was open to participation from January 2018 through January 2019 for all institutions monitored by IROC. After quality control, 2818 beam models were collected from 642 institutions. This survey, designed for Eclipse, Pinnacle, and RayStation, instructed physicists to report parameter values used to model the radiation source and MLC for each treatment machine and beam energy used clinically for intensity-modulated radiation therapy. Parameters collected included the effective source/spot size, MLC transmission, dosimetric leaf gap, tongue and groove effect, and other nondosimetric parameters specific to each TPS. To facilitate survey participation, instructions were provided on how to identify requested beam modeling parameters within each TPS environment. DATA FORMAT AND USAGE NOTES: Numeric values of the beam modeling parameters are compiled and tabulated according to TPS and calculation algorithm, linear accelerator model class, beam energy, and MLC configuration. Values are also presented as distributions, ranging from the 2.5th to the 97.5th percentile. POTENTIAL APPLICATIONS: These data provide an independent guide describing how the radiotherapy community mathematically represents its clinical radiation beams. These distributions may be used by the community for comparison during the commissioning or verification of their TPS beam models. Ultimately, we hope that the current work will allow institutions to spot potentially suspicious parameter values and help ensure more accurate radiotherapy delivery.

Independent recalculation outperforms traditional measurement-based IMRT QA methods in detecting unacceptable plans.

PURPOSE: To evaluate the performance of an independent recalculation and compare it against current measurement-based patient specific intensity-modulated radiation therapy (IMRT) quality assurance (QA) in predicting unacceptable phantom results as measured by the Imaging and Radiation Oncology Core (IROC). METHODS: When institutions irradiate the IROC head and neck IMRT phantom, they are also asked to submit their internal IMRT QA results. Separately from this, IROC has previously created reference beam models on the Mobius3D platform to independently recalculate phantom results based on the institution's DICOM plan data. The ability of the institutions' IMRT QA to predict the IROC phantom result was compared against the independent recalculation for 339 phantom results collected since 2012. This was done to determine the ability of these systems to detect failing phantom results (i.e., large errors) as well as poor phantom results (i.e., modest errors). Sensitivity and specificity were evaluated using common clinical thresholds, and receiver operator characteristic (ROC) curves were used to compare across different thresholds. RESULTS: Overall, based on common clinical criteria, the independent recalculation was 12 times more sensitive at detecting unacceptable (failing) IROC phantom results than clinical measurement-based IMRT QA. The recalculation was superior, in head-to-head comparison, to the EPID, ArcCheck, and MapCheck devices. The superiority of the recalculation vs these array-based measurements persisted under ROC analysis as the recalculation curve had a greater area under it and was always above that for these measurement devices. For detecting modest errors (poor phantom results rather than failing phantom results), neither the recalculation nor measurement-based IMRT QA performed well. CONCLUSIONS: A simple recalculation outperformed current measurement-based IMRT QA methods at detecting unacceptable plans. These findings highlight the value of an independent recalculation, and raise further questions about the current standard of measurement-based IMRT QA.

Treatment plan complexity does not predict IROC Houston anthropomorphic head and neck phantom performance.

Previous works indicate that intensity-modulated radiotherapy (IMRT) and volumetric modulated arc therapy (VMAT) plans that are highly complex may produce more errors in dose calculation and treatment delivery. Multiple complexity metrics have been proposed and associated with IMRT QA results, but their relationships with plan performance using in situ dose measurements have not been thoroughly investigated. This study aimed to evaluate the relationships between IMRT treatment plan complexity and anthropomorphic phantom performance in order to assess the extent to which plan complexity is related to dosimetric performance in the IROC phantom credentialing program. Sixteen complexity metrics, including the modulation complexity score (MCS), several modulation indices, and total monitor units (MU) delivered, were evaluated for 343 head and neck phantom irradiations, comprising both IMRT (step-and-shoot and sliding window techniques) and VMAT. Spearman's correlations were used to explore the relationship between complexity and plan performance, as measured by the dosimetric differences between the treatment planning system (TPS) and thermoluminescent dosimeter (TLD) measurement, as well as film gamma analysis. Relationships were likewise determined for several combinations of subpopulations, based on the linear accelerator model, TPS used, and delivery modality. Evaluation of the complexity metrics presented here yielded no significant relationships (p > 0.01, Bonferroni-corrected) and all correlations were weak (less than ±0.30). These results indicate that complexity metrics have limited predictive utility in assessing plan performance in multi-institutional comparisons of IMRT plans. Other factors affecting plan accuracy, such as dosimetric modeling or multileaf collimator (MLC) performance, should be investigated to determine a more probable cause for dose delivery errors.