Associations Between Radiation Oncologist Demographic Factors and Segmentation Similarity Benchmarks: Insights From a Crowd-Sourced Challenge Using Bayesian Estimation.

PURPOSE: The quality of radiotherapy auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of clinician-derived segmentations are poorly understood; our study aims to quantify these factors. METHODS: Organ at risk (OAR) and tumor-related segmentations provided by radiation oncologists from the Contouring Collaborative for Consensus in Radiation Oncology data set were used. Segmentations were derived from five disease sites: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and GI. Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus, which served as a reference standard benchmark. The Dice similarity coefficient (DSC) was primarily used as a metric for the comparisons. DSC was stratified into binary groups on the basis of structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Bayesian estimation were used to investigate the association between demographic variables and the binarized DSC for each disease site. Variables with a highest density interval excluding zero were considered to substantially affect the outcome measure. RESULTS: Five hundred seventy-four, 110, 452, 112, and 48 segmentations were used for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of segmentations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumors, respectively. Regression analysis revealed that the structure being tumor-related had a substantial negative impact on binarized DSC for the breast, sarcoma, H&N, and GI cases. There were no recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations. CONCLUSION: Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality relative to benchmarks.

Intra-operative extracorporeal irradiation of tumour-invaded craniotomy bone flap in meningioma: a case series.

BACKGROUND: Extracorporeal irradiation of tumorous calvaria (EITC) can be performed to restore function and form of the skull after resection of bone-invasive meningioma. We sought to examine the rate of tumour recurrence and other selected outcomes in patients undergoing meningioma resection and EITC. METHODS: Retrospective single-centre study of adult patients undergoing meningioma resection and EITC between January 2015 and November 2022 at a tertiary neurosurgical centre. Patient demographics, surgery data, tumour data, use of adjuvant therapy, surgical complications, and tumour recurrences were collected. RESULTS: Eighteen patients with 11 (61%) CNS WHO grade 1, 6 (33%) grade 2, and 1 (6%) grade 3 meningiomas were included. Median follow-up was 42 months (range 3-88). Five (28%) patients had a recurrence, but none were associated with the bone flap. Two (11%) wound infections requiring explant surgery occurred. Six (33%) patients required a further operation. Two operations were for recurrences, one was for infection, one was a washout and wound exploration but no evidence of infection was found, one patient requested the removal of a small titanium implant, and one patient required a ventriculoperitoneal shunt for a persistent CSF collection. There were no cases of bone flap resorption and cosmetic outcome was not routinely recorded. CONCLUSION: EITC is feasible and fast to perform with good outcomes and cost-effectiveness compared to other reconstructive methods. We observed similar recurrence rates and lower infection rates requiring explant compared to the largest series of cranioplasty in meningioma. Cosmetic outcome is universally under-reported and should be reported in future studies.

Determining The Role Of Radiation Oncologist Demographic Factors On Segmentation Quality: Insights From A Crowd-Sourced Challenge Using Bayesian Estimation.

BACKGROUND: Medical image auto-segmentation is poised to revolutionize radiotherapy workflows. The quality of auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of these clinician-derived segmentations have yet to be fully understood or quantified. Therefore, the purpose of this study was to determine the role of common observer demographic variables on quantitative segmentation performance. METHODS: Organ at risk (OAR) and tumor volume segmentations provided by radiation oncologist observers from the Contouring Collaborative for Consensus in Radiation Oncology public dataset were utilized for this study. Segmentations were derived from five separate disease sites comprised of one patient case each: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and gastrointestinal (GI). Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus gold standard primarily using the Dice Similarity Coefficient (DSC); surface DSC was investigated as a secondary metric. Metrics were stratified into binary groups based on previously established structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Markov chain Monte Carlo Bayesian estimation were used to investigate the association between demographic variables and the binarized segmentation quality for each disease site separately. Variables with a highest density interval excluding zero - loosely analogous to frequentist significance - were considered to substantially impact the outcome measure. RESULTS: After filtering by practicing radiation oncologists, 574, 110, 452, 112, and 48 structure observations remained for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of observations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumor volumes, respectively. Bayesian regression analysis revealed tumor category had a substantial negative impact on binarized DSC for the breast (coefficient mean ± standard deviation: -0.97 ± 0.20), sarcoma (-1.04 ± 0.54), H&N (-1.00 ± 0.24), and GI (-2.95 ± 0.98) cases. There were no clear recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations and wide highest density intervals. CONCLUSION: Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality. Future studies should investigate additional demographic variables, more patients and imaging modalities, and alternative metrics of segmentation acceptability.

Large scale crowdsourced radiotherapy segmentations across a variety of cancer anatomic sites.

Clinician generated segmentation of tumor and healthy tissue regions of interest (ROIs) on medical images is crucial for radiotherapy. However, interobserver segmentation variability has long been considered a significant detriment to the implementation of high-quality and consistent radiotherapy dose delivery. This has prompted the increasing development of automated segmentation approaches. However, extant segmentation datasets typically only provide segmentations generated by a limited number of annotators with varying, and often unspecified, levels of expertise. In this data descriptor, numerous clinician annotators manually generated segmentations for ROIs on computed tomography images across a variety of cancer sites (breast, sarcoma, head and neck, gynecologic, gastrointestinal; one patient per cancer site) for the Contouring Collaborative for Consensus in Radiation Oncology challenge. In total, over 200 annotators (experts and non-experts) contributed using a standardized annotation platform (ProKnow). Subsequently, we converted Digital Imaging and Communications in Medicine data into Neuroimaging Informatics Technology Initiative format with standardized nomenclature for ease of use. In addition, we generated consensus segmentations for experts and non-experts using the Simultaneous Truth and Performance Level Estimation method. These standardized, structured, and easily accessible data are a valuable resource for systematically studying variability in segmentation applications.

E pluribus unum: prospective acceptability benchmarking from the Contouring Collaborative for Consensus in Radiation Oncology crowdsourced initiative for multiobserver segmentation.

Purpose: Contouring Collaborative for Consensus in Radiation Oncology (C3RO) is a crowdsourced challenge engaging radiation oncologists across various expertise levels in segmentation. An obstacle to artificial intelligence (AI) development is the paucity of multiexpert datasets; consequently, we sought to characterize whether aggregate segmentations generated from multiple nonexperts could meet or exceed recognized expert agreement. Approach: Participants who contoured ≥ 1 region of interest (ROI) for the breast, sarcoma, head and neck (H&N), gynecologic (GYN), or gastrointestinal (GI) cases were identified as a nonexpert or recognized expert. Cohort-specific ROIs were combined into single simultaneous truth and performance level estimation (STAPLE) consensus segmentations. STAPLE nonexpert ROIs were evaluated against STAPLE expert contours using Dice similarity coefficient (DSC). The expert interobserver DSC ( IODSC expert ) was calculated as an acceptability threshold between STAPLE nonexpert and STAPLE expert . To determine the number of nonexperts required to match the IODSC expert for each ROI, a single consensus contour was generated using variable numbers of nonexperts and then compared to the IODSC expert . Results: For all cases, the DSC values for STAPLE nonexpert versus STAPLE expert were higher than comparator expert IODSC expert for most ROIs. The minimum number of nonexpert segmentations needed for a consensus ROI to achieve IODSC expert acceptability criteria ranged between 2 and 4 for breast, 3 and 5 for sarcoma, 3 and 5 for H&N, 3 and 5 for GYN, and 3 for GI. Conclusions: Multiple nonexpert-generated consensus ROIs met or exceeded expert-derived acceptability thresholds. Five nonexperts could potentially generate consensus segmentations for most ROIs with performance approximating experts, suggesting nonexpert segmentations as feasible cost-effective AI inputs.

Usability: An introduction to and literature review of usability testing for educational resources in radiation oncology.

Usability, or the ease with which something can be used, is a key aspect in ensuring end-users can achieve the best possible outcomes from a given educational resource. Ideally usability testing should take place iteratively throughout the design of the resource, and there are several approaches for undertaking usability testing described in the wider literature. Within radiation oncology education, the extent to which usability testing occurs remains unclear. This literature review aimed to assess current practice and provide a practical introduction to usability testing for educational resource design within radiation oncology. Two web databases were searched for articles describing planned or completed usability testing during the design of a radiation oncology educational resource. Fifteen studies were identified. Data was gathered describing the type of usability testing performed, the number of cycles of testing and the number of test subjects. Articles described design of educational resources for both patients and trainees, with the number of test subjects ranging from 8 to 18. Various testing methods were used, including questionnaires, think aloud studies and heuristic evaluation. Usability testing comprised a range of single cycle through to several rounds of testing. Through illustrative examples identified in the literature review, we demonstrate that usability testing is feasible and beneficial for educational resources varying in size and context. In doing so we hope to encourage radiation oncologists to incorporate usability testing into future educational resource design.

Metrics to evaluate the performance of auto-segmentation for radiation treatment planning: A critical review.

Advances in artificial intelligence-based methods have led to the development and publication of numerous systems for auto-segmentation in radiotherapy. These systems have the potential to decrease contour variability, which has been associated with poor clinical outcomes and increased efficiency in the treatment planning workflow. However, there are no uniform standards for evaluating auto-segmentation platforms to assess their efficacy at meeting these goals. Here, we review the most frequently used evaluation techniques which include geometric overlap, dosimetric parameters, time spent contouring, and clinical rating scales. These data suggest that many of the most commonly used geometric indices, such as the Dice Similarity Coefficient, are not well correlated with clinically meaningful endpoints. As such, a multi-domain evaluation, including composite geometric and/or dosimetric metrics with physician-reported assessment, is necessary to gauge the clinical readiness of auto-segmentation for radiation treatment planning.

Education and training for image-guided adaptive brachytherapy for cervix cancer-The (GEC)-ESTRO/EMBRACE perspective.

Image-guided adaptive brachytherapy (IGABT) has been shown to improve local/regional control and survival for cervix cancer patients while reducing morbidity. However, the technique is complex involving several conceptual, methodological, and technical innovations compared to conventional brachytherapy. The delivery of high-quality IGABT which will translate into improved outcomes is therefore critically dependent on effective education and training of all health professionals involved in the brachytherapy treatment process. This paper reviews the (GEC)-ESTRO/EMBRACE initiatives for education and training to promote the dissemination and implementation of IGABT for cervix cancer worldwide. The new skills required in different health professionals for successful implementation of IGABT are described. The achievements and challenges of current educational strategies for disseminating IGABT are discussed. Innovations to improve the effectiveness of current and future educational strategies are explored.

Implementing an online radiotherapy quality assurance programme with supporting continuous medical education - report from the EMBRACE-II evaluation of cervix cancer IMRT contouring.

BACKGROUND AND PURPOSE: EMBRACE-II is an international prospective study of IMRT and MRI-guided adaptive brachytherapy (IGABT) in locally advanced cervix cancer. An online radiotherapy quality assurance (RTQA) programme with minimal data transfer and supporting continuing medical education (CME) was implemented for IMRT contouring. MATERIALS AND METHODS: Participant contours for six volumes-of-interest (VOIs) on one benchmark case were scored (2 = excellent, 1 = fair, 0 = revision required) against a consensus reference contour. For contours receiving a 0 or 1 score, additional qualitative comments were provided. The Jaccard conformity index (JCI) was retrospectively calculated. User interaction with CME content (pre-accreditation questionnaire, contouring atlas, practice cases, quizzes, internal target volume (ITV-T) guide) was analysed. RESULTS: 78 clinicians submitted contours for evaluation. 41% passed at the first attempt, 44% after one revision and 6% after two or more revisions. 9% did not re-submit after failing. The lowest mean scores were for the elective nodal CTV (CTV-E) (1.01/2) and ITV-T (1.06/2). 60 different errors across the six VOIs were identified; five potentially had high impact on loco-regional control. A JCI cut-off of 0.7 would have identified 87% contours that failed expert assessment, but also excluded 54% of passing contours. 39 clinicians responded to the pre-accreditation questionnaire - 36% anticipated difficulties with the ITV-T and 13% with the CTV-E. 35% clinicians contoured on the practice cases, 17% answered a quiz, 96% used the atlas and 38% the ITV-T guide. CONCLUSION: Expert evaluation with qualitative feedback improved contouring compliance. The JCI is not a reliable alternative to expert assessment. Moderate uptake of optional CME content limited evaluation.

Challenges for Quality Assurance of Target Volume Delineation in Clinical Trials.

In recent years, new radiotherapy techniques have emerged that aim to improve treatment outcome and reduce toxicity. The standard method of evaluating such techniques is to conduct large scale multicenter clinical trials, often across continents. A major challenge for such trials is quality assurance to ensure consistency of treatment across all participating centers. Analyses from previous studies have shown that poor compliance and protocol violation have a significant adverse effect on treatment outcomes. The results of the clinical trials may, therefore, be confounded by poor quality radiotherapy. Target volume delineation (TVD) is one of the most critical steps in the radiotherapy process. Many studies have shown large inter-observer variations in contouring, both within and outside of clinical trials. High precision techniques, such as intensity-modulated radiotherapy, image-guided brachytherapy, and stereotactic radiotherapy have steep dose gradients, and errors in contouring may lead to inadequate dose to the tumor and consequently, reduce the chance of cure. Similarly, variation in organ at risk delineation will make it difficult to evaluate dose response for toxicity. This article reviews the literature on TVD variability and its impact on dosimetry and clinical outcomes. The implications for quality assurance in clinical trials are discussed.

The Cambridge post-mastectomy radiotherapy (C-PMRT) index: a practical tool for patient selection.

BACKGROUND AND PURPOSE: Post mastectomy radiotherapy (PMRT) reduces loco-regional recurrence (LRR) and has been associated with survival benefit. It is recommended for patients with T3/T4 tumours and/or ⩾ 4 positive lymph nodes (LN). The role of PMRT in 1-3 positive LN and LN negative patients is contentious. The C-PMRT index has been designed for selecting PMRT patients, using independent prognostic factors for LRR. This study reports a 10 year experience using this index. MATERIALS AND METHODS: The C-PMRT index was constructed using the following prognostic factors (a) number of positive LN/lymphovascular invasion, (b) tumour size (c) margin status and (d) tumour grade. Patients were categorised as high (H) risk, intermediate (I) risk and low (L) risk. PMRT was recommended for H and I risk patients. The LRR, distant metastasis and overall survival (OS) rates were measured from the day of mastectomy. RESULTS: From 1999 to 2009, 898 invasive breast cancers in 883 patients were treated by mastectomy (H: 323, I: 231 and L: 344). At a median follow up of 5.2 years, 4.7% (42/898) developed LRR. The 5-year actuarial LRR rates were 6%, 2% and 2% for the H, I and L risk groups, respectively. 1.6% (14/898) developed isolated LRR (H risk n = 4, I risk group n = 0 and L risk n = 10). The 5-year actuarial overall survival rates were 67%, 77% and 90% for H, I and L risk groups, respectively. CONCLUSION: Based on published literature, one would have expected a higher LRR rate in the I risk group without adjuvant RT. We hypothesise that the I risk group LRR rates have been reduced to that of the L risk group by the addition of RT. Apart from LN status and tumour size, other prognostic factors should also be considered in selecting patients for PMRT. This pragmatic tool requires further validation.

Anal cancer: are we making progress?

Anal cancer is an uncommon malignancy. There have been some intriguing developments in the past 3 years, in terms of our understanding of the molecular biology and processes that lead to anal cancer. There have also been some notable successes in prevention, imaging and treatment. Nonsurgical treatment is highly effective. The primary aim of such treatment is to achieve loco-regional control with chemoradiation (CRT), and preserve anal function without a colostomy. Randomised phase III trials presented or published over the past 3 years have explored novel strategies of neoadjuvant chemotherapy, maintenance chemotherapy, radiotherapy dose escalation and replacement of mitomycln C (MMC) with cisplatin in CRT. All have failed to improve on the current standard of care; i.e. MMC/ 5 fluorouracil (5FU) chemoradiation. However, more conformal strategies such as intensity modulated radiotherapy (IMRT) appear feasible to deliver with reduced toxicity, and may offer an opportunity to dose-escalate both to gross tumour and areas of potential nodal spread. Preliminary outcome data suggest no loss of efficacy. We evaluate the relevant recent literature published over the past 2 years, and summarize interesting and important new findings, with the aim of bringing the reader up-to-date on anal cancer.

Finding needles in a haystack: annual low-dose computed tomography screening reduces lung cancer mortality in a high-risk group.

Lung cancer is a global health issue. Compared with other common malignancies, the prognosis is poor as many patients present with advanced disease. The National Lung Screening Trial (NLST) aimed to identify and treat early lung cancers using annual low-dose computed tomography (CT) screening in a high-risk group. When compared with chest x-ray screening, low-dose CT screening reduced lung cancer mortality by 20%; the NLST is the first lung cancer screening trial to demonstrate such a mortality benefit. However, we must wait for cost-effectiveness data from the NLST, as well as the results of ongoing European studies comparing low-dose CT with observation alone, before firm conclusions can be drawn regarding the overall benefits of introducing a CT screening program to clinical practice.