Development of a national deep learning-based auto-segmentation model for the heart on clinical delineations from the DBCG RT nation cohort.

BACKGROUND: This study aimed at investigating the feasibility of developing a deep learning-based auto-segmentation model for the heart trained on clinical delineations. MATERIAL AND METHODS: This study included two different datasets. The first dataset contained clinical heart delineations from the DBCG RT Nation study (1,561 patients). The second dataset was smaller (114 patients), but with corrected heart delineations. Before training the model on the clinical delineations an outlier-detection was performed, to remove cases with gross deviations from the delineation guideline. No outlier detection was performed for the dataset with corrected heart delineations. Both models were trained with a 3D full resolution nnUNet. The models were evaluated with the dice similarity coefficient (DSC), 95% Hausdorff distance (HD95) and Mean Surface Distance (MSD). The difference between the models were tested with the Mann-Whitney U-test. The balance of dataset quantity versus quality was investigated, by stepwise reducing the cohort size for the model trained on clinical delineations. RESULTS: During the outlier-detection 137 patients were excluded from the clinical cohort due to non-compliance with delineation guidelines. The model trained on the curated clinical cohort performed with a median DSC of 0.96 (IQR 0.94-0.96), median HD95 of 4.00 mm (IQR 3.00 mm-6.00 mm) and a median MSD of 1.49 mm (IQR 1.12 mm-2.02 mm). The model trained on the dedicated and corrected cohort performed with a median DSC of 0.95 (IQR 0.93-0.96), median HD95 of 5.65 mm (IQR 3.37 mm-8.62 mm) and median MSD of 1.63 mm (IQR 1.35 mm-2.11 mm). The difference between the two models were found non-significant for all metrics (p > 0.05). Reduction of cohort size showed no significant difference for all metrics (p > 0.05). However, with the smallest cohort size, a few outlier structures were found. CONCLUSIONS: This study demonstrated a deep learning-based auto-segmentation model trained on curated clinical delineations which performs on par with a model trained on dedicated delineations, making it easier to develop multi-institutional auto-segmentation models.

Target coverage and local recurrences after radiotherapy for sinonasal cancer in Denmark 2008-2015. A DAHANCA study.

PURPOSE: The study aimed to investigate the pattern of failure and describe compromises in the definition and coverage of the target for patients treated with curatively intended radiotherapy (RT) for sinonasal cancer (SNC). METHODS AND MATERIAL: Patients treated with curatively intended RT in 2008-2015 in Denmark for SNC were eligible for the retrospective cohort study. Information regarding diagnosis and treatment was retrieved from the national database of the Danish Head and Neck Cancer Group (DAHANCA). Imaging from the diagnosis of recurrences was collected, and the point of origin (PO) of the recurrent tumour was estimated. All treatment plans were collected and reviewed with the focus on target coverage, manual modifications of target volumes, and dose to organs at risk (OARs) above defined constraints. RESULTS: A total of 184 patients were included in the analysis, and 76 (41%) relapsed. The majority of recurrences involved T-site (76%). Recurrence imaging of 39 patients was evaluated, and PO was established. Twenty-nine POs (74%) were located within the CTV, and the minimum dose to the PO was median 64.1 Gy (3.1-70.7). The criteria for target coverage (V95%) was not met in 89/184 (48%) of the CTV and 131/184 (71%) of the PTV. A total of 24% of CTVs had been manually modified to spare OARs of high-dose irradiation. No difference in target volume modifications was observed between patients who suffered recurrence and patients with lasting remission. CONCLUSION: The majority of relapses after radical treatment of SNC were located in the T-site (the primary tumour site). Multiple compromises with regards to target coverage and tolerance levels for OARs in the sinonasal region, as defined from RT guidelines, were taken. No common practice in this respect could be derived from the study.

Comparing different CT, PET and MRI multi-modality image combinations for deep learning-based head and neck tumor segmentation.

BACKGROUND: Manual delineation of gross tumor volume (GTV) is essential for radiotherapy treatment planning, but it is time-consuming and suffers inter-observer variability (IOV). In clinics, CT, PET, and MRI are used to inform delineation accuracy due to their different complementary characteristics. This study aimed to investigate deep learning to assist GTV delineation in head and neck squamous cell carcinoma (HNSCC) by comparing various modality combinations. MATERIALS AND METHODS: This retrospective study had 153 patients with multiple sites of HNSCC including their planning CT, PET, and MRI (T1-weighted and T2-weighted). Clinical delineations of gross tumor volume (GTV-T) and involved lymph nodes (GTV-N) were collected as the ground truth. The dataset was randomly divided into 92 patients for training, 31 for validation, and 30 for testing. We applied a residual 3 D UNet as the deep learning architecture. We independently trained the UNet with four different modality combinations (CT-PET-MRI, CT-MRI, CT-PET, and PET-MRI). Additionally, analogical to post-processing, an average fusion of three bi-modality combinations (CT-PET, CT-MRI, and PET-MRI) was produced as an ensemble. Segmentation accuracy was evaluated on the test set, using Dice similarity coefficient (Dice), Hausdorff Distance 95 percentile (HD95), and Mean Surface Distance (MSD). RESULTS: All imaging combinations including PET provided similar average scores in range of Dice: 0.72-0.74, HD95: 8.8-9.5 mm, MSD: 2.6-2.8 mm. Only CT-MRI had a lower score with Dice: 0.58, HD95: 12.9 mm, MSD: 3.7 mm. The average of three bi-modality combinations reached Dice: 0.74, HD95: 7.9 mm, MSD: 2.4 mm. CONCLUSION: Multimodal deep learning-based auto segmentation of HNSCC GTV was demonstrated and inclusion of the PET image was shown to be crucial. Training on combined MRI, PET, and CT data provided limited improvements over CT-PET and PET-MRI. However, when combining three bimodal trained networks into an ensemble, promising improvements were shown.

RadDeploy: A framework for integrating in-house developed software and artificial intelligence models seamlessly into radiotherapy workflows.

The use of and research in automation and artificial intelligence (AI) in radiotherapy is moving with incredible pace. Many innovations do, however, not make it into the clinic. One technical reason for this may be the lack of a platform to deploy such software into clinical practice. We suggest RadDeploy as a framework for integrating containerized software in clinical workflows outside of treatment planning systems. RadDeploy supports multiple DICOM as input for model containers and can run model containers asynchronously across GPUs and computers. This technical note summarizes the inner workings of RadDeploy and demonstrates three use-cases with varying complexity.

Enhancing the reliability of deep learning-based head and neck tumour segmentation using uncertainty estimation with multi-modal images.

Objective.Deep learning shows promise in autosegmentation of head and neck cancer (HNC) primary tumours (GTV-T) and nodal metastases (GTV-N). However, errors such as including non-tumour regions or missing nodal metastases still occur. Conventional methods often make overconfident predictions, compromising reliability. Incorporating uncertainty estimation, which provides calibrated confidence intervals can address this issue. Our aim was to investigate the efficacy of various uncertainty estimation methods in improving segmentation reliability. We evaluated their confidence levels in voxel predictions and ability to reveal potential segmentation errors.Approach.We retrospectively collected data from 567 HNC patients with diverse cancer sites and multi-modality images (CT, PET, T1-, and T2-weighted MRI) along with their clinical GTV-T/N delineations. Using the nnUNet 3D segmentation pipeline, we compared seven uncertainty estimation methods, evaluating them based on segmentation accuracy (Dice similarity coefficient, DSC), confidence calibration (Expected Calibration Error, ECE), and their ability to reveal segmentation errors (Uncertainty-Error overlap using DSC, UE-DSC).Main results.Evaluated on the hold-out test dataset (n= 97), the median DSC scores for GTV-T and GTV-N segmentation across all uncertainty estimation methods had a narrow range, from 0.73 to 0.76 and 0.78 to 0.80, respectively. In contrast, the median ECE exhibited a wider range, from 0.30 to 0.12 for GTV-T and 0.25 to 0.09 for GTV-N. Similarly, the median UE-DSC also ranged broadly, from 0.21 to 0.38 for GTV-T and 0.22 to 0.36 for GTV-N. A probabilistic network-PhiSeg method consistently demonstrated the best performance in terms of ECE and UE-DSC.Significance.Our study highlights the importance of uncertainty estimation in enhancing the reliability of deep learning for autosegmentation of HNC GTV. The results show that while segmentation accuracy can be similar across methods, their reliability, measured by calibration error and uncertainty-error overlap, varies significantly. Used with visualisation maps, these methods may effectively pinpoint uncertainties and potential errors at the voxel level.

The effect of editing clinical contours on deep-learning segmentation accuracy of the gross tumor volume in glioblastoma.

Background and purpose: Deep-learning (DL) models for segmentation of the gross tumor volume (GTV) in radiotherapy are generally based on clinical delineations which suffer from inter-observer variability. The aim of this study was to compare performance of a DL-model based on clinical glioblastoma GTVs to a model based on a single-observer edited version of the same GTVs. Materials and methods: The dataset included imaging data (Computed Tomography (CT), T1, contrast-T1 (T1C), and fluid-attenuated-inversion-recovery (FLAIR)) of 259 glioblastoma patients treated with post-operative radiotherapy between 2012 and 2019 at a single institute. The clinical GTVs were edited using all imaging data. The dataset was split into 207 cases for training/validation and 52 for testing.GTV segmentation models (nnUNet) were trained on clinical and edited GTVs separately and compared using Surface Dice with 1 mm tolerance (sDSC1mm). We also evaluated model performance with respect to extent of resection (EOR), and different imaging combinations (T1C/T1/FLAIR/CT, T1C/FLAIR/CT, T1C/FLAIR, T1C/CT, T1C/T1, T1C). A Wilcoxon test was used for significance testing. Results: The median (range) sDSC1mm of the clinical-GTV-model and edited-GTV-model both evaluated with the edited contours, was 0.76 (0.43-0.94) vs. 0.92 (0.60-0.98) respectively (p < 0.001). sDSC1mm was not significantly different between patients with a biopsy, partial, and complete resection. T1C as single input performed as good as use of imaging combinations. Conclusions: High segmentation accuracy was obtained by the DL-models. Editing of the clinical GTVs significantly increased DL performance with a relevant effect size. DL performance was robust for EOR and highly accurate using only T1C.

Artificial Intelligence Uncertainty Quantification in Radiotherapy Applications - A Scoping Review.

Background/purpose: The use of artificial intelligence (AI) in radiotherapy (RT) is expanding rapidly. However, there exists a notable lack of clinician trust in AI models, underscoring the need for effective uncertainty quantification (UQ) methods. The purpose of this study was to scope existing literature related to UQ in RT, identify areas of improvement, and determine future directions. Methods: We followed the PRISMA-ScR scoping review reporting guidelines. We utilized the population (human cancer patients), concept (utilization of AI UQ), context (radiotherapy applications) framework to structure our search and screening process. We conducted a systematic search spanning seven databases, supplemented by manual curation, up to January 2024. Our search yielded a total of 8980 articles for initial review. Manuscript screening and data extraction was performed in Covidence. Data extraction categories included general study characteristics, RT characteristics, AI characteristics, and UQ characteristics. Results: We identified 56 articles published from 2015-2024. 10 domains of RT applications were represented; most studies evaluated auto-contouring (50%), followed by image-synthesis (13%), and multiple applications simultaneously (11%). 12 disease sites were represented, with head and neck cancer being the most common disease site independent of application space (32%). Imaging data was used in 91% of studies, while only 13% incorporated RT dose information. Most studies focused on failure detection as the main application of UQ (60%), with Monte Carlo dropout being the most commonly implemented UQ method (32%) followed by ensembling (16%). 55% of studies did not share code or datasets. Conclusion: Our review revealed a lack of diversity in UQ for RT applications beyond auto-contouring. Moreover, there was a clear need to study additional UQ methods, such as conformal prediction. Our results may incentivize the development of guidelines for reporting and implementation of UQ in RT.

Is full-automation in radiotherapy treatment planning ready for take off?

Radiotherapy treatment planning is undergoing a transformation with the increasing integration of automation. This transition draws parallels with the aviation industry, which has a long-standing history of addressing challenges and opportunities introduced by automated systems. Both fields witness a shift from manual operations to systems capable of operating independently, raising questions about the risks and evolving role of humans within automated workflows. In response to this shift, a working group assembled during the ESTRO Physics Workshop 2023, reflected on parallels to draw lessons for radiotherapy. A taxonomy is proposed, leveraging insights from aviation, that outlines the observed levels of automation within the context of radiotherapy and their corresponding implications for human involvement. Among the common identified risks associated with automation integration are complacency, overreliance, attention tunneling, data overload, a lack of transparency and training. These risks require mitigation strategies. Such strategies include ensuring role complementarity, introducing checklists and safety requirements for human-automation interaction and using automation for cognitive unload and workflow management. Focusing on already automated processes, such as dose calculation and auto-contouring as examples, we have translated lessons learned from aviation. It remains crucial to strike a balance between automation and human involvement. While automation offers the potential for increased efficiency and accuracy, it must be complemented by human oversight, expertise, and critical decision-making. The irreplaceable value of human judgment remains, particularly in complex clinical situations. Learning from aviation, we identify a need for human factors engineering research in radiation oncology and a continued requirement for proactive incident learning.

Artificial intelligence uncertainty quantification in radiotherapy applications - A scoping review.

BACKGROUND/PURPOSE: The use of artificial intelligence (AI) in radiotherapy (RT) is expanding rapidly. However, there exists a notable lack of clinician trust in AI models, underscoring the need for effective uncertainty quantification (UQ) methods. The purpose of this study was to scope existing literature related to UQ in RT, identify areas of improvement, and determine future directions. METHODS: We followed the PRISMA-ScR scoping review reporting guidelines. We utilized the population (human cancer patients), concept (utilization of AI UQ), context (radiotherapy applications) framework to structure our search and screening process. We conducted a systematic search spanning seven databases, supplemented by manual curation, up to January 2024. Our search yielded a total of 8980 articles for initial review. Manuscript screening and data extraction was performed in Covidence. Data extraction categories included general study characteristics, RT characteristics, AI characteristics, and UQ characteristics. RESULTS: We identified 56 articles published from 2015 to 2024. 10 domains of RT applications were represented; most studies evaluated auto-contouring (50 %), followed by image-synthesis (13 %), and multiple applications simultaneously (11 %). 12 disease sites were represented, with head and neck cancer being the most common disease site independent of application space (32 %). Imaging data was used in 91 % of studies, while only 13 % incorporated RT dose information. Most studies focused on failure detection as the main application of UQ (60 %), with Monte Carlo dropout being the most commonly implemented UQ method (32 %) followed by ensembling (16 %). 55 % of studies did not share code or datasets. CONCLUSION: Our review revealed a lack of diversity in UQ for RT applications beyond auto-contouring. Moreover, we identified a clear need to study additional UQ methods, such as conformal prediction. Our results may incentivize the development of guidelines for reporting and implementation of UQ in RT.

Fully automated radiotherapy treatment planning: A scan to plan challenge.

BACKGROUND AND PURPOSE: Over the past decade, tools for automation of various sub-tasks in radiotherapy planning have been introduced, such as auto-contouring and auto-planning. The purpose of this study was to benchmark what degree of automation is possible. MATERIALS AND METHODS: A challenge to perform automated treatment planning for prostate and prostate bed radiotherapy was set up. Participants were provided with simulation CTs and a treatment prescription and were asked to use automated tools to produce a deliverable radiotherapy treatment plan with as little human intervention as possible. Plans were scored for their adherence to the protocol when assessed using consensus expert contours. RESULTS: Thirteen entries were received. The top submission adhered to 81.8% of the minimum objectives across all cases using the consensus contour, meeting all objectives in one of the ten cases. The same system met 89.5% of objectives when assessed with their own auto-contours, meeting all objectives in four of the ten cases. The majority of systems used in the challenge had regulatory clearance (Auto-contouring: 82.5%, Auto-planning: 77%). Despite the 'hard' rule that participants should not check or edit contours or plans, 69% reported looking at their results before submission. CONCLUSIONS: Automation of the full planning workflow from simulation CT to deliverable treatment plan is possible for prostate and prostate bed radiotherapy. While many generated plans were found to require none or minor adjustment to be regarded as clinically acceptable, the result indicated there is still a lack of trust in such systems preventing full automation.

A joint ESTRO and AAPM guideline for development, clinical validation and reporting of artificial intelligence models in radiation therapy.

BACKGROUND AND PURPOSE: Artificial Intelligence (AI) models in radiation therapy are being developed with increasing pace. Despite this, the radiation therapy community has not widely adopted these models in clinical practice. A cohesive guideline on how to develop, report and clinically validate AI algorithms might help bridge this gap. METHODS AND MATERIALS: A Delphi process with all co-authors was followed to determine which topics should be addressed in this comprehensive guideline. Separate sections of the guideline, including Statements, were written by subgroups of the authors and discussed with the whole group at several meetings. Statements were formulated and scored as highly recommended or recommended. RESULTS: The following topics were found most relevant: Decision making, image analysis, volume segmentation, treatment planning, patient specific quality assurance of treatment delivery, adaptive treatment, outcome prediction, training, validation and testing of AI model parameters, model availability for others to verify, model quality assurance/updates and upgrades, ethics. Key references were given together with an outlook on current hurdles and possibilities to overcome these. 19 Statements were formulated. CONCLUSION: A cohesive guideline has been written which addresses main topics regarding AI in radiation therapy. It will help to guide development, as well as transparent and consistent reporting and validation of new AI tools and facilitate adoption.

Evaluating Danish Breast Cancer Group locoregional radiotherapy guideline adherence in clinical treatment data 2008-2016: The DBCG RT Nation study.

BACKGROUND AND PURPOSE: Guideline adherence in radiotherapy is crucial for maintaining treatment quality and consistency, particularly in non-trial patient settings where most treatments occur. The study aimed to assess the impact of guideline changes on treatment planning practices and compare manual registry data accuracy with treatment planning data. MATERIALS AND METHODS: This study utilised the DBCG RT Nation cohort, a collection of breast cancer radiotherapy data in Denmark, to evaluate adherence to guidelines from 2008 to 2016. The cohort included 7448 high-risk breast cancer patients. National guideline changes included, fractionation, introduction of respiratory gating, irradiation of the internal mammary lymph nodes, use of the simultaneous integrated boost technique and inclusion of the Left Anterior Descending coronary artery in delineation practice. Methods for structure name mapping, laterality detection, detection of temporal changes in population mean lung volume, and dose evaluation were presented and applied. Manually registered treatment characteristic data was obtained from the Danish Breast Cancer Database for comparison. RESULTS: The study found immediate and consistent adherence to guideline changes across Danish radiotherapy centres. Treatment practices before guideline implementation were documented and showed a variation among centres. Discrepancies between manual registry data and actual treatment planning data were as high as 10% for some measures. CONCLUSION: National guideline changes could be detected in the routine treatment data, with a high degree of compliance and short implementation time. Data extracted from treatment planning data files provides a more accurate and detailed characterisation of treatments and guideline adherence than medical register data.

Associations Between Radiation Oncologist Demographic Factors and Segmentation Similarity Benchmarks: Insights From a Crowd-Sourced Challenge Using Bayesian Estimation.

PURPOSE: The quality of radiotherapy auto-segmentation training data, primarily derived from clinician observers, is of utmost importance. However, the factors influencing the quality of clinician-derived segmentations are poorly understood; our study aims to quantify these factors. METHODS: Organ at risk (OAR) and tumor-related segmentations provided by radiation oncologists from the Contouring Collaborative for Consensus in Radiation Oncology data set were used. Segmentations were derived from five disease sites: breast, sarcoma, head and neck (H&N), gynecologic (GYN), and GI. Segmentation quality was determined on a structure-by-structure basis by comparing the observer segmentations with an expert-derived consensus, which served as a reference standard benchmark. The Dice similarity coefficient (DSC) was primarily used as a metric for the comparisons. DSC was stratified into binary groups on the basis of structure-specific expert-derived interobserver variability (IOV) cutoffs. Generalized linear mixed-effects models using Bayesian estimation were used to investigate the association between demographic variables and the binarized DSC for each disease site. Variables with a highest density interval excluding zero were considered to substantially affect the outcome measure. RESULTS: Five hundred seventy-four, 110, 452, 112, and 48 segmentations were used for the breast, sarcoma, H&N, GYN, and GI cases, respectively. The median percentage of segmentations that crossed the expert DSC IOV cutoff when stratified by structure type was 55% and 31% for OARs and tumors, respectively. Regression analysis revealed that the structure being tumor-related had a substantial negative impact on binarized DSC for the breast, sarcoma, H&N, and GI cases. There were no recurring relationships between segmentation quality and demographic variables across the cases, with most variables demonstrating large standard deviations. CONCLUSION: Our study highlights substantial uncertainty surrounding conventionally presumed factors influencing segmentation quality relative to benchmarks.

Development and comprehensive evaluation of a national DBCG consensus-based auto-segmentation model for lymph node levels in breast cancer radiotherapy.

BACKGROUND AND PURPOSE: This study aimed at training and validating a multi-institutional deep learning (DL) auto segmentation model for nodal clinical target volume (CTVn) in high-risk breast cancer (BC) patients with both training and validation dataset created with multi-institutional participation, with the overall aim of national clinical implementation in Denmark. MATERIALS AND METHODS: A gold standard (GS) dataset and a high-quality training dataset were created by 21 BC delineation experts from all radiotherapy centres in Denmark. The delineations were created according to ESTRO consensus delineation guidelines. Four models were trained: One per laterality and extension of CTVn internal mammary nodes. The DL models were tested quantitatively in their own test-set and in relation to interobserver variation (IOV) in the GS dataset with geometrical metrics, such as the Dice Similarity Coefficient (DSC). A blinded qualitative evaluation was conducted with a national board, presented to both DL and manual delineations. RESULTS: A median DSC > 0.7 was found for all, except the CTVn interpectoral node in one of the models. In the qualitative evaluation 'no corrections needed' were acquired for 297 (36 %) in the DL structures and 286 (34 %) for manual delineations. A higher rate of 'major corrections' and 'easier to start from scratch' was found in the manual delineations. The models performed within the IOV of an expert group, with two exceptions. CONCLUSION: DL models were developed on a national consensus cohort and performed on par with the IOV between BC experts and had a comparable or higher clinical acceptance than expert manual delineations.

Potential of E-Learning Interventions and Artificial Intelligence-Assisted Contouring Skills in Radiotherapy: The ELAISA Study.

PURPOSE: Most research on artificial intelligence-based auto-contouring as template (AI-assisted contouring) for organs-at-risk (OARs) stem from high-income countries. The effect and safety are, however, likely to depend on local factors. This study aimed to investigate the effects of AI-assisted contouring and teaching on contouring time and contour quality among radiation oncologists (ROs) working in low- and middle-income countries (LMICs). MATERIALS AND METHODS: Ninety-seven ROs were randomly assigned to either manual or AI-assisted contouring of eight OARs for two head-and-neck cancer cases with an in-between teaching session on contouring guidelines. Thereby, the effect of teaching (yes/no) and AI-assisted contouring (yes/no) was quantified. Second, ROs completed short-term and long-term follow-up cases all using AI assistance. Contour quality was quantified with Dice Similarity Coefficient (DSC) between ROs' contours and expert consensus contours. Groups were compared using absolute differences in medians with 95% CIs. RESULTS: AI-assisted contouring without previous teaching increased absolute DSC for optic nerve (by 0.05 [0.01; 0.10]), oral cavity (0.10 [0.06; 0.13]), parotid (0.07 [0.05; 0.12]), spinal cord (0.04 [0.01; 0.06]), and mandible (0.02 [0.01; 0.03]). Contouring time decreased for brain stem (-1.41 [-2.44; -0.25]), mandible (-6.60 [-8.09; -3.35]), optic nerve (-0.19 [-0.47; -0.02]), parotid (-1.80 [-2.66; -0.32]), and thyroid (-1.03 [-2.18; -0.05]). Without AI-assisted contouring, teaching increased DSC for oral cavity (0.05 [0.01; 0.09]) and thyroid (0.04 [0.02; 0.07]), and contouring time increased for mandible (2.36 [-0.51; 5.14]), oral cavity (1.42 [-0.08; 4.14]), and thyroid (1.60 [-0.04; 2.22]). CONCLUSION: The study suggested that AI-assisted contouring is safe and beneficial to ROs working in LMICs. Prospective clinical trials on AI-assisted contouring should, however, be conducted upon clinical implementation to confirm the effects.

Evaluation of methods for selecting the midventilation bin in 4DCT scans of lung cancer patients.

BACKGROUND: In lung cancer radiotherapy, planning on the midventilation (MidV) bin of a four-dimensional (4D) CT scan can reduce the systematic errors introduced by respiratory tumour motion compared to conventional CT. In this study four different methods for MidV bin selection are evaluated. MATERIAL AND METHODS: The study is based on 4DCT scans of 19 patients with a total of 23 peripheral lung tumours having peak-to-peak displacement ≥ 5 mm in at least one of the left-right (LR), anterior-posterior (AP) or cranio-caudal (CC) directions. For each tumour, the MidV bin was selected based on: 1) visual evaluation of tumour displacement; 2) rigid registration of tumour position; 3) diaphragm displacement in the CC direction; and 4) carina displacement in the CC direction. Determination of the MidV bin based on the displacement of the manually delineated gross tumour volume (GTV) was used as a reference method. The accuracy of each method was evaluated by the distance between GTV position in the selected MidV bin and the time-weighted mean position of GTV throughout the bins (i.e. the geometric MidV error). RESULTS: Median (range) geometric MidV error was 1.4 (0.4-5.4) mm, 1.4 (0.4-5.4) mm, 1.9 (0.5-6.9) mm, 2.0 (0.5-12.3) mm and 1.1 (0.4-5.4) mm for the visual, rigid registration, diaphragm, carina, and reference method. Median (range) absolute difference between geometric MidV error for the evaluated methods and the reference method was 0.0 (0.0-1.2) mm, 0.0 (0.0-1.7) mm, 0.7 (0.0-3.9) mm and 1.0 (0.0-6.9) mm for the visual, rigid registration, diaphragm and carina method. CONCLUSION: The visual and semi-automatic rigid registration methods were equivalent in accuracy for selecting the MidV bin of a 4DCT scan. The methods based on diaphragm and carina displacement cannot be recommended without modifications.