Knowledge-based quality assurance of a comprehensive set of organ at risk contours for head and neck radiotherapy.

Introduction: Manual review of organ at risk (OAR) contours is crucial for creating safe radiotherapy plans but can be time-consuming and error prone. Statistical and deep learning models show the potential to automatically detect improper contours by identifying outliers using large sets of acceptable data (knowledge-based outlier detection) and may be able to assist human reviewers during review of OAR contours. Methods: This study developed an automated knowledge-based outlier detection method and assessed its ability to detect erroneous contours for all common head and neck (HN) OAR types used clinically at our institution. We utilized 490 accurate CT-based HN structure sets from unique patients, each with forty-two HN OAR contours when anatomically present. The structure sets were distributed as 80% for training, 10% for validation, and 10% for testing. In addition, 190 and 37 simulated contours containing errors were added to the validation and test sets, respectively. Single-contour features, including location, shape, orientation, volume, and CT number, were used to train three single-contour feature models (z-score, Mahalanobis distance [MD], and autoencoder [AE]). Additionally, a novel contour-to-contour relationship (CCR) model was trained using the minimum distance and volumetric overlap between pairs of OAR contours to quantify overlap and separation. Inferences from single-contour feature models were combined with the CCR model inferences and inferences evaluating the number of disconnected parts in a single contour and then compared. Results: In the test dataset, before combination with the CCR model, the area under the curve values were 0.922/0.939/0.939 for the z-score, MD, and AE models respectively for all contours. After combination with CCR model inferences, the z-score, MD, and AE had sensitivities of 0.838/0.892/0.865, specificities of 0.922/0.907/0.887, and balanced accuracies (BA) of 0.880/0.900/0.876 respectively. In the validation dataset, with similar overall performance and no signs of overfitting, model performance for individual OAR types was assessed. The combined AE model demonstrated minimum, median, and maximum BAs of 0.729, 0.908, and 0.980 across OAR types. Discussion: Our novel knowledge-based method combines models utilizing single-contour and CCR features to effectively detect erroneous OAR contours across a comprehensive set of 42 clinically used OAR types for HN radiotherapy.

Automated testing platform for radiotherapy treatment planning scripts.

Realizing the potential of user-developed automation software interacting with a treatment planning system (TPS) requires rigorous testing to ensure patient safety and data integrity. We developed an automated test platform to allow comparison of the treatment planning database before and after the execution of a write-enabled script interacting with a commercial TPS (Eclipse, Varian Medical Systems, Palo Alto, CA) using the vendor-provided Eclipse Scripting Application Programming Interface (ESAPI). The C#-application known as Write-Enable Script Testing Engine (WESTE) serializes the treatment planning objects (Patient, Structure Set, PlanSetup) accessible through ESAPI, and then compares the serialization acquired before and after the execution of the script being tested, documenting identified differences to highlight the changes made to the treatment planning data. The first two uses of WESTE demonstrated that the testing platform could acquire and analyze the data quickly (<4 s per test case) and facilitate the clinical implementation of write-enabled scripts.