Development and external validation of a machine learning model for prediction of survival in extremity leiomyosarcoma.

PURPOSE: Machine learning (ML) models have been used to predict cancer survival in several sarcoma subtypes. However, none have investigated extremity leiomyosarcoma (LMS). ML is a powerful tool that has the potential to better prognosticate extremity LMS. METHODS: The Surveillance, Epidemiology, and End Results (SEER) database was queried for cases of histologic extremity LMS (n = 634). Patient, tumor, and treatment characteristics were recorded, and ML models were developed to predict 1-, 3-, and 5-year survival. The best performing ML model was externally validated using an institutional cohort of extremity LMS patients (n = 46). RESULTS: All ML models performed best at the 1-year time point and worst at the 5-year time point. On internal validation within the SEER cohort, the best models had c-statistics of 0.75-0.76 at the 5-year time point. The Random Forest (RF) model was the best performing model and used for external validation. This model also performed best at 1-year and worst at 5-year on external validation with c-statistics of 0.90 and 0.87, respectively. The RF model was well calibrated on external validation. This model has been made publicly available at https://rachar.shinyapps.io/lms_app/ CONCLUSIONS: ML models had excellent performance for survival prediction of extremity LMS. Future studies incorporating a larger institutional cohort may be needed to further validate the ML model for LMS prognostication.

The GenoVA study: Equitable implementation of a pragmatic randomized trial of polygenic-risk scoring in primary care.

Polygenic risk scores (PRSs) hold promise for disease risk assessment and prevention. The Genomic Medicine at Veterans Affairs (GenoVA) Study is addressing three main challenges to the clinical implementation of PRSs in preventive care: defining and determining their clinical utility, implementing them in time-constrained primary care settings, and countering their potential to exacerbate healthcare disparities. The study processes used to test patients, report their PRS results to them and their primary care providers (PCPs), and promote the use of those results in clinical decision-making are modeled on common practices in primary care. The following diseases were chosen for their prevalence and familiarity to PCPs: coronary artery disease; type 2 diabetes; atrial fibrillation; and breast, colorectal, and prostate cancers. A randomized clinical trial (RCT) design and primary outcome of time-to-new-diagnosis of a target disease bring methodological rigor to the question of the clinical utility of PRS implementation. The study's pragmatic RCT design enhances its relevance to how PRS might reasonably be implemented in primary care. Steps the study has taken to promote health equity include the thoughtful handling of genetic ancestry in PRS construction and reporting and enhanced recruitment strategies to address underrepresentation in research participation. To date, enhanced recruitment efforts have been both necessary and successful: participants of underrepresented race and ethnicity groups have been less likely to enroll in the study than expected but ultimately achieved proportional representation through targeted efforts. The GenoVA Study experience to date offers insights for evaluating the clinical utility of equitable PRS implementation in adult primary care.

Performance of EHR classifiers for patient eligibility in a clinical trial of precision screening.

BACKGROUND: Validated computable eligibility criteria use real-world data and facilitate the conduct of clinical trials. The Genomic Medicine at VA (GenoVA) Study is a pragmatic trial of polygenic risk score testing enrolling patients without known diagnoses of 6 common diseases: atrial fibrillation, coronary artery disease, type 2 diabetes, breast cancer, colorectal cancer, and prostate cancer. We describe the validation of computable disease classifiers as eligibility criteria and their performance in the first 16 months of trial enrollment. METHODS: We identified well-performing published computable classifiers for the 6 target diseases and validated these in the target population using blinded physician review. If needed, classifiers were refined and then underwent a subsequent round of blinded review until true positive and true negative rates ≥80% were achieved. The optimized classifiers were then implemented as pre-screening exclusion criteria; telephone screens enabled an assessment of their real-world negative predictive value (NPV-RW). RESULTS: Published classifiers for type 2 diabetes and breast and prostate cancer achieved desired performance in blinded chart review without modification; the classifier for atrial fibrillation required two rounds of refinement before achieving desired performance. Among the 1077 potential participants screened in the first 16 months of enrollment, NPV-RW of the classifiers ranged from 98.4% for coronary artery disease to 99.9% for colorectal cancer. Performance did not differ by gender or race/ethnicity. CONCLUSIONS: Computable disease classifiers can serve as efficient and accurate pre-screening classifiers for clinical trials, although performance will depend on the trial objectives and diseases under study.

Prognostication of patients with COVID-19 using artificial intelligence based on chest x-rays and clinical data: a retrospective study.

BACKGROUND: Chest x-ray is a relatively accessible, inexpensive, fast imaging modality that might be valuable in the prognostication of patients with COVID-19. We aimed to develop and evaluate an artificial intelligence system using chest x-rays and clinical data to predict disease severity and progression in patients with COVID-19. METHODS: We did a retrospective study in multiple hospitals in the University of Pennsylvania Health System in Philadelphia, PA, USA, and Brown University affiliated hospitals in Providence, RI, USA. Patients who presented to a hospital in the University of Pennsylvania Health System via the emergency department, with a diagnosis of COVID-19 confirmed by RT-PCR and with an available chest x-ray from their initial presentation or admission, were retrospectively identified and randomly divided into training, validation, and test sets (7:1:2). Using the chest x-rays as input to an EfficientNet deep neural network and clinical data, models were trained to predict the binary outcome of disease severity (ie, critical or non-critical). The deep-learning features extracted from the model and clinical data were used to build time-to-event models to predict the risk of disease progression. The models were externally tested on patients who presented to an independent multicentre institution, Brown University affiliated hospitals, and compared with severity scores provided by radiologists. FINDINGS: 1834 patients who presented via the University of Pennsylvania Health System between March 9 and July 20, 2020, were identified and assigned to the model training (n=1285), validation (n=183), or testing (n=366) sets. 475 patients who presented via the Brown University affiliated hospitals between March 1 and July 18, 2020, were identified for external testing of the models. When chest x-rays were added to clinical data for severity prediction, area under the receiver operating characteristic curve (ROC-AUC) increased from 0·821 (95% CI 0·796-0·828) to 0·846 (0·815-0·852; p<0·0001) on internal testing and 0·731 (0·712-0·738) to 0·792 (0·780-0 ·803; p<0·0001) on external testing. When deep-learning features were added to clinical data for progression prediction, the concordance index (C-index) increased from 0·769 (0·755-0·786) to 0·805 (0·800-0·820; p<0·0001) on internal testing and 0·707 (0·695-0·729) to 0·752 (0·739-0·764; p<0·0001) on external testing. The image and clinical data combined model had significantly better prognostic performance than combined severity scores and clinical data on internal testing (C-index 0·805 vs 0·781; p=0·0002) and external testing (C-index 0·752 vs 0·715; p<0·0001). INTERPRETATION: In patients with COVID-19, artificial intelligence based on chest x-rays had better prognostic performance than clinical data or radiologist-derived severity scores. Using artificial intelligence, chest x-rays can augment clinical data in predicting the risk of progression to critical illness in patients with COVID-19. FUNDING: Brown University, Amazon Web Services Diagnostic Development Initiative, Radiological Society of North America, National Cancer Institute and National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health.

Evaluation of a convolutional neural network for ovarian tumor differentiation based on magnetic resonance imaging.

OBJECTIVES: There currently lacks a noninvasive and accurate method to distinguish benign and malignant ovarian lesion prior to treatment. This study developed a deep learning algorithm that distinguishes benign from malignant ovarian lesion by applying a convolutional neural network on routine MR imaging. METHODS: Five hundred forty-five lesions (379 benign and 166 malignant) from 451 patients from a single institution were divided into training, validation, and testing set in a 7:2:1 ratio. Model performance was compared with four junior and three senior radiologists on the test set. RESULTS: Compared with junior radiologists averaged, the final ensemble model combining MR imaging and clinical variables had a higher test accuracy (0.87 vs 0.64, p < 0.001) and specificity (0.92 vs 0.64, p < 0.001) with comparable sensitivity (0.75 vs 0.63, p = 0.407). Against the senior radiologists averaged, the final ensemble model also had a higher test accuracy (0.87 vs 0.74, p = 0.033) and specificity (0.92 vs 0.70, p < 0.001) with comparable sensitivity (0.75 vs 0.83, p = 0.557). Assisted by the model's probabilities, the junior radiologists achieved a higher average test accuracy (0.77 vs 0.64, Δ = 0.13, p < 0.001) and specificity (0.81 vs 0.64, Δ = 0.17, p < 0.001) with unchanged sensitivity (0.69 vs 0.63, Δ = 0.06, p = 0.302). With the AI probabilities, the junior radiologists had higher specificity (0.81 vs 0.70, Δ = 0.11, p = 0.005) but similar accuracy (0.77 vs 0.74, Δ = 0.03, p = 0.409) and sensitivity (0.69 vs 0.83, Δ = -0.146, p = 0.097) when compared with the senior radiologists. CONCLUSIONS: These results demonstrate that artificial intelligence based on deep learning can assist radiologists in assessing the nature of ovarian lesions and improve their performance. KEY POINTS: • Artificial Intelligence based on deep learning can assess the nature of ovarian lesions on routine MRI with higher accuracy and specificity than radiologists. • Assisted by the deep learning model's probabilities, junior radiologists achieved better performance that matched those of senior radiologists.

Deep learning-based classification of primary bone tumors on radiographs: A preliminary study.

BACKGROUND: To develop a deep learning model to classify primary bone tumors from preoperative radiographs and compare performance with radiologists. METHODS: A total of 1356 patients (2899 images) with histologically confirmed primary bone tumors and pre-operative radiographs were identified from five institutions' pathology databases. Manual cropping was performed by radiologists to label the lesions. Binary discriminatory capacity (benign versus not-benign and malignant versus not-malignant) and three-way classification (benign versus intermediate versus malignant) performance of our model were evaluated. The generalizability of our model was investigated on data from external test set. Final model performance was compared with interpretation from five radiologists of varying level of experience using the Permutations tests. FINDINGS: For benign vs. not benign, model achieved area under curve (AUC) of 0•894 and 0•877 on cross-validation and external testing, respectively. For malignant vs. not malignant, model achieved AUC of 0•907 and 0•916 on cross-validation and external testing, respectively. For three-way classification, model achieved 72•1% accuracy vs. 74•6% and 72•1% for the two subspecialists on cross-validation (p = 0•03 and p = 0•52, respectively). On external testing, model achieved 73•4% accuracy vs. 69•3%, 73•4%, 73•1%, 67•9%, and 63•4% for the two subspecialists and three junior radiologists (p = 0•14, p = 0•89, p = 0•93, p = 0•02, p < 0•01 for radiologists 1-5, respectively). INTERPRETATION: Deep learning can classify primary bone tumors using conventional radiographs in a multi-institutional dataset with similar accuracy compared to subspecialists, and better performance than junior radiologists. FUNDING: The project described was supported by RSNA Research & Education Foundation, through grant number RSCH2004 to Harrison X. Bai.

Ablation Therapy for Advanced Stage Non-Small Cell Lung Cancer: A National Cancer Database Study.

PURPOSE: To compare overall survival (OS) of ablation with no treatment for patients with advanced stage non-small cell lung cancer. METHODS: Patients with clinical stage IIIB (T1-4N3M0, T4N2M0) and stage IV (T1-4N0-3M1) non-small cell lung cancer, in accordance with the American Joint Committee on Cancer, 7th edition, who did not receive treatment or who received ablation as their sole primary treatment besides chemotherapy from 2004 to 2014, were identified from the National Cancer Data Base. OS was estimated using the Kaplan-Meier method and evaluated by log-rank test, univariate and multivariate Cox proportional hazard regression, and propensity score-matched analysis. Relative survival analyses comparing age- and sex-matched United States populations were performed. RESULTS: A total of 140,819 patients were included. The 1-, 2-, 3- and 5-year survival rates relative to age- and sex-matched United States population were 28%, 18%, 12%, and 10%, respectively, for ablation (n = 249); and 30%, 15%, 9%, and 5%, respectively for no treatment (n = 140,570). Propensity score matching resulted in 249 patients in the ablation group versus 498 patients in the no-treatment group. After matching, ablation was associated with longer OS than that in the no-treatment group (median, 5.9 vs 4.7 months, respectively; hazard ratio, 0.844; 95% confidence interval, 0.719-0.990; P = .037). These results persisted in patients with an initial tumor size of ≤3 cm. CONCLUSIONS: Preliminary results suggest ablation may be associated with longer OS in patients with late-stage non-small cell lung cancer than survival in those who received no treatment.

Robotic drill guide positioning using known-component 3D-2D image registration.

A method for x-ray image-guided robotic instrument positioning is reported and evaluated in preclinical studies of spinal pedicle screw placement with the aim of improving delivery of transpedicle K-wires and screws. The known-component (KC) registration algorithm was used to register the three-dimensional patient CT and drill guide surface model to intraoperative two-dimensional radiographs. Resulting transformations, combined with offline hand-eye calibration, drive the robotically held drill guide to target trajectories defined in the preoperative CT. The method was assessed in comparison with a more conventional tracker-based approach, and robustness to clinically realistic errors was tested in phantom and cadaver. Deviations from planned trajectories were analyzed in terms of target registration error (TRE) at the tooltip (mm) and approach angle (deg). In phantom studies, the KC approach resulted in [Formula: see text] and [Formula: see text], comparable with accuracy in tracker-based approach. In cadaver studies with realistic anatomical deformation, the KC approach yielded [Formula: see text] and [Formula: see text], with statistically significant improvement versus tracker ([Formula: see text] and [Formula: see text]). Robustness to deformation is attributed to relatively local rigidity of anatomy in radiographic views. X-ray guidance offered accurate robotic positioning and could fit naturally within clinical workflow of fluoroscopically guided procedures.

Development and testing of a database of NIH research funding of AAPM members: A report from the AAPM Working Group for the Development of a Research Database (WGDRD).

PURPOSE: To produce and maintain a database of National Institutes of Health (NIH) funding of the American Association of Physicists in Medicine (AAPM) members, to perform a top-level analysis of these data, and to make these data (hereafter referred to as the AAPM research database) available for the use of the AAPM and its members. METHODS: NIH-funded research dating back to 1985 is available for public download through the NIH exporter website, and AAPM membership information dating back to 2002 was supplied by the AAPM. To link these two sources of data, a data mining algorithm was developed in Matlab. The false-positive rate was manually estimated based on a random sample of 100 records, and the false-negative rate was assessed by comparing against 99 member-supplied PI_ID numbers. The AAPM research database was queried to produce an analysis of trends and demographics in research funding dating from 2002 to 2015. RESULTS: A total of 566 PI_ID numbers were matched to AAPM members. False-positive and -negative rates were respectively 4% (95% CI: 1-10%, N = 100) and 10% (95% CI: 5-18%, N = 99). Based on analysis of the AAPM research database, in 2015 the NIH awarded $USD 110M to members of the AAPM. The four NIH institutes which historically awarded the most funding to AAPM members were the National Cancer Institute, National Institute of Biomedical Imaging and Bioengineering, National Heart Lung and Blood Institute, and National Institute of Neurological Disorders and Stroke. In 2015, over 85% of the total NIH research funding awarded to AAPM members was via these institutes, representing 1.1% of their combined budget. In the same year, 2.0% of AAPM members received NIH funding for a total of $116M, which is lower than the historic mean of $120M (in 2015 USD). CONCLUSIONS: A database of NIH-funded research awarded to AAPM members has been developed and tested using a data mining approach, and a top-level analysis of funding trends has been performed. Current funding of AAPM members is lower than the historic mean. The database will be maintained by members of the Working group for the development of a research database (WGDRD) on an annual basis, and is available to the AAPM, its committees, working groups, and members for download through the AAPM electronic content website. A wide range of questions regarding financial and demographic funding trends can be addressed by these data. This report has been approved for publication by the AAPM Science Council.

Utility of the LevelCheck Algorithm for Decision Support in Vertebral Localization.

STUDY DESIGN: An automatic radiographic labeling algorithm called "LevelCheck" was analyzed as a means of decision support for target localization in spine surgery. The potential clinical utility and scenarios in which LevelCheck is likely to be the most beneficial were assessed in a retrospective clinical data set (398 cases) in terms of expert consensus from a multi-reader study (three spine surgeons). OBJECTIVE: The aim of this study was to evaluate the potential utility of the LevelCheck algorithm for vertebrae localization. SUMMARY OF BACKGROUND DATA: Three hundred ninety-eight intraoperative radiographs and 178 preoperative computed tomographic (CT) images for patients undergoing spine surgery in cervical, thoracic, lumbar regions. METHODS: Vertebral labels annotated in preoperative CT image were overlaid on intraoperative radiographs via 3D-2D registration. Three spine surgeons assessed the radiographs and LevelCheck labeling according to a questionnaire evaluating performance, utility, and suitability to surgical workflow. Geometric accuracy and registration run time were measured for each case. RESULTS: LevelCheck was judged to be helpful in 42.2% of the cases (168/398), to improve confidence in 30.6% of the cases (122/398), and in no case diminished performance (0/398), supporting its potential as an independent check and assistant to decision support in spine surgery. The clinical contexts for which the method was judged most likely to be beneficial included the following scenarios: images with a lack of conspicuous anatomical landmarks; level counting across long spine segments; vertebrae obscured by other anatomy (e.g., shoulders); poor radiographic image quality; and anatomical variations/abnormalities. The method demonstrated 100% geometric accuracy (i.e., overlaid labels within the correct vertebral level in all cases) and did not introduce ambiguity in image interpretation. CONCLUSION: LevelCheck is a potentially useful means of decision support in vertebral level localization in spine surgery. LEVEL OF EVIDENCE: N/A.