Large-scale validation of the prediction model risk of bias assessment Tool (PROBAST) using a short form: high risk of bias models show poorer discrimination.

OBJECTIVE: To assess whether the Prediction model Risk Of Bias ASsessment Tool (PROBAST) and a shorter version of this tool can identify clinical prediction models (CPMs) that perform poorly at external validation. STUDY DESIGN AND SETTING: We evaluated risk of bias (ROB) on 102 CPMs from the Tufts CPM Registry, comparing PROBAST to a short form consisting of six PROBAST items anticipated to best identify high ROB. We then applied the short form to all CPMs in the Registry with at least 1 validation (n=556) and assessed the change in discrimination (dAUC) in external validation cohorts (n=1,147). RESULTS: PROBAST classified 98/102 CPMS as high ROB. The short form identified 96 of these 98 as high ROB (98% sensitivity), with perfect specificity. In the full CPM registry, 527 of 556 CPMs (95%) were classified as high ROB, 20 (3.6%) low ROB, and 9 (1.6%) unclear ROB. Only one model with unclear ROB was reclassified to high ROB after full PROBAST assessment of all low and unclear ROB models. Median change in discrimination was significantly smaller in low ROB models (dAUC -0.9%, IQR -6.2-4.2%) compared to high ROB models (dAUC -11.7%, IQR -33.3-2.6%; P<0.001). CONCLUSION: High ROB is pervasive among published CPMs. It is associated with poor discriminative performance at validation, supporting the application of PROBAST or a shorter version in CPM reviews.

Comorbidity Burden and Adverse Outcomes After Transcatheter Aortic Valve Replacement.

Background Transcatheter aortic valve replacement (TAVR) has become the preferred treatment for symptomatic patients with aortic stenosis and elevated procedural risk. Many deaths following TAVR are because of noncardiac causes and comorbid disease burden may be a major determinant of postprocedure outcomes. The prevalence of comorbid conditions and associations with outcomes after TAVR has not been studied. Methods and Results This was a retrospective single-center study of patients treated with TAVR from January 2015 to October 2018. The association between 21 chronic conditions and short- and medium-term outcomes was assessed. A total of 341 patients underwent TAVR and had 1-year follow-up. The mean age was 81.4 (SD 8.0) years with a mean Society of Thoracic Surgeons predicted risk of mortality score of 6.7% (SD 4.8). Two hundred twenty (65%) patients had ≥4 chronic conditions present at the time of TAVR. There was modest correlation between Society of Thoracic Surgeons predicted risk of mortality and comorbid disease burden (r=0.32, P<0.001). After adjusting for Society of Thoracic Surgeons predicted risk of mortality, age, and vascular access, each additional comorbid condition was associated with increased rates of 30-day rehospitalizations (odds ratio, 1.21; 95% CI, 1.02-1.44), a composite of 30-day rehospitalization and 30-day mortality (odds ratio, 1.20; 95% CI, 1.02-1.42), and 1-year mortality (odds ratio, 1.29; 95% CI, 1.05-1.59). Conclusions Comorbid disease burden is associated with worse clinical outcomes in high-risk patients treated with TAVR. The risks associated with comorbid disease burden are not adequately captured by standard risk assessment. A systematic assessment of comorbid conditions may improve risk stratification efforts.