An integrated approach to geographic validation helped scrutinize prediction model performance and its variability.
J Clin Epidemiol
; 157: 13-21, 2023 05.
Article
de En
| MEDLINE
| ID: mdl-36822443
ABSTRACT
OBJECTIVES:
To illustrate in-depth validation of prediction models developed on multicenter data.METHODS:
For each hospital in a multicenter registry, we evaluated predictive performance of a 30-day mortality prediction model for transcatheter aortic valve implantation (TAVI) using the Netherlands heart registration (NHR) dataset. We measured discrimination and calibration per hospital in a leave-center-out analysis (LCOA). Meta-analysis was used to calculate I2 values per performance metric from the LCOA and to compute mean and confidence interval (CI) estimates. Case mix differences between studies were inspected using the framework of Debray et al. for understanding external validation. We also aimed to discover subgroups (SGs) with high model prediction error (PE) and their distribution over the centers.RESULTS:
We studied 16 hospitals with 11,599 TAVI patients with an early mortality of 3.7%. The models' area under the curve (AUCs) had a wide range between hospitals from 0.59 to 0.79, and miscalibration occurred in seven hospitals. Mean AUC from meta-analysis was 0.68 (95% CI 0.65-0.70). I2 values were 0%, 74%, and 0% for AUC, calibration intercept and slope, respectively. Between-hospital case-mix differences were substantial, and model transportability was low. One SG was discovered with marked global PE and was associated with poor performance on validation centers.CONCLUSION:
The illustrated combination of approaches provides useful insights to inspect multicenter-based prediction models, and it exposes their limitations in transportability and performance variability when applied to different populations.Mots clés
Texte intégral:
1
Collection:
01-internacional
Base de données:
MEDLINE
Sujet principal:
Remplacement valvulaire aortique par cathéter
Type d'étude:
Clinical_trials
/
Prognostic_studies
/
Risk_factors_studies
/
Systematic_reviews
Limites:
Humans
Pays/Région comme sujet:
Europa
Langue:
En
Journal:
J Clin Epidemiol
Sujet du journal:
EPIDEMIOLOGIA
Année:
2023
Type de document:
Article