Comparison of Prediction Model Performance Updating Protocols: Using a Data-Driven Testing Procedure to Guide Updating.
AMIA Annu Symp Proc
; 2019: 1002-1010, 2019.
Article
em En
| MEDLINE
| ID: mdl-32308897
ABSTRACT
In evolving clinical environments, the accuracy of prediction models deteriorates over time. Guidance on the design of model updating policies is limited, and there is limited exploration of the impact of different policies on future model performance and across different model types. We implemented a new data-driven updating strategy based on a nonparametric testing procedure and compared this strategy to two baseline approaches in which models are never updated or fully refit annually. The test-based strategy generally recommended intermittent recalibration and delivered more highly calibrated predictions than either of the baseline strategies. The test-based strategy highlighted differences in the updating requirements between logistic regression, L1-regularized logistic regression, random forest, and neural network models, both in terms of the extent and timing of updates. These findings underscore the potential improvements in using a data-driven maintenance approach over "one-size fits all" to sustain more stable and accurate model performance over time.
Texto completo:
1
Coleções:
01-internacional
Base de dados:
MEDLINE
Assunto principal:
Modelos Estatísticos
/
Redes Neurais de Computação
/
Mortalidade Hospitalar
Tipo de estudo:
Guideline
/
Prognostic_studies
/
Risk_factors_studies
Limite:
Humans
País/Região como assunto:
America do norte
Idioma:
En
Revista:
AMIA Annu Symp Proc
Ano de publicação:
2019
Tipo de documento:
Article