Missing data is poorly handled and reported in prediction model studies using machine learning: a literature review.

OBJECTIVES: Missing data is a common problem during the development, evaluation, and implementation of prediction models. Although machine learning (ML) methods are often said to be capable of circumventing missing data, it is unclear how these methods are used in medical research. We aim to find out if and how well prediction model studies using machine learning report on their handling of missing data. STUDY DESIGN AND SETTING: We systematically searched the literature on published papers between 2018 and 2019 about primary studies developing and/or validating clinical prediction models using any supervised ML methodology across medical fields. From the retrieved studies information about the amount and nature (e.g. missing completely at random, potential reasons for missingness) of missing data and the way they were handled were extracted. RESULTS: We identified 152 machine learning-based clinical prediction model studies. A substantial amount of these 152 papers did not report anything on missing data (n = 56/152). A majority (n = 96/152) reported details on the handling of missing data (e.g., methods used), though many of these (n = 46/96) did not report the amount of the missingness in the data. In these 96 papers the authors only sometimes reported possible reasons for missingness (n = 7/96) and information about missing data mechanisms (n = 8/96). The most common approach for handling missing data was deletion (n = 65/96), mostly via complete-case analysis (CCA) (n = 43/96). Very few studies used multiple imputation (n = 8/96) or built-in mechanisms such as surrogate splits (n = 7/96) that directly address missing data during the development, validation, or implementation of the prediction model. CONCLUSION: Though missing values are highly common in any type of medical research and certainly in the research based on routine healthcare data, a majority of the prediction model studies using machine learning does not report sufficient information on the presence and handling of missing data. Strategies in which patient data are simply omitted are unfortunately the most often used methods, even though it is generally advised against and well known that it likely causes bias and loss of analytical power in prediction model development and in the predictive accuracy estimates. Prediction model researchers should be much more aware of alternative methodologies to address missing data.

Impact of preventive screening and lifestyle interventions in women with a history of preeclampsia: A micro-simulation study.

BACKGROUND: Preeclampsia is a female-specific risk factor for the development of future cardiovascular disease. Whether early preventive cardiovascular disease risk screenings combined with risk-based lifestyle interventions in women with previous preeclampsia are beneficial and cost-effective is unknown. METHODS: A micro-simulation model was developed to assess the life-long impact of preventive cardiovascular screening strategies initiated after women experienced preeclampsia during pregnancy. Screening was started at the age of 30 or 40 years and repeated every five years. Data (initial and follow-up) from women with a history of preeclampsia was used to calculate 10-year cardiovascular disease risk estimates according to Framingham Risk Score. An absolute risk threshold of 2% was evaluated for treatment selection, i.e. lifestyle interventions (e.g. increasing physical activity). Screening benefits were assessed in terms of costs and quality-adjusted-life-years, and incremental cost-effectiveness ratios compared with no screening. RESULTS: Expected health outcomes for no screening are 27.35 quality-adjusted-life-years and increase to 27.43 quality-adjusted-life-years (screening at 30 years with 2% threshold). The expected costs for no screening are €9426 and around €13,881 for screening at 30 years (for a 2% threshold). Preventive screening at 40 years with a 2% threshold has the most favourable incremental cost-effectiveness ratio, i.e. €34,996/quality-adjusted-life-year, compared with other screening scenarios and no screening. CONCLUSIONS: Early cardiovascular disease risk screening followed by risk-based lifestyle interventions may lead to small long-term health benefits in women with a history of preeclampsia. However, the cost-effectiveness of a lifelong cardiovascular prevention programme starting early after preeclampsia with risk-based lifestyle advice alone is relatively unfavourable. A combination of risk-based lifestyle advice plus medical therapy may be more beneficial.