Association between pregnancy-related complications and development of type 2 diabetes and hypertension in women: an umbrella review.

BACKGROUND: Despite many systematic reviews and meta-analyses examining the associations of pregnancy complications with risk of type 2 diabetes mellitus (T2DM) and hypertension, previous umbrella reviews have only examined a single pregnancy complication. Here we have synthesised evidence from systematic reviews and meta-analyses on the associations of a wide range of pregnancy-related complications with risk of developing T2DM and hypertension. METHODS: Medline, Embase and Cochrane Database of Systematic Reviews were searched from inception until 26 September 2022 for systematic reviews and meta-analysis examining the association between pregnancy complications and risk of T2DM and hypertension. Screening of articles, data extraction and quality appraisal (AMSTAR2) were conducted independently by two reviewers using Covidence software. Data were extracted for studies that examined the risk of T2DM and hypertension in pregnant women with the pregnancy complication compared to pregnant women without the pregnancy complication. Summary estimates of each review were presented using tables, forest plots and narrative synthesis and reported following Preferred Reporting Items for Overviews of Reviews (PRIOR) guidelines. RESULTS: Ten systematic reviews were included. Two pregnancy complications were identified. Gestational diabetes mellitus (GDM): One review showed GDM was associated with a 10-fold higher risk of T2DM at least 1 year after pregnancy (relative risk (RR) 9.51 (95% confidence interval (CI) 7.14 to 12.67) and although the association differed by ethnicity (white: RR 16.28 (95% CI 15.01 to 17.66), non-white: RR 10.38 (95% CI 4.61 to 23.39), mixed: RR 8.31 (95% CI 5.44 to 12.69)), the between subgroups difference were not statistically significant at 5% significance level. Another review showed GDM was associated with higher mean blood pressure at least 3 months postpartum (mean difference in systolic blood pressure: 2.57 (95% CI 1.74 to 3.40) mmHg and mean difference in diastolic blood pressure: 1.89 (95% CI 1.32 to 2.46) mmHg). Hypertensive disorders of pregnancy (HDP): Three reviews showed women with a history of HDP were 3 to 6 times more likely to develop hypertension at least 6 weeks after pregnancy compared to women without HDP (meta-analysis with largest number of studies: odds ratio (OR) 4.33 (3.51 to 5.33)) and one review reported a higher rate of T2DM after HDP (hazard ratio (HR) 2.24 (1.95 to 2.58)) at least a year after pregnancy. One of the three reviews and five other reviews reported women with a history of preeclampsia were 3 to 7 times more likely to develop hypertension at least 6 weeks postpartum (meta-analysis with the largest number of studies: OR 3.90 (3.16 to 4.82) with one of these reviews reporting the association was greatest in women from Asia (Asia: OR 7.54 (95% CI 2.49 to 22.81), Europe: OR 2.19 (95% CI 0.30 to 16.02), North and South America: OR 3.32 (95% CI 1.26 to 8.74)). CONCLUSIONS: GDM and HDP are associated with a greater risk of developing T2DM and hypertension. Common confounders adjusted for across the included studies in the reviews were maternal age, body mass index (BMI), socioeconomic status, smoking status, pre-pregnancy and current BMI, parity, family history of T2DM or cardiovascular disease, ethnicity, and time of delivery. Further research is needed to evaluate the value of embedding these pregnancy complications as part of assessment for future risk of T2DM and chronic hypertension.

Risk of bias assessments in individual participant data meta-analyses of test accuracy and prediction models: a review shows improvements are needed.

OBJECTIVES: Risk of bias assessments are important in meta-analyses of both aggregate and individual participant data (IPD). There is limited evidence on whether and how risk of bias of included studies or datasets in IPD meta-analyses (IPDMAs) is assessed. We review how risk of bias is currently assessed, reported, and incorporated in IPDMAs of test accuracy and clinical prediction model studies and provide recommendations for improvement. STUDY DESIGN AND SETTING: We searched PubMed (January 2018-May 2020) to identify IPDMAs of test accuracy and prediction models, then elicited whether each IPDMA assessed risk of bias of included studies and, if so, how assessments were reported and subsequently incorporated into the IPDMAs. RESULTS: Forty-nine IPDMAs were included. Nineteen of 27 (70%) test accuracy IPDMAs assessed risk of bias, compared to 5 of 22 (23%) prediction model IPDMAs. Seventeen of 19 (89%) test accuracy IPDMAs used Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2), but no tool was used consistently among prediction model IPDMAs. Of IPDMAs assessing risk of bias, 7 (37%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided details on the information sources (e.g., the original manuscript, IPD, primary investigators) used to inform judgments, and 4 (21%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided information or whether assessments were done before or after obtaining the IPD of the included studies or datasets. Of all included IPDMAs, only seven test accuracy IPDMAs (26%) and one prediction model IPDMA (5%) incorporated risk of bias assessments into their meta-analyses. For future IPDMA projects, we provide guidance on how to adapt tools such as Prediction model Risk Of Bias ASsessment Tool (for prediction models) and QUADAS-2 (for test accuracy) to assess risk of bias of included primary studies and their IPD. CONCLUSION: Risk of bias assessments and their reporting need to be improved in IPDMAs of test accuracy and, especially, prediction model studies. Using recommended tools, both before and after IPD are obtained, will address this.

Development and External Validation of Individualized Prediction Models for Pain Intensity Outcomes in Patients With Neck Pain, Low Back Pain, or Both in Primary Care Settings.

OBJECTIVE: The purpose of this study was to develop and externally validate multivariable prediction models for future pain intensity outcomes to inform targeted interventions for patients with neck or low back pain in primary care settings. METHODS: Model development data were obtained from a group of 679 adults with neck or low back pain who consulted a participating United Kingdom general practice. Predictors included self-report items regarding pain severity and impact from the STarT MSK Tool. Pain intensity at 2 and 6 months was modeled separately for continuous and dichotomized outcomes using linear and logistic regression, respectively. External validation of all models was conducted in a separate group of 586 patients recruited from a similar population with patients' predictor information collected both at point of consultation and 2 to 4 weeks later using self-report questionnaires. Calibration and discrimination of the models were assessed separately using STarT MSK Tool data from both time points to assess differences in predictive performance. RESULTS: Pain intensity and patients reporting their condition would last a long time contributed most to predictions of future pain intensity conditional on other variables. On external validation, models were reasonably well calibrated on average when using tool measurements taken 2 to 4 weeks after consultation (calibration slope = 0.848 [95% CI = 0.767 to 0.928] for 2-month pain intensity score), but performance was poor using point-of-consultation tool data (calibration slope for 2-month pain intensity score of 0.650 [95% CI = 0.549 to 0.750]). CONCLUSION: Model predictive accuracy was good when predictors were measured 2 to 4 weeks after primary care consultation, but poor when measured at the point of consultation. Future research will explore whether additional, nonmodifiable predictors improve point-of-consultation predictive performance. IMPACT: External validation demonstrated that these individualized prediction models were not sufficiently accurate to recommend their use in clinical practice. Further research is required to improve performance through inclusion of additional nonmodifiable risk factors.

Calcium supplementation to prevent pre-eclampsia: protocol for an individual participant data meta-analysis, network meta-analysis and health economic evaluation.

INTRODUCTION: Low dietary calcium intake is a risk factor for pre-eclampsia, a major contributor to maternal and perinatal mortality and morbidity worldwide. Calcium supplementation can prevent pre-eclampsia in women with low dietary calcium. However, the optimal dose and timing of calcium supplementation are not known. We plan to undertake an individual participant data (IPD) meta-analysis of randomised trials to determine the effects of various calcium supplementation regimens in preventing pre-eclampsia and its complications and rank these by effectiveness. We also aim to evaluate the cost-effectiveness of calcium supplementation to prevent pre-eclampsia. METHODS AND ANALYSIS: We will identify randomised trials on calcium supplementation before and during pregnancy by searching major electronic databases including Embase, CINAHL, MEDLINE, CENTRAL, PubMed, Scopus, AMED, LILACS, POPLINE, AIM, IMSEAR, ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform, without language restrictions, from inception to February 2022. Primary researchers of the identified trials will be invited to join the International Calcium in Pregnancy Collaborative Network and share their IPD. We will check each study's IPD for consistency with the original authors before standardising and harmonising the data. We will perform a series of one-stage and two-stage IPD random-effect meta-analyses to obtain the summary intervention effects on pre-eclampsia with 95% CIs and summary treatment-covariate interactions (maternal risk status, dietary intake, timing of intervention, daily dose of calcium prescribed and total intake of calcium). Heterogeneity will be summarised using tau2, I2 and 95% prediction intervals for effect in a new study. Sensitivity analysis to explore robustness of statistical and clinical assumptions will be carried out. Minor study effects (potential publication bias) will be investigated using funnel plots. A decision analytical model for use in low-income and middle-income countries will assess the cost-effectiveness of calcium supplementation to prevent pre-eclampsia. ETHICS AND DISSEMINATION: No ethical approvals are required. We will store the data in a secure repository in an anonymised format. The results will be published in peer-reviewed journals. PROSPERO REGISTRATION NUMBER: CRD42021231276.

Diabetic retinopathy progression in patients under monitoring for treatment or vision loss: external validation and update of a multivariable prediction model.

INTRODUCTION: The number of people with diabetes mellitus is increasing globally and consequently so too is diabetic retinopathy (DR). Most patients with diabetes are monitored through the diabetic eye screening programme (DESP) until they have signs of retinopathy and these changes progress, requiring referral into hospital eye services (HES). Here, they continue to be monitored until they require treatment. Due to current pressures on HES, delays can occur, leading to harm. There is a need to triage patients based on their individual risk. At present, patients are stratified according to retinopathy stage alone, yet other risk factors like glycated haemoglobin (HbA1c) may be useful. Therefore, a prediction model that combines multiple prognostic factors to predict progression will be useful for triage in this setting to improve care.We previously developed a Diabetic Retinopathy Progression model to Treatment or Vision Loss (DRPTVL-UK) using a large primary care database. The aim of the present study is to externally validate the DRPTVL-UK model in a secondary care setting, specifically in a population under care by HES. This study will also provide an opportunity to update the model by considering additional predictors not previously available. METHODS AND ANALYSIS: We will use a retrospective cohort of 2400 patients with diabetes aged 12 years and over, referred from DESP to the NHS hospital trusts with referable DR between 2013 and 2016, with follow-up information recorded until December 2021.We will evaluate the external validity of the DRPTVL-UK model using measures of discrimination, calibration and net benefit. In addition, consensus meetings will be held to agree on acceptable risk thresholds for triage within the HES system. ETHICS AND DISSEMINATION: This study was approved by REC (ref 22/SC/0425, 05/12/2022, Hampshire A Research Ethics Committee). The results of the study will be published in a peer-reviewed journal, presented at clinical conferences. TRIAL REGISTRATION NUMBER: ISRCTN 10956293.

Anticholinergic burden measures, symptoms, and fall-associated risk in older adults with polypharmacy: Development and validation of a prognostic model.

BACKGROUND: Anticholinergic burden has been associated with adverse outcomes such as falls. To date, no gold standard measure has been identified to assess anticholinergic burden, and no conclusion has been drawn on which of the different measure algorithms best predicts falls in older patients from general practice. This study compared the ability of five measures of anticholinergic burden to predict falls. To account for patients' individual susceptibility to medications, the added predictive value of typical anticholinergic symptoms was further quantified in this context. METHODS AND FINDINGS: To predict falls, models were developed and validated based on logistic regression models created using data from two German cluster-randomized controlled trials. The outcome was defined as "≥ 1 fall" vs. "no fall" within a 6-month follow-up period. Data from the RIME study (n = 1,197) were used in model development, and from PRIMUM (n = 502) for external validation. The models were developed step-wise in order to quantify the predictive ability of anticholinergic burden measures, and anticholinergic symptoms. In the development set, 1,015 patients had complete data and 188 (18.5%) experienced ≥ 1 fall within the 6-month follow-up period. The overall predictive value of the five anticholinergic measures was limited, with neither the employed anticholinergic variable (binary / count / burden), nor dose-dependent or dose-independent measures differing significantly in their ability to predict falls. The highest c-statistic was obtained using the German Anticholinergic Burden Score (0.73), whereby the optimism-corrected c-statistic was 0.71 after interval validation using bootstrapping and 0.63 in the external validation. Previous falls and dizziness / vertigo had the strongest prognostic value in all models. CONCLUSIONS: The ability of anticholinergic burden measures to predict falls does not appear to differ significantly, and the added value they contribute to risk classification in fall-prediction models is limited. Previous falls and dizziness / vertigo contributed most to model performance.

Development and external validation of a risk prediction model for falls in patients with an indication for antihypertensive treatment: retrospective cohort study.

OBJECTIVE: To develop and externally validate the STRAtifying Treatments In the multi-morbid Frail elderlY (STRATIFY)-Falls clinical prediction model to identify the risk of hospital admission or death from a fall in patients with an indication for antihypertensive treatment. DESIGN: Retrospective cohort study. SETTING: Primary care data from electronic health records contained within the UK Clinical Practice Research Datalink (CPRD). PARTICIPANTS: Patients aged 40 years or older with at least one blood pressure measurement between 130 mm Hg and 179 mm Hg. MAIN OUTCOME MEASURE: First serious fall, defined as hospital admission or death with a primary diagnosis of a fall within 10 years of the index date (12 months after cohort entry). Model development was conducted using a Fine-Gray approach in data from CPRD GOLD, accounting for the competing risk of death from other causes, with subsequent recalibration at one, five, and 10 years using pseudo values. External validation was conducted using data from CPRD Aurum, with performance assessed through calibration curves and the observed to expected ratio, C statistic, and D statistic, pooled across general practices, and clinical utility using decision curve analysis at thresholds around 10%. RESULTS: Analysis included 1 772 600 patients (experiencing 62 691 serious falls) from CPRD GOLD used in model development, and 3 805 366 (experiencing 206 956 serious falls) from CPRD Aurum in the external validation. The final model consisted of 24 predictors, including age, sex, ethnicity, alcohol consumption, living in an area of high social deprivation, a history of falls, multiple sclerosis, and prescriptions of antihypertensives, antidepressants, hypnotics, and anxiolytics. Upon external validation, the recalibrated model showed good discrimination, with pooled C statistics of 0.833 (95% confidence interval 0.831 to 0.835) and 0.843 (0.841 to 0.844) at five and 10 years, respectively. Original model calibration was poor on visual inspection and although this was improved with recalibration, under-prediction of risk remained (observed to expected ratio at 10 years 1.839, 95% confidence interval 1.811 to 1.865). Nevertheless, decision curve analysis suggests potential clinical utility, with net benefit larger than other strategies. CONCLUSIONS: This prediction model uses commonly recorded clinical characteristics and distinguishes well between patients at high and low risk of falls in the next 1-10 years. Although miscalibration was evident on external validation, the model still had potential clinical utility around risk thresholds of 10% and so could be useful in routine clinical practice to help identify those at high risk of falls who might benefit from closer monitoring or early intervention to prevent future falls. Further studies are needed to explore the appropriate thresholds that maximise the model's clinical utility and cost effectiveness.

Predicting relapse or recurrence of depression: systematic review of prognostic models.

BACKGROUND: Relapse and recurrence of depression are common, contributing to the overall burden of depression globally. Accurate prediction of relapse or recurrence while patients are well would allow the identification of high-risk individuals and may effectively guide the allocation of interventions to prevent relapse and recurrence. AIMS: To review prognostic models developed to predict the risk of relapse, recurrence, sustained remission, or recovery in adults with remitted major depressive disorder. METHOD: We searched the Cochrane Library (current issue); Ovid MEDLINE (1946 onwards); Ovid Embase (1980 onwards); Ovid PsycINFO (1806 onwards); and Web of Science (1900 onwards) up to May 2021. We included development and external validation studies of multivariable prognostic models. We assessed risk of bias of included studies using the Prediction model risk of bias assessment tool (PROBAST). RESULTS: We identified 12 eligible prognostic model studies (11 unique prognostic models): 8 model development-only studies, 3 model development and external validation studies and 1 external validation-only study. Multiple estimates of performance measures were not available and meta-analysis was therefore not necessary. Eleven out of the 12 included studies were assessed as being at high overall risk of bias and none examined clinical utility. CONCLUSIONS: Due to high risk of bias of the included studies, poor predictive performance and limited external validation of the models identified, presently available clinical prediction models for relapse and recurrence of depression are not yet sufficiently developed for deploying in clinical settings. There is a need for improved prognosis research in this clinical area and future studies should conform to best practice methodological and reporting guidelines.

Minimum sample size calculations for external validation of a clinical prediction model with a time-to-event outcome.

Previous articles in Statistics in Medicine describe how to calculate the sample size required for external validation of prediction models with continuous and binary outcomes. The minimum sample size criteria aim to ensure precise estimation of key measures of a model's predictive performance, including measures of calibration, discrimination, and net benefit. Here, we extend the sample size guidance to prediction models with a time-to-event (survival) outcome, to cover external validation in datasets containing censoring. A simulation-based framework is proposed, which calculates the sample size required to target a particular confidence interval width for the calibration slope measuring the agreement between predicted risks (from the model) and observed risks (derived using pseudo-observations to account for censoring) on the log cumulative hazard scale. Precise estimation of calibration curves, discrimination, and net-benefit can also be checked in this framework. The process requires assumptions about the validation population in terms of the (i) distribution of the model's linear predictor and (ii) event and censoring distributions. Existing information can inform this; in particular, the linear predictor distribution can be approximated using the C-index or Royston's D statistic from the model development article, together with the overall event risk. We demonstrate how the approach can be used to calculate the sample size required to validate a prediction model for recurrent venous thromboembolism. Ideally the sample size should ensure precise calibration across the entire range of predicted risks, but must at least ensure adequate precision in regions important for clinical decision-making. Stata and R code are provided.

Lessons learnt when accounting for competing events in the external validation of time-to-event prognostic models.

BACKGROUND: External validation of prognostic models is necessary to assess the accuracy and generalizability of the model to new patients. If models are validated in a setting in which competing events occur, these competing risks should be accounted for when comparing predicted risks to observed outcomes. METHODS: We discuss existing measures of calibration and discrimination that incorporate competing events for time-to-event models. These methods are illustrated using a clinical-data example concerning the prediction of kidney failure in a population with advanced chronic kidney disease (CKD), using the guideline-recommended Kidney Failure Risk Equation (KFRE). The KFRE was developed using Cox regression in a diverse population of CKD patients and has been proposed for use in patients with advanced CKD in whom death is a frequent competing event. RESULTS: When validating the 5-year KFRE with methods that account for competing events, it becomes apparent that the 5-year KFRE considerably overestimates the real-world risk of kidney failure. The absolute overestimation was 10%age points on average and 29%age points in older high-risk patients. CONCLUSIONS: It is crucial that competing events are accounted for during external validation to provide a more reliable assessment the performance of a model in clinical settings in which competing risks occur.

Predicting hospital admissions from individual patient data (IPD): an applied example to explore key elements driving external validity.

OBJECTIVE: To explore factors that potentially impact external validation performance while developing and validating a prognostic model for hospital admissions (HAs) in complex older general practice patients. STUDY DESIGN AND SETTING: Using individual participant data from four cluster-randomised trials conducted in the Netherlands and Germany, we used logistic regression to develop a prognostic model to predict all-cause HAs within a 6-month follow-up period. A stratified intercept was used to account for heterogeneity in baseline risk between the studies. The model was validated both internally and by using internal-external cross-validation (IECV). RESULTS: Prior HAs, physical components of the health-related quality of life comorbidity index, and medication-related variables were used in the final model. While achieving moderate discriminatory performance, internal bootstrap validation revealed a pronounced risk of overfitting. The results of the IECV, in which calibration was highly variable even after accounting for between-study heterogeneity, agreed with this finding. Heterogeneity was equally reflected in differing baseline risk, predictor effects and absolute risk predictions. CONCLUSIONS: Predictor effect heterogeneity and differing baseline risk can explain the limited external performance of HA prediction models. With such drivers known, model adjustments in external validation settings (eg, intercept recalibration, complete updating) can be applied more purposefully. TRIAL REGISTRATION NUMBER: PROSPERO id: CRD42018088129.

The development and validation of a prognostic model to PREDICT Relapse of depression in adult patients in primary care: protocol for the PREDICTR study.

BACKGROUND: Most patients who present with depression are treated in primary care by general practitioners (GPs). Relapse of depression is common (at least 50% of patients treated for depression will relapse after a single episode) and leads to considerable morbidity and decreased quality of life for patients. The majority of patients will relapse within 6 months, and those with a history of relapse are more likely to relapse in the future than those with no such history. GPs see a largely undifferentiated case-mix of patients, and once patients with depression reach remission, there is limited guidance to help GPs stratify patients according to risk of relapse. We aim to develop a prognostic model to predict an individual's risk of relapse within 6-8 months of entering remission. The long-term objective is to inform the clinical management of depression after the acute phase. METHODS: We will develop a prognostic model using secondary analysis of individual participant data drawn from seven RCTs and one longitudinal cohort study in primary or community care settings. We will use logistic regression to predict the outcome of relapse of depression within 6-8 months. We plan to include the following established relapse predictors in the model: residual depressive symptoms, number of previous depressive episodes, co-morbid anxiety and severity of index episode. We will use a "full model" development approach, including all available predictors. Performance statistics (optimism-adjusted C-statistic, calibration-in-the-large, calibration slope) and calibration plots (with smoothed calibration curves) will be calculated. Generalisability of predictive performance will be assessed through internal-external cross-validation. Clinical utility will be explored through net benefit analysis. DISCUSSION: We will derive a statistical model to predict relapse of depression in remitted depressed patients in primary care. Assuming the model has sufficient predictive performance, we outline the next steps including independent external validation and further assessment of clinical utility and impact. STUDY REGISTRATION: ClinicalTrials.gov ID: NCT04666662.

Minimum sample size for external validation of a clinical prediction model with a binary outcome.

In prediction model research, external validation is needed to examine an existing model's performance using data independent to that for model development. Current external validation studies often suffer from small sample sizes and consequently imprecise predictive performance estimates. To address this, we propose how to determine the minimum sample size needed for a new external validation study of a prediction model for a binary outcome. Our calculations aim to precisely estimate calibration (Observed/Expected and calibration slope), discrimination (C-statistic), and clinical utility (net benefit). For each measure, we propose closed-form and iterative solutions for calculating the minimum sample size required. These require specifying: (i) target SEs (confidence interval widths) for each estimate of interest, (ii) the anticipated outcome event proportion in the validation population, (iii) the prediction model's anticipated (mis)calibration and variance of linear predictor values in the validation population, and (iv) potential risk thresholds for clinical decision-making. The calculations can also be used to inform whether the sample size of an existing (already collected) dataset is adequate for external validation. We illustrate our proposal for external validation of a prediction model for mechanical heart valve failure with an expected outcome event proportion of 0.018. Calculations suggest at least 9835 participants (177 events) are required to precisely estimate the calibration and discrimination measures, with this number driven by the calibration slope criterion, which we anticipate will often be the case. Also, 6443 participants (116 events) are required to precisely estimate net benefit at a risk threshold of 8%. Software code is provided.

Internal-external cross-validation helped to evaluate the generalizability of prediction models in large clustered datasets.

OBJECTIVE: To illustrate how to evaluate the need of complex strategies for developing generalizable prediction models in large clustered datasets. STUDY DESIGN AND SETTING: We developed eight Cox regression models to estimate the risk of heart failure using a large population-level dataset. These models differed in the number of predictors, the functional form of the predictor effects (non-linear effects and interaction) and the estimation method (maximum likelihood and penalization). Internal-external cross-validation was used to evaluate the models' generalizability across the included general practices. RESULTS: Among 871,687 individuals from 225 general practices, 43,987 (5.5%) developed heart failure during a median follow-up time of 5.8 years. For discrimination, the simplest prediction model yielded a good concordance statistic, which was not much improved by adopting complex strategies. Between-practice heterogeneity in discrimination was similar in all models. For calibration, the simplest model performed satisfactorily. Although accounting for non-linear effects and interaction slightly improved the calibration slope, it also led to more heterogeneity in the observed/expected ratio. Similar results were found in a second case study involving patients with stroke. CONCLUSION: In large clustered datasets, prediction model studies may adopt internal-external cross-validation to evaluate the generalizability of competing models, and to identify promising modelling strategies.