Establishment and evaluation of a clinical prediction model for cognitive impairment in patients with cerebral small vessel disease.

BACKGROUND: There are currently no effective prediction methods for evaluating the occurrence of cognitive impairment in patients with cerebral small vessel disease (CSVD). AIMS: To investigate the risk factors for cognitive dysfunction in patients with CSVD and to construct a risk prediction model. METHODS: A retrospective study was conducted on 227 patients with CSVD. All patients were assessed by brain magnetic resonance imaging (MRI), and the Montreal Cognitive Assessment (MoCA) was used to assess cognitive status. In addition, the patient's medical records were also recorded. The clinical data were divided into a normal cognitive function group and a cognitive impairment group. A MoCA score < 26 (an additional 1 point for education < 12 years) is defined as cognitive dysfunction. RESULTS: A total of 227 patients (mean age 66.7 ± 6.99 years) with CSVD were included in this study, of whom 68.7% were male and 100 patients (44.1%) developed cognitive impairment. Age (OR = 1.070; 95% CI = 1.015 ~ 1.128, p < 0.05), hypertension (OR = 2.863; 95% CI = 1.438 ~ 5.699, p < 0.05), homocysteine(HCY) (OR = 1.065; 95% CI = 1.005 ~ 1.127, p < 0.05), lacunar infarct score(Lac_score) (OR = 2.732; 95% CI = 1.094 ~ 6.825, P < 0.05), and CSVD total burden (CSVD_score) (OR = 3.823; 95% CI = 1.496 ~ 9.768, P < 0.05) were found to be independent risk factors for cognitive decline in the present study. The above 5 variables were used to construct a nomogram, and the model was internally validated by using bootstrapping with a C-index of 0.839. The external model validation C-index was 0.867. CONCLUSIONS: The nomogram model based on brain MR images and clinical data helps in individualizing the probability of cognitive impairment progression in patients with CSVD.

Development of an IVF prediction model for donor oocytes: a retrospective analysis of 10 877 embryo transfers.

STUDY QUESTION: Can we develop a prediction model for the chance of a live birth following the transfer of an embryo created using donated oocytes? SUMMARY ANSWER: Three primary models that included patient, past treatment, and cycle characteristics were developed using Australian data to predict the chance of a live birth following the transfer of an embryo created using donated oocytes; these models were well-calibrated to the population studied, achieved reasonable predictive power and generalizability when tested on New Zealand data. WHAT IS KNOWN ALREADY: Nearly 9% of ART embryo transfer cycles performed globally use embryos created using donated oocytes. This percentage rises to one-quarter and one-half in same-sex couples and women aged over 45 years, respectively. STUDY DESIGN, SIZE, DURATION: This study uses population-based Australian clinical registry data comprising 9384 embryo transfer cycles that occurred between 2015 and 2021 for model development, with an external validation cohort of 1493 New Zealand embryo transfer cycles. PARTICIPANTS/MATERIALS, SETTING, METHODS: Three prediction models were compared that incorporated patient characteristics, but differed in whether they considered use of prior autologous treatment factors and current treatment parameters. We internally validated the models on Australian data using grouped cross-validation and reported several measures of model discrimination and calibration. Variable importance was measured through calculating the change in predictive performance that resulted from variable permutation. The best-performing model was externally validated on data from New Zealand. MAIN RESULTS AND THE ROLE OF CHANCE: The best-performing model had an internal validation AUC-ROC of 0.60 and Brier score of 0.20, and external validation AUC-ROC of 0.61 and Brier score of 0.23. While these results indicate ∼15% less discriminatory ability compared to models assessed on an autologous cohort from the same population the performance of the models was clearly statistically significantly better than random, demonstrated generalizability, and was well-calibrated to the population studied. The most important variables for predicting the chance of a live birth were the oocyte donor age, the number of prior oocyte recipient embryo transfer cycles, whether the transferred embryo was cleavage or blastocyst stage and oocyte recipient age. Of lesser importance were the oocyte-recipient parity, whether donor or partner sperm was used, the number of prior autologous embryo transfer cycles and the number of embryos transferred. LIMITATIONS, REASONS FOR CAUTION: The models had relatively weak discrimination suggesting further features need to be added to improve their predictive power. Variation in donor oocyte cohorts across countries due to differences such as whether anonymous and compensated donation are allowed may necessitate the models be recalibrated prior to application in non-Australian cohorts. WIDER IMPLICATIONS OF THE FINDINGS: These results confirm the well-established importance of oocyte age and ART treatment history as the key prognostic factors in predicting treatment outcomes. One of the developed models has been incorporated into a consumer-facing website (YourIVFSuccess.com.au/Estimator) to allow patients to obtain personalized estimates of their chance of success using donor oocytes. STUDY FUNDING/COMPETING INTEREST(S): This research was funded by the Australian government as part of the Medical Research Future Fund (MRFF) Emerging Priorities and Consumer Driven Research initiative: EPCD000007. L.R. declares personal consulting fees from Abbott and Merck, lecture fees from Abbott, receipt of an educational grant from Merck, past presidency of the Fertility Society of Australia & New Zealand and World Endometriosis Society and being a minor shareholder in Monash IVF Group (ASX:MVF). G.M.C. declares receipt of Australian government grant funding for the research study and the development and maintenance of the YourIVFSuccess website. O.F., J.N., and A.P. report no conflicts of interest. TRIAL REGISTRATION NUMBER: N/A.

A predictive model for depression in Chinese middle-aged and elderly people with physical disabilities.

BACKGROUND: Middle-aged and older adults with physical disabilities exhibit more common and severe depressive symptoms than those without physical disabilities. Such symptoms can greatly affect the physical and mental health and life expectancy of middle-aged and older persons with disabilities. METHOD: This study selected 2015 and 2018 data from the China Longitudinal Study of Health and Retirement. After analyzing the effect of age on depression, we used whether middle-aged and older adults with physical disabilities were depressed as the dependent variable and included a total of 24 predictor variables, including demographic factors, health behaviors, physical functioning and socialization, as independent variables. The data were randomly divided into training and validation sets on a 7:3 basis. LASSO regression analysis combined with binary logistic regression analysis was performed in the training set to screen the predictor variables of the model. Construct models in the training set and perform model evaluation, model visualization and internal validation. Perform external validation of the model in the validation set. RESULT: A total of 1052 middle-aged and elderly persons with physical disabilities were included in this study, and the prevalence of depression in the elderly group > middle-aged group. Restricted triple spline indicated that age had different effects on depression in the middle-aged and elderly groups. LASSO regression analysis combined with binary logistic regression screened out Gender, Location of Residential Address, Shortsightedness, Hearing, Any possible helper in the future, Alcoholic in the Past Year, Difficulty with Using the Toilet, Difficulty with Preparing Hot Meals, and Unable to work due to disability constructed the Chinese Depression Prediction Model for Middle-aged and Older People with Physical Disabilities. The nomogram shows that living in a rural area, lack of assistance, difficulties with activities of daily living, alcohol abuse, visual and hearing impairments, unemployment and being female are risk factors for depression in middle-aged and older persons with physical disabilities. The area under the ROC curve for the model, internal validation and external validation were all greater than 0.70, the mean absolute error was less than 0.02, and the recall and precision were both greater than 0.65, indicating that the model performs well in terms of discriminability, accuracy and generalisation. The DCA curve and net gain curve of the model indicate that the model has high gain in predicting depression. CONCLUSION: In this study, we showed that being female, living in rural areas, having poor vision and/or hearing, lack of assistance from others, drinking alcohol, having difficulty using the restroom and preparing food, and being unable to work due to a disability were risk factors for depression among middle-aged and older adults with physical disabilities. We developed a depression prediction model to assess the likelihood of depression in Chinese middle-aged and older adults with physical disabilities based on the above risk factors, so that early identification, intervention, and treatment can be provided to middle-aged and older adults with physical disabilities who are at high risk of developing depression.

Clinician perspectives and recommendations regarding design of clinical prediction models for deteriorating patients in acute care.

BACKGROUND: Successful deployment of clinical prediction models for clinical deterioration relates not only to predictive performance but to integration into the decision making process. Models may demonstrate good discrimination and calibration, but fail to match the needs of practising acute care clinicians who receive, interpret, and act upon model outputs or alerts. We sought to understand how prediction models for clinical deterioration, also known as early warning scores (EWS), influence the decision-making of clinicians who regularly use them and elicit their perspectives on model design to guide future deterioration model development and implementation. METHODS: Nurses and doctors who regularly receive or respond to EWS alerts in two digital metropolitan hospitals were interviewed for up to one hour between February 2022 and March 2023 using semi-structured formats. We grouped interview data into sub-themes and then into general themes using reflexive thematic analysis. Themes were then mapped to a model of clinical decision making using deductive framework mapping to develop a set of practical recommendations for future deterioration model development and deployment. RESULTS: Fifteen nurses (n = 8) and doctors (n = 7) were interviewed for a mean duration of 42 min. Participants emphasised the importance of using predictive tools for supporting rather than supplanting critical thinking, avoiding over-protocolising care, incorporating important contextual information and focusing on how clinicians generate, test, and select diagnostic hypotheses when managing deteriorating patients. These themes were incorporated into a conceptual model which informed recommendations that clinical deterioration prediction models demonstrate transparency and interactivity, generate outputs tailored to the tasks and responsibilities of end-users, avoid priming clinicians with potential diagnoses before patients were physically assessed, and support the process of deciding upon subsequent management. CONCLUSIONS: Prediction models for deteriorating inpatients may be more impactful if they are designed in accordance with the decision-making processes of acute care clinicians. Models should produce actionable outputs that assist with, rather than supplant, critical thinking.

Clinical prediction models for the management of blunt chest trauma in the emergency department: a systematic review.

BACKGROUND: The aim of this systematic review was to investigate how clinical prediction models compare in terms of their methodological development, validation, and predictive capabilities, for patients with blunt chest trauma presenting to the Emergency Department. METHODS: A systematic review was conducted across databases from 1st Jan 2000 until 1st April 2024. Studies were categorised into three types of multivariable prediction research and data extracted regarding methodological issues and the predictive capabilities of each model. Risk of bias and applicability were assessed. RESULTS: 41 studies were included that discussed 22 different models. The most commonly observed study design was a single-centre, retrospective, chart review. The most widely externally validated clinical prediction models with moderate to good discrimination were the Thoracic Trauma Severity Score and the STUMBL Score. DISCUSSION: This review demonstrates that the predictive ability of some of the existing clinical prediction models is acceptable, but high risk of bias and lack of subsequent external validation limits the extensive application of the models. The Thoracic Trauma Severity Score and STUMBL Score demonstrate better predictive accuracy in both development and external validation studies than the other models, but require recalibration and / or update and evaluation of their clinical and cost effectiveness. REVIEW REGISTRATION: PROSPERO database ( https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=351638 ).

Clinical prediction models for serious infections in children: external validation in ambulatory care.

BACKGROUND: Early distinction between mild and serious infections (SI) is challenging in children in ambulatory care. Clinical prediction models (CPMs), developed to aid physicians in clinical decision-making, require broad external validation before clinical use. We aimed to externally validate four CPMs, developed in emergency departments, in ambulatory care. METHODS: We applied the CPMs in a prospective cohort of acutely ill children presenting to general practices, outpatient paediatric practices or emergency departments in Flanders, Belgium. For two multinomial regression models, Feverkidstool and Craig model, discriminative ability and calibration were assessed, and a model update was performed by re-estimation of coefficients with correction for overfitting. For two risk scores, the SBI score and PAWS, the diagnostic test accuracy was assessed. RESULTS: A total of 8211 children were included, comprising 498 SI and 276 serious bacterial infections (SBI). Feverkidstool had a C-statistic of 0.80 (95% confidence interval 0.77-0.84) with good calibration for pneumonia and 0.74 (0.70-0.79) with poor calibration for other SBI. The Craig model had a C-statistic of 0.80 (0.77-0.83) for pneumonia, 0.75 (0.70-0.80) for complicated urinary tract infections and 0.63 (0.39-0.88) for bacteraemia, with poor calibration. The model update resulted in improved C-statistics for all outcomes and good overall calibration for Feverkidstool and the Craig model. SBI score and PAWS performed extremely weak with sensitivities of 0.12 (0.09-0.15) and 0.32 (0.28-0.37). CONCLUSIONS: Feverkidstool and the Craig model show good discriminative ability for predicting SBI and a potential for early recognition of SBI, confirming good external validity in a low prevalence setting of SBI. The SBI score and PAWS showed poor diagnostic performance. TRIAL REGISTRATION: ClinicalTrials.gov, NCT02024282. Registered on 31 December 2013.

External validation and extension of the TIMI risk score for heart failure in diabetes for patients with recent acute coronary syndrome: An analysis of the EXAMINE trial.

AIMS: The Thrombolysis in Myocardial Infarction Risk Score for Heart Failure (HF) in Diabetes (TRS-HFDM ) prognosticates HF hospitalization in people with type 2 diabetes (T2D). This study aimed to externally validate and extend its use for those with recent acute coronary syndrome (ACS). MATERIALS AND METHODS: The TRS-HFDM was externally validated in the Examination of Cardiovascular Outcomes with Alogliptin versus Standard of Care (EXAMINE) trial (n = 5380) and extended with natriuretic biomarkers. Missing data were multiply imputed. Initial TRS-HFDM variables were previous HF (2 points), atrial fibrillation (1 point), coronary artery disease (1 point), estimated glomerular filtration rate <60 ml/min/1.73 m2 (1 point), and urine albumin-to-creatinine ratio 30-300 mg/g (1 point) and >300 mg/g (2 points). RESULTS: In total, HF hospitalization occurred in 193 (3.6%) patients. Based on the TRS-HFDM , 25% of patients were classified as intermediate risk (1 point), 30% were classified as high risk (2 points), 19% were classified as very-high risk (3 points) and 26% were classified as severe risk (≥4 points). Before model extension, discrimination (C-index 0.76, 95%·CI 0.73-0.80) and calibration (calibration slope 0.82, 95%·CI 0.65-1.0; calibration-in-the-large -0.15, 95%·CI -0.37-0.64) were moderate-to-good in individuals with T2D and recent ACS. The extension of TRS-HFDM with the addition of N-terminal pro-B-type natriuretic peptide (NT-ProBNP) improved discrimination (C-index 0.82, 95%·CI 0.79-0.85) and calibration (calibration slope 0.84, 95%·CI 0.66-1.02; calibration-in-the-large -0.12, 95%·CI -0.33-0.081) for this higher-risk population. CONCLUSION: The TRS-HFDM with the extension of NT-ProBNP improves risk stratification and generalizes the use of the risk score for patients with T2D and ACS. Future validation studies in ACS populations may be warranted.

Attention-based neural networks for clinical prediction modelling on electronic health records.

BACKGROUND: Deep learning models have had a lot of success in various fields. However, on structured data they have struggled. Here we apply four state-of-the-art supervised deep learning models using the attention mechanism and compare against logistic regression and XGBoost using discrimination, calibration and clinical utility. METHODS: We develop the models using a general practitioners database. We implement a recurrent neural network, a transformer with and without reverse distillation and a graph neural network. We measure discrimination using the area under the receiver operating characteristic curve (AUC) and the area under the precision recall curve (AUPRC). We assess smooth calibration using restricted cubic splines and clinical utility with decision curve analysis. RESULTS: Our results show that deep learning approaches can improve discrimination up to 2.5% points AUC and 7.4% points AUPRC. However, on average the baselines are competitive. Most models are similarly calibrated as the baselines except for the graph neural network. The transformer using reverse distillation shows the best performance in clinical utility on two out of three prediction problems over most of the prediction thresholds. CONCLUSION: In this study, we evaluated various approaches in supervised learning using neural networks and attention. Here we do a rigorous comparison, not only looking at discrimination but also calibration and clinical utility. There is value in using deep learning models on electronic health record data since it can improve discrimination and clinical utility while providing good calibration. However, good baseline methods are still competitive.

Inpatient Fall Prediction Models: A Scoping Review.

INTRODUCTION: The digitization of hospital systems, including integrated electronic medical records, has provided opportunities to improve the prediction performance of inpatient fall risk models and their application to computerized clinical decision support systems. This review describes the data sources and scope of methods reported in studies that developed inpatient fall prediction models, including machine learning and more traditional approaches to inpatient fall risk prediction. METHODS: This scoping review used methods recommended by the Arksey and O'Malley framework and its recent advances. PubMed, CINAHL, IEEE Xplore, and EMBASE databases were systematically searched. Studies reporting the development of inpatient fall risk prediction approaches were included. There was no restriction on language or recency. Reference lists and manual searches were also completed. Reporting quality was assessed using adherence to Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis statement (TRIPOD), where appropriate. RESULTS: Database searches identified 1,396 studies, 63 were included for scoping assessment and 45 for reporting quality assessment. There was considerable overlap in data sources and methods used for model development. Fall prediction models typically relied on features from patient assessments, including indicators of physical function or impairment, or cognitive function or impairment. All but two studies used patient information at or soon after admission and predicted fall risk over the entire admission, without consideration of post-admission interventions, acuity changes or length of stay. Overall, reporting quality was poor, but improved in the past decade. CONCLUSION: There was substantial homogeneity in data sources and prediction model development methods. Use of artificial intelligence, including machine learning with high-dimensional data, remains underexplored in the context of hospital falls. Future research should consider approaches with the potential to utilize high-dimensional data from digital hospital systems, which may contribute to greater performance and clinical usefulness.

Prognostic models for COVID-19 needed updating to warrant transportability over time and space.

BACKGROUND: Supporting decisions for patients who present to the emergency department (ED) with COVID-19 requires accurate prognostication. We aimed to evaluate prognostic models for predicting outcomes in hospitalized patients with COVID-19, in different locations and across time. METHODS: We included patients who presented to the ED with suspected COVID-19 and were admitted to 12 hospitals in the New York City (NYC) area and 4 large Dutch hospitals. We used second-wave patients who presented between September and December 2020 (2137 and 3252 in NYC and the Netherlands, respectively) to evaluate models that were developed on first-wave patients who presented between March and August 2020 (12,163 and 5831). We evaluated two prognostic models for in-hospital death: The Northwell COVID-19 Survival (NOCOS) model was developed on NYC data and the COVID Outcome Prediction in the Emergency Department (COPE) model was developed on Dutch data. These models were validated on subsequent second-wave data at the same site (temporal validation) and at the other site (geographic validation). We assessed model performance by the Area Under the receiver operating characteristic Curve (AUC), by the E-statistic, and by net benefit. RESULTS: Twenty-eight-day mortality was considerably higher in the NYC first-wave data (21.0%), compared to the second-wave (10.1%) and the Dutch data (first wave 10.8%; second wave 10.0%). COPE discriminated well at temporal validation (AUC 0.82), with excellent calibration (E-statistic 0.8%). At geographic validation, discrimination was satisfactory (AUC 0.78), but with moderate over-prediction of mortality risk, particularly in higher-risk patients (E-statistic 2.9%). While discrimination was adequate when NOCOS was tested on second-wave NYC data (AUC 0.77), NOCOS systematically overestimated the mortality risk (E-statistic 5.1%). Discrimination in the Dutch data was good (AUC 0.81), but with over-prediction of risk, particularly in lower-risk patients (E-statistic 4.0%). Recalibration of COPE and NOCOS led to limited net benefit improvement in Dutch data, but to substantial net benefit improvement in NYC data. CONCLUSIONS: NOCOS performed moderately worse than COPE, probably reflecting unique aspects of the early pandemic in NYC. Frequent updating of prognostic models is likely to be required for transportability over time and space during a dynamic pandemic.

Multicenter Validation of Individual Preoperative Motor Outcome Prediction for Deep Brain Stimulation in Parkinson's Disease.

BACKGROUND: Subthalamic nucleus deep brain stimulation (STN DBS) is an established therapy for Parkinson's disease (PD) patients suffering from motor response fluctuations despite optimal medical treatment, or severe dopaminergic side effects. Despite careful clinical selection and surgical procedures, some patients do not benefit from STN DBS. Preoperative prediction models are suggested to better predict individual motor response after STN DBS. We validate a preregistered model, DBS-PREDICT, in an external multicenter validation cohort. METHODS: DBS-PREDICT considered eleven, solely preoperative, clinical characteristics and applied a logistic regression to differentiate between weak and strong motor responders. Weak motor response was defined as no clinically relevant improvement on the Unified Parkinson's Disease Rating Scale (UPDRS) II, III, or IV, 1 year after surgery, defined as, respectively, 3, 5, and 3 points or more. Lower UPDRS III and IV scores and higher age at disease onset contributed most to weak response predictions. Individual predictions were compared with actual clinical outcomes. RESULTS: 322 PD patients treated with STN DBS from 6 different centers were included. DBS-PREDICT differentiated between weak and strong motor responders with an area under the receiver operator curve of 0.76 and an accuracy up to 77%. CONCLUSION: Proving generalizability and feasibility of preoperative STN DBS outcome prediction in an external multicenter cohort is an important step in creating clinical impact in DBS with data-driven tools. Future prospective studies are required to overcome several inherent practical and statistical limitations of including clinical decision support systems in DBS care.

Development and Internal Validation of a Prediction Model to Risk Stratify Children With Suspected Community-Acquired Pneumonia.

BACKGROUND: Although community-acquired pneumonia (CAP) is one of the most common infections in children, no tools exist to risk stratify children with suspected CAP. We developed and validated a prediction model to risk stratify and inform hospitalization decisions in children with suspected CAP. METHODS: We performed a prospective cohort study of children aged 3 months to 18 years with suspected CAP in a pediatric emergency department. Primary outcome was disease severity, defined as mild (discharge home or hospitalization for <24 hours with no oxygen or intravenous [IV] fluids), moderate (hospitalization <24 hours with oxygen or IV fluids, or hospitalization >24 hours), or severe (intensive care unit stay for >24 hours, septic shock, vasoactive agents, positive-pressure ventilation, chest drainage, extracorporeal membrane oxygenation, or death). Ordinal logistic regression and bootstrapped backwards selection were used to derive and internally validate our model. RESULTS: Of 1128 children, 371 (32.9%) developed moderate disease and 48 (4.3%) severe disease. Severity models demonstrated excellent discrimination (optimism-corrected c-indices of 0.81) and outstanding calibration. Severity predictors in the final model included respiratory rate, systolic blood pressure, oxygenation, retractions, capillary refill, atelectasis or pneumonia on chest radiograph, and pleural effusion. CONCLUSIONS: We derived and internally validated a score that accurately predicts disease severity in children with suspected CAP. Once externally validated, this score has potential to facilitate management decisions by providing individualized risk estimates that can be used in conjunction with clinical judgment to improve the care of children with suspected CAP.

Predictive models of infection in patients with systemic lupus erythematosus: A systematic literature review.

INTRODUCTION: Having reliable predictive models of prognosis/the risk of infection in systemic lupus erythematosus (SLE) patients would allow this problem to be addressed on an individual basis to study and implement possible preventive or therapeutic interventions. OBJECTIVE: To identify and analyze all predictive models of prognosis/the risk of infection in patients with SLE that exist in medical literature. METHODS: A structured search in PubMed, Embase, and LILACS databases was carried out until May 9, 2020. In addition, a search for abstracts in the American Congress of Rheumatology (ACR) and European League Against Rheumatism (EULAR) annual meetings' archives published over the past eight years was also conducted. Studies on developing, validating or updating predictive prognostic models carried out in patients with SLE, in which the outcome to be predicted is some type of infection, that were generated in any clinical context and with any time horizon were included. There were no restrictions on language, date, or status of the publication. To carry out the systematic review, the CHARMS (Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies) guideline recommendations were followed. The PROBAST tool (A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies) was used to assess the risk of bias and the applicability of each model. RESULTS: We identified four models of infection prognosis in patients with SLE. Mostly, there were very few events per candidate predictor. In addition, to construct the models, an initial selection was made based on univariate analyses with no contraction of the estimated coefficients being carried out. This suggests that the proposed models have a high probability of overfitting and being optimistic. CONCLUSIONS: To date, very few prognostic models have been published on the infection of SLE patients. These models are very heterogeneous and are rated as having a high risk of bias and methodological weaknesses. Despite the widespread recognition of the frequency and severity of infections in SLE patients, there is no reliable predictive prognostic model that facilitates the study and implementation of personalized preventive or therapeutic measures.Protocol registration number: PROSPERO CRD42020171638.

Predicting the chances of having a baby with or without treatment at different time points in couples with unexplained subfertility.

STUDY QUESTION: Can we develop a prediction model that can estimate the chances of conception leading to live birth with and without treatment at different points in time in couples with unexplained subfertility? SUMMARY ANSWER: Yes, a dynamic model was developed that predicted the probability of conceiving under expectant management and following active treatments (in vitro fertilisation (IVF), intrauterine insemination with ovarian stimulation (IUI + SO), clomiphene) at different points in time since diagnosis. WHAT IS KNOWN ALREADY: Couples with no identified cause for their subfertility continue to have a realistic chance of conceiving naturally, which makes it difficult for clinicians to decide when to intervene. Previous fertility prediction models have attempted to address this by separately estimating either the chances of natural conception or the chances of conception following certain treatments. These models only make predictions at a single point in time and are therefore inadequate for informing continued decision-making at subsequent consultations. STUDY DESIGN, SIZE, DURATION: A population-based study of 1316 couples with unexplained subfertility attending a regional clinic between 1998 and 2011. PARTICIPANTS/MATERIALS, SETTING, METHODS: A dynamic prediction model was developed that estimates the chances of conception within 6 months from the point when a diagnosis of unexplained subfertility was made. These predictions were recomputed each month to provide a dynamic assessment of the individualised chances of conception while taking account of treatment status in each month. Conception must have led to live birth and treatments included clomiphene, IUI + SO, and IVF. Predictions for natural conception were externally validated using a prospective cohort from The Netherlands. MAIN RESULTS AND THE ROLE OF CHANCE: A total of 554 (42%) couples started fertility treatment within 2 years of their first fertility consultation. The natural conception leading to live birth rate was 0.24 natural conceptions per couple per year. Active treatment had a higher chance of conception compared to those who remained under expectant management. This association ranged from weak with clomiphene to strong with IVF [clomiphene, hazard ratio (HR) = 1.42 (95% confidence interval, 1.05 to 1.91); IUI + SO, HR = 2.90 (2.06 to 4.08); IVF, HR = 5.09 (4.04 to 6.40)]. Female age and duration of subfertility were significant predictors, without clear interaction with the relative effect of treatment. LIMITATIONS, REASONS FOR CAUTION: We were unable to adjust for other potentially important predictors, e.g. measures of ovarian reserve, which were not available in the linked Grampian dataset that may have made predictions more specific. This study was conducted using single centre data meaning that it may not be generalizable to other centres. However, the model performed as well as previous models in reproductive medicine when externally validated using the Dutch cohort. WIDER IMPLICATIONS OF THE FINDINGS: For the first time, it is possible to estimate the chances of conception following expectant management and different fertility treatments over time in couples with unexplained subfertility. This information will help inform couples and their clinicians of their likely chances of success, which may help manage expectations, not only at diagnostic workup completion but also throughout their fertility journey. STUDY FUNDING/COMPETING INTEREST(S): This work was supported by a Chief Scientist Office postdoctoral training fellowship in health services research and health of the public research (ref PDF/12/06). B.W.M. is supported by an NHMRC Practitioner Fellowship (GNT1082548). B.W.M. reports consultancy for ObsEva, Merck, and Guerbet. None of the other authors declare any conflicts of interest.

What are medical students' attitudes to clinical risk-scoring tools? An exploratory study.

Background: Clinical risk-scoring tools are increasingly recommended for use in general practice. Yet adoption of the tools has been variable and often low. Reasons for this have been explored, but medical students' perspectives have not previously been sought.Aim: To explore medical students' attitudes towards clinical risk-scoring tools.Methods: Qualitative, semi-structured interviews were conducted with eight medical students. Interviews were recorded, transcribed and analysed thematically.Results: Participants had a good understanding of the use of risk-scoring tools. They would trust them to enable evidence-based practice provided they are easy to use, not time-consuming and their results can help direct management. They were considered useful tools, especially for students and junior doctors. However, many believed the tools hold less value for experienced doctors. Their attitudes seem to have developed from discussions with clinicians, observation on placement, teaching received and exam content.Conclusion: This research recommends that implementation of risk-scoring tools will be increased if they are easier to use and if the belief that they hold less value for experienced doctors is challenged. The role of targeted teaching in changing these perceptions should be explored further, both for students and clinicians, who act as role models.

A multiple-model generalisation of updating clinical prediction models.

There is growing interest in developing clinical prediction models (CPMs) to aid local healthcare decision-making. Frequently, these CPMs are developed in isolation across different populations, with repetitive de novo derivation a common modelling strategy. However, this fails to utilise all available information and does not respond to changes in health processes through time and space. Alternatively, model updating techniques have previously been proposed that adjust an existing CPM to suit the new population, but these techniques are restricted to a single model. Therefore, we aimed to develop a generalised method for updating and aggregating multiple CPMs. The proposed "hybrid method" re-calibrates multiple CPMs using stacked regression while concurrently revising specific covariates using individual participant data (IPD) under a penalised likelihood. The performance of the hybrid method was compared with existing methods in a clinical example of mortality risk prediction after transcatheter aortic valve implantation, and in 2 simulation studies. The simulation studies explored the effect of sample size and between-population-heterogeneity on the method, with each representing a situation of having multiple distinct CPMs and 1 set of IPD. When the sample size of the IPD was small, stacked regression and the hybrid method had comparable but highest performance across modelling methods. Conversely, in large IPD samples, development of a new model and the hybrid method gave the highest performance. Hence, the proposed strategy can inform the choice between utilising existing CPMs or developing a model de novo, thereby incorporating IPD, existing research, and prior (clinical) knowledge into the modelling strategy.

Using marginal structural models to adjust for treatment drop-in when developing clinical prediction models.

Clinical prediction models (CPMs) can inform decision making about treatment initiation, which requires predicted risks assuming no treatment is given. However, this is challenging since CPMs are usually derived using data sets where patients received treatment, often initiated postbaseline as "treatment drop-ins." This study proposes the use of marginal structural models (MSMs) to adjust for treatment drop-in. We illustrate the use of MSMs in the CPM framework through simulation studies that represent randomized controlled trials and real-world observational data and the example of statin initiation for cardiovascular disease prevention. The simulations include a binary treatment and a covariate, each recorded at two timepoints and having a prognostic effect on a binary outcome. The bias in predicted risk was examined in a model ignoring treatment, a model fitted on treatment-naïve patients (at baseline), a model including baseline treatment, and the MSM. In all simulation scenarios, all models except the MSM underestimated the risk of outcome given absence of treatment. These results were supported in the statin initiation example, which showed that ignoring statin initiation postbaseline resulted in models that significantly underestimated the risk of a cardiovascular disease event occurring within 10 years. Consequently, CPMs that do not acknowledge treatment drop-in can lead to underallocation of treatment. In conclusion, when developing CPMs to predict treatment-naïve risk, researchers should consider using MSMs to adjust for treatment drop-in, and also seek to exploit the ability of MSMs to allow estimation of individual treatment effects.

Con: Most clinical risk scores are useless.

While developing prediction models has become quite popular both in nephrology and in medicine in general, most models have not been implemented in clinical practice on a larger scale. This should be no surprise, as the majority of published models has been shown to be poorly reported and often developed using inappropriate methods. The main problems identified relate to either using too few candidate predictors (based on univariable P < 0.05) or too many (for the number of events), resulting in poorly performing prediction models. Guidelines on how to develop and test a prediction model all stress the importance of external validation to test discrimination and calibration in other populations, as prediction models usually perform less well in new subjects. However, external validity has not often been tested for prediction models in renal patients. Moreover, impact studies showing improved clinical outcomes when using a prediction model in routine clinical practice have been reported rarely. By and large, notwithstanding a few notable exceptions like the kidney failure risk equation prediction model, most models have not been validated externally or are at best inadequately reported, preventing them from be used in clinical practice. Therefore, we recommend researchers to spend more energy on validation and assessing the impact of existing models, instead of merely developing more models that will most likely never be used in clinical practice as well.

Clinical prediction in defined populations: a simulation study investigating when and how to aggregate existing models.

BACKGROUND: Clinical prediction models (CPMs) are increasingly deployed to support healthcare decisions but they are derived inconsistently, in part due to limited data. An emerging alternative is to aggregate existing CPMs developed for similar settings and outcomes. This simulation study aimed to investigate the impact of between-population-heterogeneity and sample size on aggregating existing CPMs in a defined population, compared with developing a model de novo. METHODS: Simulations were designed to mimic a scenario in which multiple CPMs for a binary outcome had been derived in distinct, heterogeneous populations, with potentially different predictors available in each. We then generated a new 'local' population and compared the performance of CPMs developed for this population by aggregation, using stacked regression, principal component analysis or partial least squares, with redevelopment from scratch using backwards selection and penalised regression. RESULTS: While redevelopment approaches resulted in models that were miscalibrated for local datasets of less than 500 observations, model aggregation methods were well calibrated across all simulation scenarios. When the size of local data was less than 1000 observations and between-population-heterogeneity was small, aggregating existing CPMs gave better discrimination and had the lowest mean square error in the predicted risks compared with deriving a new model. Conversely, given greater than 1000 observations and significant between-population-heterogeneity, then redevelopment outperformed the aggregation approaches. In all other scenarios, both aggregation and de novo derivation resulted in similar predictive performance. CONCLUSION: This study demonstrates a pragmatic approach to contextualising CPMs to defined populations. When aiming to develop models in defined populations, modellers should consider existing CPMs, with aggregation approaches being a suitable modelling strategy particularly with sparse data on the local population.

An external validation of models to predict the onset of chronic kidney disease using population-based electronic health records from Salford, UK.

BACKGROUND: Chronic kidney disease (CKD) is a major and increasing constituent of disease burdens worldwide. Early identification of patients at increased risk of developing CKD can guide interventions to slow disease progression, initiate timely referral to appropriate kidney care services, and support targeting of care resources. Risk prediction models can extend laboratory-based CKD screening to earlier stages of disease; however, to date, only a few of them have been externally validated or directly compared outside development populations. Our objective was to validate published CKD prediction models applicable in primary care. METHODS: We synthesised two recent systematic reviews of CKD risk prediction models and externally validated selected models for a 5-year horizon of disease onset. We used linked, anonymised, structured (coded) primary and secondary care data from patients resident in Salford (population ~234 k), UK. All adult patients with at least one record in 2009 were followed-up until the end of 2014, death, or CKD onset (n = 178,399). CKD onset was defined as repeated impaired eGFR measures over a period of at least 3 months, or physician diagnosis of CKD Stage 3-5. For each model, we assessed discrimination, calibration, and decision curve analysis. RESULTS: Seven relevant CKD risk prediction models were identified. Five models also had an associated simplified scoring system. All models discriminated well between patients developing CKD or not, with c-statistics around 0.90. Most of the models were poorly calibrated to our population, substantially over-predicting risk. The two models that did not require recalibration were also the ones that had the best performance in the decision curve analysis. CONCLUSIONS: Included CKD prediction models showed good discriminative ability but over-predicted the actual 5-year CKD risk in English primary care patients. QKidney, the only UK-developed model, outperformed the others. Clinical prediction models should be (re)calibrated for their intended uses.