Clinical impact of the Kidney Failure Risk Equation for vascular access planning.

BACKGROUND: Risk-based thresholds for arteriovenous (AV) access creation has been proposed to aid vascular access planning. We aimed to assess the clinical impact of implementing the kidney failure risk equation (KFRE) for vascular access referral. METHODS: 16,102 nephrology-referred chronic kidney disease (CKD) patients from the Swedish Renal Registry 2008-2018 were included. The KFRE was calculated repeatedly, and the timing was identified for when the KFRE risk exceeded several pre-defined thresholds and/or the estimated glomerular filtration rate <15 ml/min/1.73m2 (eGFR15). To assess the utility of the KFRE/eGFR thresholds, cumulative incidence curves of kidney replacement therapy (KRT) or death, and decision-curve analyses were computed at 6, 12 months, and 2 years. The potential impact of using the different thresholds was illustrated by an example from the Swedish access registry. RESULTS: The 12-month specificity for KRT initiation was highest for KFRE>50% 94.5 (95% Confidence interval [CI] 94.3-94.7), followed by KFRE>40% 90.0 (95% CI 89.7-90.3), while sensitivity was highest for KFRE>30% 79.3 (95% CI 78.2-80.3) and eGFR<15 ml/min/1.73m2 81.2 (95% CI 80.2-82.2). The 2-year positive predictive value was 71.5 (95% CI 70.2-72.8), 61.7 (95% CI 60.4-63.0) and 47.2 (95% CI 46.1-48.3) for KFRE>50%, KFRE>40%, and eGFR<15 respectively. Decision curve analyses suggested the largest net benefit for KFRE>40% over two years and KFRE>50% over 12 months when it is important to avoid the harm of possibly unnecessary surgery. In Sweden, 54% of nephrology-referred patients started hemodialysis in a central venous catheter (CVC) of which only 5% had AV access surgery >6 months before initiation. 60% of the CVC patients exceeded KFRE>40% a median of 0.8 years (interquartile range 0.4-1.5) before KRT initiation. CONCLUSIONS: The utility of using KFRE>40% and KFRE>50% is higher compared to the more traditionally used eGFR threshold <15 ml/min/1.73m2 for vascular access planning.

Prognostic Models in Nephrology: Where Do We Stand and Where Do We Go from Here? Mapping Out the Evidence in a Scoping Review.

Prognostic models can strongly support individualized care provision and well-informed shared decision making. There has been an upsurge of prognostic research in the field of nephrology, but the uptake of prognostic models in clinical practice remains limited. Therefore, we map out the research field of prognostic models for kidney patients and provide directions on how to proceed from here. We performed a scoping review of studies developing, validating, or updating a prognostic model for patients with CKD. We searched all published models in PubMed and Embase and report predicted outcomes, methodological quality, and validation and/or updating efforts. We found 602 studies, of which 30.1% concerned CKD populations, 31.6% dialysis populations, and 38.4% kidney transplantation populations. The most frequently predicted outcomes were mortality ( n =129), kidney disease progression ( n =75), and kidney graft survival ( n =54). Most studies provided discrimination measures (80.4%), but much less showed calibration results (43.4%). Of the 415 development studies, 28.0% did not perform any validation and 57.6% performed only internal validation. Moreover, only 111 models (26.7%) were externally validated either in the development study itself or in an independent external validation study. Finally, in 45.8% of development studies no useable version of the model was reported. To conclude, many prognostic models have been developed for patients with CKD, mainly for outcomes related to kidney disease progression and patient/graft survival. To bridge the gap between prediction research and kidney patient care, patient-reported outcomes, methodological rigor, complete reporting of prognostic models, external validation, updating, and impact assessment urgently need more attention.

Considerable Variability Among Transplant Nephrologists in Judging Deceased Donor Kidney Offers.

Introduction: Transplant clinicians may disagree on whether or not to accept a deceased donor kidney offer. We investigated the interobserver variability between transplant nephrologists regarding organ acceptance and whether the use of a prediction model impacted their decisions. Methods: We developed an observational online survey with 6 real-life cases of deceased donor kidneys offered to a waitlisted recipient. Per case, nephrologists were asked to estimate the risk of adverse outcome and whether they would accept the offer for this patient, or for a patient of their own choice, and how certain they felt. These questions were repeated after revealing the risk of adverse outcome, calculated by a validated prediction model. Results: Sixty Dutch nephrologists completed the survey. The intraclass correlation coefficient of their estimated risk of adverse outcome was poor (0.20, 95% confidence interval [CI] 0.08-0.62). Interobserver agreement of the decision on whether or not to accept the kidney offer was also poor (Fleiss kappa 0.13, 95% CI 0.129-0.130). The acceptance rate before and after providing the outcome of the prediction model was significantly influenced in 2 of 6 cases. Acceptance rates varied considerably among transplant centers. Conclusion: In this study, the estimated risk of adverse outcome and subsequent decision to accept a suboptimal donor kidney varied greatly among transplant nephrologists. The use of a prediction model could influence this decision and may enhance nephrologists' certainty about their decision.

Removing race from the CKD-EPI equation and its impact on prognosis in a predominantly White European population.

BACKGROUND: While American nephrology societies recommend using the 2021 Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) estimated glomerular filtration rate (eGFR) equation without a Black race coefficient, it is unknown how this would impact disease distribution, prognosis and kidney failure risk prediction in predominantly White non-US populations. METHODS: We studied 1.6 million Stockholm adults with serum/plasma creatinine measurements between 2007 and 2019. We calculated changes in eGFR and reclassification across KDIGO GFR categories when changing from the 2009 to 2021 CKD-EPI equation; estimated associations between eGFR and the clinical outcomes kidney failure with replacement therapy (KFRT), (cardiovascular) mortality and major adverse cardiovascular events using Cox regression; and investigated prognostic accuracy (discrimination and calibration) of both equations within the Kidney Failure Risk Equation. RESULTS: Compared with the 2009 equation, the 2021 equation yielded a higher eGFR by a median [interquartile range (IQR)] of 3.9 (2.9-4.8) mL/min/1.73 m2, which was larger at older age and for men. Consequently, 9.9% of the total population and 36.2% of the population with CKD G3a-G5 was reclassified to a higher eGFR category. Reclassified individuals exhibited a lower risk of KFRT, but higher risks of all-cause/cardiovascular death and major adverse cardiovascular events, compared with non-reclassified participants of similar eGFR. eGFR by both equations strongly predicted study outcomes, with equal discrimination and calibration for the Kidney Failure Risk Equation. CONCLUSIONS: Implementing the 2021 CKD-EPI equation in predominantly White European populations would raise eGFR by a modest amount (larger at older age and in men) and shift a major proportion of CKD patients to a higher eGFR category. eGFR by both equations strongly predicted outcomes.

Predicting Kidney Failure, Cardiovascular Disease and Death in Advanced CKD Patients.

Introduction: Predicting the timing and occurrence of kidney replacement therapy (KRT), cardiovascular events, and death among patients with advanced chronic kidney disease (CKD) is clinically useful and relevant. We aimed to externally validate a recently developed CKD G4+ risk calculator for these outcomes and to assess its potential clinical impact in guiding vascular access placement. Methods: We included 1517 patients from the European Quality (EQUAL) study, a European multicentre prospective cohort study of nephrology-referred advanced CKD patients aged ≥65 years. Model performance was assessed based on discrimination and calibration. Potential clinical utility for timing of referral for vascular access placement was studied with diagnostic measures and decision curve analysis (DCA). Results: The model showed a good discrimination for KRT and "death after KRT," with 2-year concordance (C) statistics of 0.74 and 0.76, respectively. Discrimination for cardiovascular events (2-year C-statistic: 0.70) and overall death (2-year C-statistic: 0.61) was poorer. Calibration was fairly accurate. Decision curves illustrated that using the model to guide vascular access referral would generally lead to less unused arteriovenous fistulas (AVFs) than following estimated glomerular filtration rate (eGFR) thresholds. Conclusion: This study shows moderate to good predictive performance of the model in an older cohort of nephrology-referred patients with advanced CKD. Using the model to guide referral for vascular access placement has potential in combating unnecessary vascular surgeries.

External validation of risk scores to predict in-hospital mortality in patients hospitalized due to coronavirus disease 2019.

BACKGROUND: The coronavirus disease 2019 (COVID-19) presents an urgent threat to global health. Prediction models that accurately estimate mortality risk in hospitalized patients could assist medical staff in treatment and allocating limited resources. AIMS: To externally validate two promising previously published risk scores that predict in-hospital mortality among hospitalized COVID-19 patients. METHODS: Two prospective cohorts were available; a cohort of 1028 patients admitted to one of nine hospitals in Lombardy, Italy (the Lombardy cohort) and a cohort of 432 patients admitted to a hospital in Leiden, the Netherlands (the Leiden cohort). The endpoint was in-hospital mortality. All patients were adult and tested COVID-19 PCR-positive. Model discrimination and calibration were assessed. RESULTS: The C-statistic of the 4C mortality score was good in the Lombardy cohort (0.85, 95CI: 0.82-0.89) and in the Leiden cohort (0.87, 95CI: 0.80-0.94). Model calibration was acceptable in the Lombardy cohort but poor in the Leiden cohort due to the model systematically overpredicting the mortality risk for all patients. The C-statistic of the CURB-65 score was good in the Lombardy cohort (0.80, 95CI: 0.75-0.85) and in the Leiden cohort (0.82, 95CI: 0.76-0.88). The mortality rate in the CURB-65 development cohort was much lower than the mortality rate in the Lombardy cohort. A similar but less pronounced trend was found for patients in the Leiden cohort. CONCLUSION: Although performances did not differ greatly, the 4C mortality score showed the best performance. However, because of quickly changing circumstances, model recalibration may be necessary before using the 4C mortality score.

Optimising Access Surgery in Senior Haemodialysis Patients (OASIS): study protocol for a multicentre randomised controlled trial.

INTRODUCTION: Current evidence on vascular access strategies for haemodialysis patients is based on observational studies that are at high risk of selection bias. For elderly patients, autologous arteriovenous fistulas that are typically created in usual care may not be the best option because a significant proportion of fistulas either fail to mature or remain unused. In addition, long-term complications associated with arteriovenous grafts and central venous catheters may be less relevant when considering the limited life expectancy of these patients. Therefore, we designed the Optimising Access Surgery in Senior Haemodialysis Patients (OASIS) trial to determine the best strategy for vascular access creation in elderly haemodialysis patients. METHODS AND ANALYSIS: OASIS is a multicentre randomised controlled trial with an equal participant allocation in three treatment arms. Patients aged 70 years or older who are expected to initiate haemodialysis treatment in the next 6 months or who have started haemodialysis urgently with a catheter will be enrolled. To detect and exclude patients with an unusually long life expectancy, we will use a previously published mortality prediction model after external validation. Participants allocated to the usual care arm will be treated according to current guidelines on vascular access creation and will undergo fistula creation. Participants allocated to one of the two intervention arms will undergo graft placement or catheter insertion. The primary outcome is the number of access-related interventions required for each patient-year of haemodialysis treatment. We will enrol 195 patients to have sufficient statistical power to detect an absolute decrease of 0.80 interventions per year. ETHICS AND DISSEMINATION: Because of clinical equipoise, we believe it is justified to randomly allocate elderly patients to the different vascular access strategies. The study was approved by an accredited medical ethics review committee. The results will be disseminated through peer-reviewed publications and will be implemented in clinical practice guidelines. TRIAL REGISTRATION NUMBER: NL7933. PROTOCOL VERSION AND DATE: V.5, 25 February 2021.

Lessons learnt when accounting for competing events in the external validation of time-to-event prognostic models.

BACKGROUND: External validation of prognostic models is necessary to assess the accuracy and generalizability of the model to new patients. If models are validated in a setting in which competing events occur, these competing risks should be accounted for when comparing predicted risks to observed outcomes. METHODS: We discuss existing measures of calibration and discrimination that incorporate competing events for time-to-event models. These methods are illustrated using a clinical-data example concerning the prediction of kidney failure in a population with advanced chronic kidney disease (CKD), using the guideline-recommended Kidney Failure Risk Equation (KFRE). The KFRE was developed using Cox regression in a diverse population of CKD patients and has been proposed for use in patients with advanced CKD in whom death is a frequent competing event. RESULTS: When validating the 5-year KFRE with methods that account for competing events, it becomes apparent that the 5-year KFRE considerably overestimates the real-world risk of kidney failure. The absolute overestimation was 10%age points on average and 29%age points in older high-risk patients. CONCLUSIONS: It is crucial that competing events are accounted for during external validation to provide a more reliable assessment the performance of a model in clinical settings in which competing risks occur.

Prediction or causality? A scoping review of their conflation within current observational research.

Etiological research aims to uncover causal effects, whilst prediction research aims to forecast an outcome with the best accuracy. Causal and prediction research usually require different methods, and yet their findings may get conflated when reported and interpreted. The aim of the current study is to quantify the frequency of conflation between etiological and prediction research, to discuss common underlying mistakes and provide recommendations on how to avoid these. Observational cohort studies published in January 2018 in the top-ranked journals of six distinct medical fields (Cardiology, Clinical Epidemiology, Clinical Neurology, General and Internal Medicine, Nephrology and Surgery) were included for the current scoping review. Data on conflation was extracted through signaling questions. In total, 180 studies were included. Overall, 26% (n = 46) contained conflation between etiology and prediction. The frequency of conflation varied across medical field and journal impact factor. From the causal studies 22% was conflated, mainly due to the selection of covariates based on their ability to predict without taking the causal structure into account. Within prediction studies 38% was conflated, the most frequent reason was a causal interpretation of covariates included in a prediction model. Conflation of etiology and prediction is a common methodological error in observational medical research and more frequent in prediction studies. As this may lead to biased estimations and erroneous conclusions, researchers must be careful when designing, interpreting and disseminating their research to ensure this conflation is avoided.

Appraising prediction research: a guide and meta-review on bias and applicability assessment using the Prediction model Risk Of Bias ASsessment Tool (PROBAST).

Over the past few years, a large number of prediction models have been published, often of poor methodological quality. Seemingly objective and straightforward, prediction models provide a risk estimate for the outcome of interest, usually based on readily available clinical information. Yet, using models of substandard methodological rigour, especially without external validation, may result in incorrect risk estimates and consequently misclassification. To assess and combat bias in prediction research the prediction model risk of bias assessment tool (PROBAST) was published in 2019. This risk of bias (ROB) tool includes four domains and 20 signalling questions highlighting methodological flaws, and provides guidance in assessing the applicability of the model. In this paper, the PROBAST will be discussed, along with an in-depth review of two commonly encountered pitfalls in prediction modelling that may induce bias: overfitting and composite endpoints. We illustrate the prevalence of potential bias in prediction models with a meta-review of 50 systematic reviews that used the PROBAST to appraise their included studies, thus including 1510 different studies on 2104 prediction models. All domains showed an unclear or high ROB; these results were markedly stable over time, highlighting the urgent need for attention on bias in prediction research. This article aims to do just that by providing (1) the clinician with tools to evaluate the (methodological) quality of a clinical prediction model, (2) the researcher working on a review with methods to appraise the included models, and (3) the researcher developing a model with suggestions to improve model quality.

Kidney Failure Prediction Models: A Comprehensive External Validation Study in Patients with Advanced CKD.

BACKGROUND: Various prediction models have been developed to predict the risk of kidney failure in patients with CKD. However, guideline-recommended models have yet to be compared head to head, their validation in patients with advanced CKD is lacking, and most do not account for competing risks. METHODS: To externally validate 11 existing models of kidney failure, taking the competing risk of death into account, we included patients with advanced CKD from two large cohorts: the European Quality Study (EQUAL), an ongoing European prospective, multicenter cohort study of older patients with advanced CKD, and the Swedish Renal Registry (SRR), an ongoing registry of nephrology-referred patients with CKD in Sweden. The outcome of the models was kidney failure (defined as RRT-treated ESKD). We assessed model performance with discrimination and calibration. RESULTS: The study included 1580 patients from EQUAL and 13,489 patients from SRR. The average c statistic over the 11 validated models was 0.74 in EQUAL and 0.80 in SRR, compared with 0.89 in previous validations. Most models with longer prediction horizons overestimated the risk of kidney failure considerably. The 5-year Kidney Failure Risk Equation (KFRE) overpredicted risk by 10%-18%. The four- and eight-variable 2-year KFRE and the 4-year Grams model showed excellent calibration and good discrimination in both cohorts. CONCLUSIONS: Some existing models can accurately predict kidney failure in patients with advanced CKD. KFRE performed well for a shorter time frame (2 years), despite not accounting for competing events. Models predicting over a longer time frame (5 years) overestimated risk because of the competing risk of death. The Grams model, which accounts for the latter, is suitable for longer-term predictions (4 years).

External validation of prognostic models: what, why, how, when and where?

Prognostic models that aim to improve the prediction of clinical events, individualized treatment and decision-making are increasingly being developed and published. However, relatively few models are externally validated and validation by independent researchers is rare. External validation is necessary to determine a prediction model's reproducibility and generalizability to new and different patients. Various methodological considerations are important when assessing or designing an external validation study. In this article, an overview is provided of these considerations, starting with what external validation is, what types of external validation can be distinguished and why such studies are a crucial step towards the clinical implementation of accurate prediction models. Statistical analyses and interpretation of external validation results are reviewed in an intuitive manner and considerations for selecting an appropriate existing prediction model and external validation population are discussed. This study enables clinicians and researchers to gain a deeper understanding of how to interpret model validation results and how to translate these results to their own patient population.

Predicting mortality risk on dialysis and conservative care: development and internal validation of a prediction tool for older patients with advanced chronic kidney disease.

BACKGROUND: Conservative care (CC) may be a valid alternative to dialysis for certain older patients with advanced chronic kidney disease (CKD). A model that predicts patient prognosis on both treatment pathways could be of value in shared decision-making. Therefore, the aim is to develop a prediction tool that predicts the mortality risk for the same patient for both dialysis and CC from the time of treatment decision. METHODS: CKD Stage 4/5 patients aged ≥70 years, treated at a single centre in the Netherlands, were included between 2004 and 2016. Predictors were collected at treatment decision and selected based on literature and an expert panel. Outcome was 2-year mortality. Basic and extended logistic regression models were developed for both the dialysis and CC groups. These models were internally validated with bootstrapping. Model performance was assessed with discrimination and calibration. RESULTS: In total, 366 patients were included, of which 126 chose CC. Pre-selected predictors for the basic model were age, estimated glomerular filtration rate, malignancy and cardiovascular disease. Discrimination was moderate, with optimism-corrected C-statistics ranging from 0.675 to 0.750. Calibration plots showed good calibration. CONCLUSIONS: A prediction tool that predicts 2-year mortality was developed to provide older advanced CKD patients with individualized prognosis estimates for both dialysis and CC. Future studies are needed to test whether our findings hold in other CKD populations. Following external validation, this prediction tool could be used to compare a patient's prognosis on both dialysis and CC, and help to inform treatment decision-making.

The Association Between Pain Perception and Care Dependency in Older Nursing Home Residents: A Prospective Cohort Study.

OBJECTIVES: Maintenance of independence is a challenge for nursing home residents whose pain is often substantial. The objective of this study was to explore the relationship between pain perception and care dependency in a population of Dutch nursing home residents. DESIGN: Prospective cohort study. SETTING AND PARTICIPANTS: Dutch nursing home residents aged 65 or older, excluding residents with a severe cognitive impairment. METHODS: The Numeric Rating Scale (NRS) was used to rate pain perception from 0 to 10 in half-point increments and the Care Dependency Scale (CDS) to measure care dependency, with scores ranging from 15 (completely care dependent) to 75 (fully independent). Both measurements were repeated after a 2-month follow-up. Multiple linear regression analysis was used to adjust for potential confounders. Missing data were dealt with by performing tenfold multiple imputation. RESULTS: A total of 1256 residents (65% women, mean age 83 years) were included. At baseline, the median NRS pain score was 3.0 (interquartile range 0.0-6.0) and the mean CDS score was 55.9 (SD 11.5). Cross-sectionally, for 1-point increase in pain score, care dependency increased 0.65 points [95% confidence interval (CI) 0.46-0.83]. More pain at baseline was associated with slightly lower care dependency after 2 months (beta 0.20, 95% CI 0.01-0.39). Compared with residents whose pain decreased over 2 months, residents with stable pain or increased pain had a 2.27-point (95% CI 0.83-3.70) and 2.39-point (95% CI 0.87-3.90) greater increase in care dependency, respectively. CONCLUSIONS AND IMPLICATIONS: Pain perception and care dependency are associated in a population of older nursing home residents, and stable or increased pain is associated with increased care dependency progression. The findings of this study emphasize that pain and care dependency should not be assessed nor treated independently.

Renal function decline in older men and women with advanced chronic kidney disease-results from the EQUAL study.

INTRODUCTION: Understanding the mechanisms underlying the differences in renal decline between men and women may improve sex-specific clinical monitoring and management. To this end, we aimed to compare the slope of renal function decline in older men and women in chronic kidney disease (CKD) Stages 4 and 5, taking into account informative censoring related to the sex-specific risks of mortality and dialysis initiation. METHODS: The European QUALity Study on treatment in advanced CKD (EQUAL) study is an observational prospective cohort study in Stages 4 and 5 CKD patients ≥65 years not on dialysis. Data on clinical and demographic patient characteristics were collected between April 2012 and December 2018. Estimated glomerular filtration rate (eGFR) was calculated using the CKD Epidemiology Collaboration equation. eGFR trajectory by sex was modelled using linear mixed models, and joint models were applied to deal with informative censoring. RESULTS: We included 7801 eGFR measurements in 1682 patients over a total of 2911 years of follow-up. Renal function declined by 14.0% [95% confidence interval (CI) 12.9-15.1%] on average each year. Renal function declined faster in men (16.2%/year, 95% CI 15.9-17.1%) compared with women (9.6%/year, 95% CI 6.3-12.1%), which remained largely unchanged after accounting for various mediators and for informative censoring due to mortality and dialysis initiation. Diabetes was identified as an important determinant of renal decline specifically in women. CONCLUSION: In conclusion, renal function declines faster in men compared with women, which remained similar after adjustment for mediators and despite a higher risk of informative censoring in men. We demonstrate a disproportional negative impact of diabetes specifically in women.

Development and external validation study combining existing models and recent data into an up-to-date prediction model for evaluating kidneys from older deceased donors for transplantation.

With a rising demand for kidney transplantation, reliable pre-transplant assessment of organ quality becomes top priority. In clinical practice, physicians are regularly in doubt whether suboptimal kidney offers from older donors should be accepted. Here, we externally validate existing prediction models in a European population of older deceased donors, and subsequently developed and externally validated an adverse outcome prediction tool. Recipients of kidney grafts from deceased donors 50 years of age and older were included from the Netherlands Organ Transplant Registry (NOTR) and United States organ transplant registry from 2006-2018. The predicted adverse outcome was a composite of graft failure, death or chronic kidney disease stage 4 plus within one year after transplantation, modelled using logistic regression. Discrimination and calibration were assessed in internal, temporal and external validation. Seven existing models were validated with the same cohorts. The NOTR development cohort contained 2510 patients and 823 events. The temporal validation within NOTR had 837 patients and the external validation used 31987 patients in the United States organ transplant registry. Discrimination of our full adverse outcome model was moderate in external validation (C-statistic 0.63), though somewhat better than discrimination of the seven existing prediction models (average C-statistic 0.57). The model's calibration was highly accurate. Thus, since existing adverse outcome kidney graft survival models performed poorly in a population of older deceased donors, novel models were developed and externally validated, with maximum achievable performance in a population of older deceased kidney donors. These models could assist transplant clinicians in deciding whether to accept a kidney from an older donor.

TRIPOD statement: a preliminary pre-post analysis of reporting and methods of prediction models.

OBJECTIVES: To assess the difference in completeness of reporting and methodological conduct of published prediction models before and after publication of the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement. METHODS: In the seven general medicine journals with the highest impact factor, we compared the completeness of the reporting and the quality of the methodology of prediction model studies published between 2012 and 2014 (pre-TRIPOD) with studies published between 2016 and 2017 (post-TRIPOD). For articles published in the post-TRIPOD period, we examined whether there was improved reporting for articles (1) citing the TRIPOD statement, and (2) published in journals that published the TRIPOD statement. RESULTS: A total of 70 articles was included (pre-TRIPOD: 32, post-TRIPOD: 38). No improvement was seen for the overall percentage of reported items after the publication of the TRIPOD statement (pre-TRIPOD 74%, post-TRIPOD 76%, 95% CI of absolute difference: -4% to 7%). For the individual TRIPOD items, an improvement was seen for 16 (44%) items, while 3 (8%) items showed no improvement and 17 (47%) items showed a deterioration. Post-TRIPOD, there was no improved reporting for articles citing the TRIPOD statement, nor for articles published in journals that published the TRIPOD statement. The methodological quality improved in the post-TRIPOD period. More models were externally validated in the same article (absolute difference 8%, post-TRIPOD: 39%), used measures of calibration (21%, post-TRIPOD: 87%) and discrimination (9%, post-TRIPOD: 100%), and used multiple imputation for handling missing data (12%, post-TRIPOD: 50%). CONCLUSIONS: Since the publication of the TRIPOD statement, some reporting and methodological aspects have improved. Prediction models are still often poorly developed and validated and many aspects remain poorly reported, hindering optimal clinical application of these models. Long-term effects of the TRIPOD statement publication should be evaluated in future studies.