How to develop, validate, and update clinical prediction models using multinomial logistic regression.

OBJECTIVES: Multicategory prediction models (MPMs) can be used in health care when the primary outcome of interest has more than two categories. The application of MPMs is scarce, possibly due to added methodological complexities compared to binary outcome models. We provide a guide of how to develop, validate, and update clinical prediction models based on multinomial logistic regression. STUDY DESIGN AND SETTING: We present guidance and recommendations based on recent methodological literature, illustrated by a previously developed and validated MPM for treatment outcomes in rheumatoid arthritis. Prediction models using multinomial logistic regression can be developed for nominal outcomes, but also for ordinal outcomes. This article is intended to supplement existing general guidance on prediction model research. RESULTS: This guide is split into three parts: 1) outcome definition and variable selection, 2) model development, and 3) model evaluation (including performance assessment, internal and external validation, and model recalibration). We outline how to evaluate and interpret the predictive performance of MPMs. R code is provided. CONCLUSION: We recommend the application of MPMs in clinical settings where the prediction of a multicategory outcome is of interest. Future methodological research could focus on MPM-specific considerations for variable selection and sample size criteria for external validation.

Prognostic factors associated with failure of total elbow arthroplasty.

Aims: The aims of this study were to identify and evaluate the current literature examining the prognostic factors which are associated with failure of total elbow arthroplasty (TEA). Methods: Electronic literature searches were conducted using MEDLINE, Embase, PubMed, and Cochrane. All studies reporting prognostic estimates for factors associated with the revision of a primary TEA were included. The risk of bias was assessed using the Quality In Prognosis Studies (QUIPS) tool, and the quality of evidence was assessed using the modified Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) framework. Due to low quality of the evidence and the heterogeneous nature of the studies, a narrative synthesis was used. Results: A total of 19 studies met the inclusion criteria, investigating 28 possible prognostic factors. Most QUIPS domains (84%) were rated as moderate to high risk of bias. The quality of the evidence was low or very low for all prognostic factors. In low-quality evidence, prognostic factors with consistent associations with failure of TEA in more than one study were: the sequelae of trauma leading to TEA, either independently or combined with acute trauma, and male sex. Several other studies investigating sex reported no association. The evidence for other factors was of very low quality and mostly involved exploratory studies. Conclusion: The current evidence investigating the prognostic factors associated with failure of TEA is of low or very low quality, and studies generally have a moderate to high risk of bias. Prognostic factors are subject to uncertainty, should be interpreted with caution, and are of little clinical value. Higher-quality evidence is required to determine robust prognostic factors for failure of TEA.

Developing and externally validating multinomial prediction models for methotrexate treatment outcomes in patients with rheumatoid arthritis: results from an international collaboration.

OBJECTIVES: In rheumatology, there is a clinical need to identify patients at high risk (>50%) of not responding to the first-line therapy methotrexate (MTX) due to lack of disease control or discontinuation due to adverse events (AEs). Despite this need, previous prediction models in this context are at high risk of bias and ignore AEs. Our objectives were to (i) develop a multinomial model for outcomes of low disease activity and discontinuing due to AEs 6 months after starting MTX, (ii) update prognosis 3-month following treatment initiation, and (iii) externally validate these models. STUDY DESIGN AND SETTING: A multinomial model for low disease activity (submodel 1) and discontinuing due to AEs (submodel 2) was developed using data from the UK Rheumatoid Arthritis Medication Study, updated using landmarking analysis, internally validated using bootstrapping, and externally validated in the Norwegian Disease-Modifying Antirheumatic Drug register. Performance was assessed using calibration (calibration-slope and calibration-in-the-large), and discrimination (concordance-statistic and polytomous discriminatory index). RESULTS: The internally validated model showed good calibration in the development setting with a calibration-slope of 1.01 (0.87, 1.14) (submodel 1) and 0.83 (0.30, 1.34) (submodel 2), and moderate discrimination with a c-statistic of 0.72 (0.69, 0.74) and 0.53 (0.48, 0.59), respectively. Predictive performance decreased after external validation (calibration-slope 0.78 (0.64, 0.93) (submodel 1) and 0.86 (0.34, 1.38) (submodel 2)), which may be due to differences in disease-specific characteristics and outcome prevalence. CONCLUSION: We addressed previously identified methodological limitations of prediction models for outcomes of MTX therapy. The multinomial approach predicted outcomes of disease activity more accurately than AEs, which should be addressed in future work to aid implementation into clinical practice.

Prognostic factors associated with failure of total elbow replacement: a protocol for a systematic review.

INTRODUCTION: Total elbow replacement (TER) has higher failure rates requiring revision surgery compared with the replacement of other joints. Understanding the factors associated with failure is essential for informed decision-making between patients and clinicians, and for reducing the failure rate. This review aims to identify, describe and appraise the literature examining prognostic factors for failure of TER. METHODS AND ANALYSIS: This systematic review will be conducted and reported in line with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guidelines. Electronic literature searches will be conducted using Medline, EMBASE, PubMed and Cochrane. The search strategy will be broad, including a combination of subject headings (MESH) and free text search. This search will be supplemented with a screening of reference lists of the included studies and relevant reviews. Two independent reviewers will screen all search results in two stages (title and abstract, and full text) based on the Population, Index prognostic factor, Comparator prognostic factor, Outcome, Time and Setting criteria. The types of evidence included will be randomised trials, non-randomised trials, prospective and retrospective cohort studies, registry studies and case-control studies. If the literature lacks enough studies, then case series with 50 or more TERs will be considered for inclusion. Data extraction and risk of bias assessment for included studies will be performed by two independent reviewers using the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies for Prognostic Factors and Quality In Prognostic Studies tools.Meta-analyses of prognostic estimates for each factor will be undertaken for studies that are deemed to be sufficiently robust and comparable. Several challenges are likely to arise due to heterogeneity between studies, therefore, subgroup and sensitivity analyses will be performed to account for the differences between studies. Heterogeneity will be assessed using Q and I2 statistics. If I2>40% then pooled estimates will not be reported. When quantitative synthesis is not possible, a narrative synthesis will be undertaken. The quality of the evidence for each prognostic factor will be assessed using the Grades of Recommendation Assessment, Development and Evaluation tool. PROSPERO REGISTRATION NUMBER: CRD42023384756.

Clinical prediction models for methotrexate treatment outcomes in patients with rheumatoid arthritis: A systematic review and meta-analysis.

BACKGROUND: In the management of rheumatoid arthritis (RA), there is a clinical need to identify which patients are at high-risk of not responding to methotrexate (MTX), or experiencing adverse events (AEs), to enable earlier alternative treatments. Many clinical prediction models (CPMs) have previously been developed, but a summary of such models and their methodological quality is lacking. This systematic review aimed to (i) identify and summarize previously published CPMs of MTX outcomes in biologic-naïve RA patients, and (ii) critically appraise their methodological properties. METHODS: Medline and Embase were searched to identify studies developing or validating CPMs of MTX outcomes in RA patients. The risk of bias (ROB) was assessed using PROBAST (prediction model risk of bias assessment tool). A fixed effects meta-analysis summarised discrimination for models with multiple external validations. RESULTS: The systematic review identified 20 CPMs across 13 studies, and 4 validation studies. Three outcome types were used: a state of disease activity (n = 14, 70%); EULAR response criteria (n = 4, 20%); or discontinuation due to AEs (n = 2, 10%). Only one model accounted for potential competing risks, and nine (45%) were internally validated. Eight (40%) models used multiple imputation for missing data, others were often limited to complete case analysis. There was overall high ROB. The meta-analysis summarised c-statistics for two models with multiple external validations was 0.77 (95% CI: 0.69, 0.84) and 0.68 (0.64, 0.71). CONCLUSION: This review highlights several methodological shortcomings that should be addressed in future model development to increase potential for implementation into practice.