Calibration plots for multistate risk predictions models.

INTRODUCTION: There is currently no guidance on how to assess the calibration of multistate models used for risk prediction. We introduce several techniques that can be used to produce calibration plots for the transition probabilities of a multistate model, before assessing their performance in the presence of random and independent censoring through a simulation. METHODS: We studied pseudo-values based on the Aalen-Johansen estimator, binary logistic regression with inverse probability of censoring weights (BLR-IPCW), and multinomial logistic regression with inverse probability of censoring weights (MLR-IPCW). The MLR-IPCW approach results in a calibration scatter plot, providing extra insight about the calibration. We simulated data with varying levels of censoring and evaluated the ability of each method to estimate the calibration curve for a set of predicted transition probabilities. We also developed evaluated the calibration of a model predicting the incidence of cardiovascular disease, type 2 diabetes and chronic kidney disease among a cohort of patients derived from linked primary and secondary healthcare records. RESULTS: The pseudo-value, BLR-IPCW, and MLR-IPCW approaches give unbiased estimates of the calibration curves under random censoring. These methods remained predominately unbiased in the presence of independent censoring, even if the censoring mechanism was strongly associated with the outcome, with bias concentrated in low-density regions of predicted transition probability. CONCLUSIONS: We recommend implementing either the pseudo-value or BLR-IPCW approaches to produce a calibration curve, combined with the MLR-IPCW approach to produce a calibration scatter plot. The methods have been incorporated into the "calibmsm" R package available on CRAN.

GRADE concept paper 8: judging the certainty of discrimination performance estimates of prognostic models in a body of validation studies.

BACKGROUND: Prognostic models incorporate multiple prognostic factors to estimate the likelihood of future events for individual patients based on their prognostic factor values. Evaluating these models crucially involves conducting studies to assess their predictive performance, like discrimination. Systematic reviews and meta-analyses of these validation studies play an essential role in selecting models for clinical practice. METHODS: In this paper, we outline 3 thresholds to determine the target for certainty rating in the discrimination of prognostic models, as observed across a body of validation studies. RESULTS AND CONCLUSION: We propose 3 thresholds when rating the certainty of evidence about a prognostic model's discrimination. The first threshold amounts to rating certainty in the model's ability to classify better than random chance. The other 2 approaches involve setting thresholds informed by other mechanisms for classification: clinician intuition or an alternative prognostic model developed for the same disease area and outcome. The choice of threshold will vary based on the context. Instead of relying on arbitrary discrimination cut-offs, our approach positions the observed discrimination within an informed spectrum, potentially aiding decisions about a prognostic model's practical utility.

SPIN-PM: a consensus framework to evaluate the presence of spin in studies on prediction models.

OBJECTIVES: To develop a framework to identify and evaluate spin practices and its facilitators in studies on clinical prediction model regardless of the modeling technique. STUDY DESIGN AND SETTING: We followed a three-phase consensus process: (1) premeeting literature review to generate items to be included; (2) a series of structured meetings to provide comments discussed and exchanged viewpoints on items to be included with a panel of experienced researchers; and (3) postmeeting review on final list of items and examples to be included. Through this iterative consensus process, a framework was derived after all panel's researchers agreed. RESULTS: This consensus process involved a panel of eight researchers and resulted in SPIN-Prediction Models which consists of two categories of spin (misleading interpretation and misleading transportability), and within these categories, two forms of spin (spin practices and facilitators of spin). We provide criteria and examples. CONCLUSION: We proposed this guidance aiming to facilitate not only the accurate reporting but also an accurate interpretation and extrapolation of clinical prediction models which will likely improve the reporting quality of subsequent research, as well as reduce research waste.

Community based complex interventions to sustain independence in older people: systematic review and network meta-analysis.

OBJECTIVE: To synthesise evidence of the effectiveness of community based complex interventions, grouped according to their intervention components, to sustain independence for older people. DESIGN: Systematic review and network meta-analysis. DATA SOURCES: Medline, Embase, CINAHL, PsycINFO, CENTRAL, clinicaltrials.gov, and International Clinical Trials Registry Platform from inception to 9 August 2021 and reference lists of included studies. ELIGIBILITY CRITERIA: Randomised controlled trials or cluster randomised controlled trials with ≥24 weeks' follow-up studying community based complex interventions for sustaining independence in older people (mean age ≥65 years) living at home, with usual care, placebo, or another complex intervention as comparators. MAIN OUTCOMES: Living at home, activities of daily living (personal/instrumental), care home placement, and service/economic outcomes at 12 months. DATA SYNTHESIS: Interventions were grouped according to a specifically developed typology. Random effects network meta-analysis estimated comparative effects; Cochrane's revised tool (RoB 2) structured risk of bias assessment. Grading of recommendations assessment, development and evaluation (GRADE) network meta-analysis structured certainty assessment. RESULTS: The review included 129 studies (74 946 participants). Nineteen intervention components, including "multifactorial action from individualised care planning" (a process of multidomain assessment and management leading to tailored actions), were identified in 63 combinations. For living at home, compared with no intervention/placebo, evidence favoured multifactorial action from individualised care planning including medication review and regular follow-ups (routine review) (odds ratio 1.22, 95% confidence interval 0.93 to 1.59; moderate certainty); multifactorial action from individualised care planning including medication review without regular follow-ups (2.55, 0.61 to 10.60; low certainty); combined cognitive training, medication review, nutritional support, and exercise (1.93, 0.79 to 4.77; low certainty); and combined activities of daily living training, nutritional support, and exercise (1.79, 0.67 to 4.76; low certainty). Risk screening or the addition of education and self-management strategies to multifactorial action from individualised care planning and routine review with medication review may reduce odds of living at home. For instrumental activities of daily living, evidence favoured multifactorial action from individualised care planning and routine review with medication review (standardised mean difference 0.11, 95% confidence interval 0.00 to 0.21; moderate certainty). Two interventions may reduce instrumental activities of daily living: combined activities of daily living training, aids, and exercise; and combined activities of daily living training, aids, education, exercise, and multifactorial action from individualised care planning and routine review with medication review and self-management strategies. For personal activities of daily living, evidence favoured combined exercise, multifactorial action from individualised care planning, and routine review with medication review and self-management strategies (0.16, -0.51 to 0.82; low certainty). For homecare recipients, evidence favoured addition of multifactorial action from individualised care planning and routine review with medication review (0.60, 0.32 to 0.88; low certainty). High risk of bias and imprecise estimates meant that most evidence was low or very low certainty. Few studies contributed to each comparison, impeding evaluation of inconsistency and frailty. CONCLUSIONS: The intervention most likely to sustain independence is individualised care planning including medicines optimisation and regular follow-up reviews resulting in multifactorial action. Homecare recipients may particularly benefit from this intervention. Unexpectedly, some combinations may reduce independence. Further research is needed to investigate which combinations of interventions work best for different participants and contexts. REGISTRATION: PROSPERO CRD42019162195.

Development and external validation of the eFalls tool: a multivariable prediction model for the risk of ED attendance or hospitalisation with a fall or fracture in older adults.

BACKGROUND: Falls are common in older adults and can devastate personal independence through injury such as fracture and fear of future falls. Methods to identify people for falls prevention interventions are currently limited, with high risks of bias in published prediction models. We have developed and externally validated the eFalls prediction model using routinely collected primary care electronic health records (EHR) to predict risk of emergency department attendance/hospitalisation with fall or fracture within 1 year. METHODS: Data comprised two independent, retrospective cohorts of adults aged ≥65 years: the population of Wales, from the Secure Anonymised Information Linkage Databank (model development); the population of Bradford and Airedale, England, from Connected Bradford (external validation). Predictors included electronic frailty index components, supplemented with variables informed by literature reviews and clinical expertise. Fall/fracture risk was modelled using multivariable logistic regression with a Least Absolute Shrinkage and Selection Operator penalty. Predictive performance was assessed through calibration, discrimination and clinical utility. Apparent, internal-external cross-validation and external validation performance were assessed across general practices and in clinically relevant subgroups. RESULTS: The model's discrimination performance (c-statistic) was 0.72 (95% confidence interval, CI: 0.68 to 0.76) on internal-external cross-validation and 0.82 (95% CI: 0.80 to 0.83) on external validation. Calibration was variable across practices, with some over-prediction in the validation population (calibration-in-the-large, -0.87; 95% CI: -0.96 to -0.78). Clinical utility on external validation was improved after recalibration. CONCLUSION: The eFalls prediction model shows good performance and could support proactive stratification for falls prevention services if appropriately embedded into primary care EHR systems.

Global disease burden of and risk factors for acute lower respiratory infections caused by respiratory syncytial virus in preterm infants and young children in 2019: a systematic review and meta-analysis of aggregated and individual participant data.

BACKGROUND: Infants and young children born prematurely are at high risk of severe acute lower respiratory infection (ALRI) caused by respiratory syncytial virus (RSV). In this study, we aimed to assess the global disease burden of and risk factors for RSV-associated ALRI in infants and young children born before 37 weeks of gestation. METHODS: We conducted a systematic review and meta-analysis of aggregated data from studies published between Jan 1, 1995, and Dec 31, 2021, identified from MEDLINE, Embase, and Global Health, and individual participant data shared by the Respiratory Virus Global Epidemiology Network on respiratory infectious diseases. We estimated RSV-associated ALRI incidence in community, hospital admission, in-hospital mortality, and overall mortality among children younger than 2 years born prematurely. We conducted two-stage random-effects meta-regression analyses accounting for chronological age groups, gestational age bands (early preterm, <32 weeks gestational age [wGA], and late preterm, 32 to <37 wGA), and changes over 5-year intervals from 2000 to 2019. Using individual participant data, we assessed perinatal, sociodemographic, and household factors, and underlying medical conditions for RSV-associated ALRI incidence, hospital admission, and three severity outcome groups (longer hospital stay [>4 days], use of supplemental oxygen and mechanical ventilation, or intensive care unit admission) by estimating pooled odds ratios (ORs) through a two-stage meta-analysis (multivariate logistic regression and random-effects meta-analysis). This study is registered with PROSPERO, CRD42021269742. FINDINGS: We included 47 studies from the literature and 17 studies with individual participant-level data contributed by the participating investigators. We estimated that, in 2019, 1 650 000 (95% uncertainty range [UR] 1 350 000-1 990 000) RSV-associated ALRI episodes, 533 000 (385 000-730 000) RSV-associated hospital admissions, 3050 (1080-8620) RSV-associated in-hospital deaths, and 26 760 (11 190-46 240) RSV-attributable deaths occurred in preterm infants worldwide. Among early preterm infants, the RSV-associated ALRI incidence rate and hospitalisation rate were significantly higher (rate ratio [RR] ranging from 1·69 to 3·87 across different age groups and outcomes) than for all infants born at any gestational age. In the second year of life, early preterm infants and young children had a similar incidence rate but still a significantly higher hospitalisation rate (RR 2·26 [95% UR 1·27-3·98]) compared with all infants and young children. Although late preterm infants had RSV-associated ALRI incidence rates similar to that of all infants younger than 1 year, they had higher RSV-associated ALRI hospitalisation rate in the first 6 months (RR 1·93 [1·11-3·26]). Overall, preterm infants accounted for 25% (95% UR 16-37) of RSV-associated ALRI hospitalisations in all infants of any gestational age. RSV-associated ALRI in-hospital case fatality ratio in preterm infants was similar to all infants. The factors identified to be associated with RSV-associated ALRI incidence were mainly perinatal and sociodemographic characteristics, and factors associated with severe outcomes from infection were mainly underlying medical conditions including congenital heart disease, tracheostomy, bronchopulmonary dysplasia, chronic lung disease, or Down syndrome (with ORs ranging from 1·40 to 4·23). INTERPRETATION: Preterm infants face a disproportionately high burden of RSV-associated disease, accounting for 25% of RSV hospitalisation burden. Early preterm infants have a substantial RSV hospitalisation burden persisting into the second year of life. Preventive products for RSV can have a substantial public health impact by preventing RSV-associated ALRI and severe outcomes from infection in preterm infants. FUNDING: EU Innovative Medicines Initiative Respiratory Syncytial Virus Consortium in Europe.

Association between pregnancy-related complications and development of type 2 diabetes and hypertension in women: an umbrella review.

BACKGROUND: Despite many systematic reviews and meta-analyses examining the associations of pregnancy complications with risk of type 2 diabetes mellitus (T2DM) and hypertension, previous umbrella reviews have only examined a single pregnancy complication. Here we have synthesised evidence from systematic reviews and meta-analyses on the associations of a wide range of pregnancy-related complications with risk of developing T2DM and hypertension. METHODS: Medline, Embase and Cochrane Database of Systematic Reviews were searched from inception until 26 September 2022 for systematic reviews and meta-analysis examining the association between pregnancy complications and risk of T2DM and hypertension. Screening of articles, data extraction and quality appraisal (AMSTAR2) were conducted independently by two reviewers using Covidence software. Data were extracted for studies that examined the risk of T2DM and hypertension in pregnant women with the pregnancy complication compared to pregnant women without the pregnancy complication. Summary estimates of each review were presented using tables, forest plots and narrative synthesis and reported following Preferred Reporting Items for Overviews of Reviews (PRIOR) guidelines. RESULTS: Ten systematic reviews were included. Two pregnancy complications were identified. Gestational diabetes mellitus (GDM): One review showed GDM was associated with a 10-fold higher risk of T2DM at least 1 year after pregnancy (relative risk (RR) 9.51 (95% confidence interval (CI) 7.14 to 12.67) and although the association differed by ethnicity (white: RR 16.28 (95% CI 15.01 to 17.66), non-white: RR 10.38 (95% CI 4.61 to 23.39), mixed: RR 8.31 (95% CI 5.44 to 12.69)), the between subgroups difference were not statistically significant at 5% significance level. Another review showed GDM was associated with higher mean blood pressure at least 3 months postpartum (mean difference in systolic blood pressure: 2.57 (95% CI 1.74 to 3.40) mmHg and mean difference in diastolic blood pressure: 1.89 (95% CI 1.32 to 2.46) mmHg). Hypertensive disorders of pregnancy (HDP): Three reviews showed women with a history of HDP were 3 to 6 times more likely to develop hypertension at least 6 weeks after pregnancy compared to women without HDP (meta-analysis with largest number of studies: odds ratio (OR) 4.33 (3.51 to 5.33)) and one review reported a higher rate of T2DM after HDP (hazard ratio (HR) 2.24 (1.95 to 2.58)) at least a year after pregnancy. One of the three reviews and five other reviews reported women with a history of preeclampsia were 3 to 7 times more likely to develop hypertension at least 6 weeks postpartum (meta-analysis with the largest number of studies: OR 3.90 (3.16 to 4.82) with one of these reviews reporting the association was greatest in women from Asia (Asia: OR 7.54 (95% CI 2.49 to 22.81), Europe: OR 2.19 (95% CI 0.30 to 16.02), North and South America: OR 3.32 (95% CI 1.26 to 8.74)). CONCLUSIONS: GDM and HDP are associated with a greater risk of developing T2DM and hypertension. Common confounders adjusted for across the included studies in the reviews were maternal age, body mass index (BMI), socioeconomic status, smoking status, pre-pregnancy and current BMI, parity, family history of T2DM or cardiovascular disease, ethnicity, and time of delivery. Further research is needed to evaluate the value of embedding these pregnancy complications as part of assessment for future risk of T2DM and chronic hypertension.

Discontinuation of anti-tumour necrosis factor alpha treatment owing to blood test abnormalities, and cost-effectiveness of alternate blood monitoring strategies.

BACKGROUND: There is no evidence base to support the use of 6-monthly monitoring blood tests for the early detection of liver, blood and renal toxicity during established anti-tumour necrosis factor alpha (TNFα) treatment. OBJECTIVES: To evaluate the incidence and risk factors of anti-TNFα treatment cessation owing to liver, blood and renal side-effects, and to estimate the cost-effectiveness of alternate intervals between monitoring blood tests. METHODS: A secondary care-based retrospective cohort study was performed. Data from the British Association of Dermatologists Biologic and Immunomodulators Register (BADBIR) were used. Patients with at least moderate psoriasis prescribed their first anti-TNFα treatment were included. Treatment discontinuation due to a monitoring blood test abnormality was the primary outcome. Patients were followed-up from start of treatment to the outcome of interest, drug discontinuation, death, 31 July 2021 or up to 5 years, whichever came first. The incidence rate (IR) and 95% confidence intervals (CIs) of anti-TNFα discontinuation with monitoring blood test abnormality was calculated. Multivariate Cox regression was used to examine the association between risk factors and outcome. A mathematical model evaluated costs and quality-adjusted life years (QALYs) associated with increasing the length of time between monitoring blood tests during anti-TNFα treatment. RESULTS: The cohort included 8819 participants [3710 (42.1%) female, mean (SD) age 44.76 (13.20) years] that contributed 25 058 person-years (PY) of follow-up and experienced 125 treatment discontinuations owing to a monitoring blood test abnormality at an IR of 5.85 (95% CI 4.91-6.97)/1000 PY. Of these, 64 and 61 discontinuations occurred within the first year and after the first year of treatment start, at IRs of 8.62 (95% CI 6.74-11.01) and 3.44 (95% CI 2.67-4.42)/1000 PY, respectively. Increasing age (in years), diabetes and liver disease were associated with anti-TNFα discontinuation after a monitoring blood test abnormality [adjusted hazard ratios of 1.02 (95% CI 1.01-1.04), 1.68 (95% CI 1.00-2.81) and 2.27 (95% CI 1.26-4.07), respectively]. Assuming a threshold of £20 000 per QALY gained, no monitoring was most cost-effective, but all extended periods were cost-effective vs. 3- or 6-monthly monitoring. CONCLUSIONS: Anti-TNFα drugs were uncommonly discontinued owing to abnormal monitoring blood tests after the first year of treatment. Extending the duration between monitoring blood tests was cost-effective. Our results produce evidence for specialist society guidance to reduce patient monitoring burden and healthcare costs.

Risk of bias assessments in individual participant data meta-analyses of test accuracy and prediction models: a review shows improvements are needed.

OBJECTIVES: Risk of bias assessments are important in meta-analyses of both aggregate and individual participant data (IPD). There is limited evidence on whether and how risk of bias of included studies or datasets in IPD meta-analyses (IPDMAs) is assessed. We review how risk of bias is currently assessed, reported, and incorporated in IPDMAs of test accuracy and clinical prediction model studies and provide recommendations for improvement. STUDY DESIGN AND SETTING: We searched PubMed (January 2018-May 2020) to identify IPDMAs of test accuracy and prediction models, then elicited whether each IPDMA assessed risk of bias of included studies and, if so, how assessments were reported and subsequently incorporated into the IPDMAs. RESULTS: Forty-nine IPDMAs were included. Nineteen of 27 (70%) test accuracy IPDMAs assessed risk of bias, compared to 5 of 22 (23%) prediction model IPDMAs. Seventeen of 19 (89%) test accuracy IPDMAs used Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2), but no tool was used consistently among prediction model IPDMAs. Of IPDMAs assessing risk of bias, 7 (37%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided details on the information sources (e.g., the original manuscript, IPD, primary investigators) used to inform judgments, and 4 (21%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided information or whether assessments were done before or after obtaining the IPD of the included studies or datasets. Of all included IPDMAs, only seven test accuracy IPDMAs (26%) and one prediction model IPDMA (5%) incorporated risk of bias assessments into their meta-analyses. For future IPDMA projects, we provide guidance on how to adapt tools such as Prediction model Risk Of Bias ASsessment Tool (for prediction models) and QUADAS-2 (for test accuracy) to assess risk of bias of included primary studies and their IPD. CONCLUSION: Risk of bias assessments and their reporting need to be improved in IPDMAs of test accuracy and, especially, prediction model studies. Using recommended tools, both before and after IPD are obtained, will address this.

Open science practices need substantial improvement in prognostic model studies in oncology using machine learning.

OBJECTIVE: To describe the frequency of open science practices in a contemporary sample of studies developing prognostic models using machine learning methods in the field of oncology. STUDY DESIGN AND SETTING: We conducted a systematic review, searching the MEDLINE database between December 1, 2022, and December 31, 2022, for studies developing a multivariable prognostic model using machine learning methods (as defined by the authors) in oncology. Two authors independently screened records and extracted open science practices. RESULTS: We identified 46 publications describing the development of a multivariable prognostic model. The adoption of open science principles was poor. Only one study reported availability of a study protocol, and only one study was registered. Funding statements and conflicts of interest statements were common. Thirty-five studies (76%) provided data sharing statements, with 21 (46%) indicating data were available on request to the authors and seven declaring data sharing was not applicable. Two studies (4%) shared data. Only 12 studies (26%) provided code sharing statements, including 2 (4%) that indicated the code was available on request to the authors. Only 11 studies (24%) provided sufficient information to allow their model to be used in practice. The use of reporting guidelines was rare: eight studies (18%) mentioning using a reporting guideline, with 4 (10%) using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis statement, 1 (2%) using Minimum Information About Clinical Artificial Intelligence Modeling and Consolidated Standards Of Reporting Trials-Artificial Intelligence, 1 (2%) using Strengthening The Reporting Of Observational Studies In Epidemiology, 1 (2%) using Standards for Reporting Diagnostic Accuracy Studies, and 1 (2%) using Transparent Reporting of Evaluations with Nonrandomized Designs. CONCLUSION: The adoption of open science principles in oncology studies developing prognostic models using machine learning methods is poor. Guidance and an increased awareness of benefits and best practices of open science are needed for prediction research in oncology.

Effect of a doctor working during the festive period on population health: natural experiment using 60 years of Doctor Who episodes (the TARDIS study).

OBJECTIVE: To examine the effect of a (fictional) doctor working during the festive period on population health. DESIGN: Natural experiment. SETTING: England, Wales, and the UK. MAIN OUTCOME MEASURES: Age standardised annual mortality rates in England, Wales, and the UK from 1963, when the BBC first broadcast Doctor Who, a fictional programme with a character called the Doctor who fights villains and intervenes to save others while travelling through space and time. Mortality rates were modelled in a time series analysis accounting for non-linear trends over time, and associations were estimated in relation to a new Doctor Who episode broadcast during the previous festive period, 24 December to 1 January. An interrupted time series analysis modelled the shift in mortality rates from 2005, when festive episodes of Doctor Who could be classed as a yearly Christmas intervention. RESULTS: 31 festive periods from 1963 have featured a new Doctor Who episode, including 14 broadcast on Christmas Day. In time series analyses, an association was found between broadcasts during the festive period and subsequent lower annual mortality rates. In particular, episodes shown on Christmas Day were associated with 0.60 fewer deaths per 1000 person years (95% confidence interval 0.21 to 0.99; P=0.003) in England and Wales and 0.40 fewer deaths per 1000 person years (0.08 to 0.73; P=0.02) in the UK. The interrupted time series analysis showed a strong shift (reduction) in mortality rates from 2005 onwards in association with the Doctor Who Christmas intervention, with a mean 0.73 fewer deaths per 1000 person years (0.21 to 1.26; P=0.01) in England and Wales and a mean 0.62 fewer deaths per 1000 person years (0.16 to 1.09; P=0.01) in the UK. CONCLUSIONS: A new Doctor Who episode shown every festive period, especially on Christmas Day, was associated with reduced mortality rates in England, Wales, and the UK, suggesting that a doctor working over the festive period could lower mortality rates. This finding reinforces why healthcare provision should not be taken for granted and may prompt the BBC and Disney+ to televise new episodes of Doctor Who every festive period, ideally on Christmas Day.

Clinical prediction models and the multiverse of madness.

BACKGROUND: Each year, thousands of clinical prediction models are developed to make predictions (e.g. estimated risk) to inform individual diagnosis and prognosis in healthcare. However, most are not reliable for use in clinical practice. MAIN BODY: We discuss how the creation of a prediction model (e.g. using regression or machine learning methods) is dependent on the sample and size of data used to develop it-were a different sample of the same size used from the same overarching population, the developed model could be very different even when the same model development methods are used. In other words, for each model created, there exists a multiverse of other potential models for that sample size and, crucially, an individual's predicted value (e.g. estimated risk) may vary greatly across this multiverse. The more an individual's prediction varies across the multiverse, the greater the instability. We show how small development datasets lead to more different models in the multiverse, often with vastly unstable individual predictions, and explain how this can be exposed by using bootstrapping and presenting instability plots. We recommend healthcare researchers seek to use large model development datasets to reduce instability concerns. This is especially important to ensure reliability across subgroups and improve model fairness in practice. CONCLUSIONS: Instability is concerning as an individual's predicted value is used to guide their counselling, resource prioritisation, and clinical decision making. If different samples lead to different models with very different predictions for the same individual, then this should cast doubt into using a particular model for that individual. Therefore, visualising, quantifying and reporting the instability in individual-level predictions is essential when proposing a new model.

Predictors of outcome in sciatica patients following an epidural steroid injection: the POiSE prospective observational cohort study protocol.

INTRODUCTION: Sciatica can be very painful and, in most cases, is due to pressure on a spinal nerve root from a disc herniation with associated inflammation. For some patients, the pain persists, and one management option is a spinal epidural steroid injection (ESI). The aim of an ESI is to relieve leg pain, improve function and reduce the need for surgery. ESIs work well in some patients but not in others, but we cannot identify these patient subgroups currently. This study aims to identify factors, including patient characteristics, clinical examination and imaging findings, that help in predicting who does well and who does not after an ESI. The overall objective is to develop a prognostic model to support individualised patient and clinical decision-making regarding ESI. METHODS: POiSE is a prospective cohort study of 439 patients with sciatica referred by their clinician for an ESI. Participants will receive weekly text messages until 12 weeks following their ESIand then again at 24 weeks following their ESI to collect data on leg pain severity. Questionnaires will be sent to participants at baseline, 6, 12 and 24 weeks after their ESI to collect data on pain, disability, recovery and additional interventions. The prognosis for the cohort will be described. The primary outcome measure for the prognostic model is leg pain at 6 weeks. Prognostic models will also be developed for secondary outcomes of disability and recovery at 6 weeks and additional interventions at 24 weeks following ESI. Statistical analyses will include multivariable linear and logistic regression with mixed effects model. ETHICS AND DISSEMINATION: The POiSE study has received ethical approval (South Central Berkshire B Research Ethics Committee 21/SC/0257). Dissemination will be guided by our patient and public engagement group and will include scientific publications, conference presentations and social media.

Up Front and Open? Shrouded in Secrecy? Or Somewhere in Between? A Meta-Research Systematic Review of Open Science Practices in Sport Medicine Research.

OBJECTIVE: To investigate open science practices in research published in the top 5 sports medicine journals from May 1, 2022, and October 1, 2022. DESIGN: A meta-research systematic review. LITERATURE SEARCH: Open science practices were searched in MEDLINE. STUDY SELECTION CRITERIA: We included original scientific research published in one of the identified top 5 sports medicine journals in 2022 as ranked by Clarivate: (1) British Journal of Sports Medicine, (2) Journal of Sport and Health Science, (3) American Journal of Sports Medicine, (4) Medicine and Science in Sports and Exercise, and (5) Sports Medicine-Open. Studies were excluded if they were systematic reviews, qualitative research, gray literature, or animal or cadaver models. DATA SYNTHESIS: Open science practices were extracted in accordance with the Transparency and Openness Promotion guidelines and patient and public involvement. RESULTS: Two hundred forty-three studies were included. The median number of open science practices in each study was 2, out of a maximum of 12 (range: 0-8; interquartile range: 2). Two hundred thirty-four studies (96%, 95% confidence interval [CI]: 94%-99%) provided an author conflict-of-interest statement and 163 (67%, 95% CI: 62%-73%) reported funding. Twenty-one studies (9%, 95% CI: 5%-12%) provided open-access data. Fifty-four studies (22%, 95% CI: 17%-27%) included a data availability statement and 3 (1%, 95% CI: 0%-3%) made code available. Seventy-six studies (32%, 95% CI: 25%-37%) had transparent materials and 30 (12%, 95% CI: 8%-16%) used a reporting guideline. Twenty-eight studies (12%, 95% CI: 8%-16%) were preregistered. Six studies (3%, 95% CI: 1%-4%) published a protocol. Four studies (2%, 95% CI: 0%-3%) reported an analysis plan a priori. Seven studies (3%, 95% CI: 1%-5%) reported patient and public involvement. CONCLUSION: Open science practices in the sports medicine field are extremely limited. The least followed practices were sharing code, data, and analysis plans. J Orthop Sports Phys Ther 2023;53(12):1-13. Epub 20 October 2023. doi:10.2519/jospt.2023.12016.

Risk-stratified monitoring for thiopurine toxicity in immune-mediated inflammatory diseases: prognostic model development, validation, and, health economic evaluation.

Background: Patients established on thiopurines (e.g., azathioprine) are recommended to undergo three-monthly blood tests for the early detection of blood, liver, or kidney toxicity. These side-effects are uncommon during long-term treatment. We developed a prognostic model that could be used to inform risk-stratified decisions on frequency of monitoring blood-tests during long-term thiopurine treatment, and, performed health-economic evaluation of alternate monitoring intervals. Methods: This was a retrospective cohort study set in the UK primary-care. Data from the Clinical Practice Research Datalink Aurum and Gold formed development and validation cohorts, respectively. People age ≥18 years, diagnosed with an immune mediated inflammatory disease, prescribed thiopurine by their general practitioner for at-least six-months between January 1, 2007 and December 31, 2019 were eligible. The outcome was thiopurine discontinuation with abnormal blood-test results. Patients were followed up from six-months after first primary-care thiopurine prescription to up to five-years. Penalised Cox regression developed the risk equation. Multiple imputation handled missing predictor data. Calibration and discrimination assessed model performance. A mathematical model evaluated costs and quality-adjusted life years associated with lengthening the interval between blood-tests. Findings: Data from 5982 (405 events over 16,117 person-years) and 3573 (269 events over 9075 person-years) participants were included in the development and validation cohorts, respectively. Fourteen candidate predictors (21 parameters) were included. The optimism adjusted R2 and Royston D statistic in development data were 0.11 and 0.76, respectively. The calibration slope and Royston D statistic (95% Confidence Interval) in the validation data were 1.10 (0.84-1.36) and 0.72 (0.52-0.92), respectively. A 2-year period between monitoring blood-test was most cost-effective in all deciles of predicted risk but the gain between monitoring annually or biennially reduced in higher risk deciles. Interpretation: This prognostic model requires information that is readily available during routine clinical care and may be used to risk-stratify blood-test monitoring for thiopurine toxicity. These findings should be considered by specialist societies when recommending blood monitoring during thiopurine prescription to bring about sustainable and equitable change in clinical practice. Funding: National Institute for Health and Care Research.