Calibration plots for multistate risk predictions models.

INTRODUCTION: There is currently no guidance on how to assess the calibration of multistate models used for risk prediction. We introduce several techniques that can be used to produce calibration plots for the transition probabilities of a multistate model, before assessing their performance in the presence of random and independent censoring through a simulation. METHODS: We studied pseudo-values based on the Aalen-Johansen estimator, binary logistic regression with inverse probability of censoring weights (BLR-IPCW), and multinomial logistic regression with inverse probability of censoring weights (MLR-IPCW). The MLR-IPCW approach results in a calibration scatter plot, providing extra insight about the calibration. We simulated data with varying levels of censoring and evaluated the ability of each method to estimate the calibration curve for a set of predicted transition probabilities. We also developed evaluated the calibration of a model predicting the incidence of cardiovascular disease, type 2 diabetes and chronic kidney disease among a cohort of patients derived from linked primary and secondary healthcare records. RESULTS: The pseudo-value, BLR-IPCW, and MLR-IPCW approaches give unbiased estimates of the calibration curves under random censoring. These methods remained predominately unbiased in the presence of independent censoring, even if the censoring mechanism was strongly associated with the outcome, with bias concentrated in low-density regions of predicted transition probability. CONCLUSIONS: We recommend implementing either the pseudo-value or BLR-IPCW approaches to produce a calibration curve, combined with the MLR-IPCW approach to produce a calibration scatter plot. The methods have been incorporated into the "calibmsm" R package available on CRAN.

GRADE concept paper 8: judging the certainty of discrimination performance estimates of prognostic models in a body of validation studies.

BACKGROUND: Prognostic models incorporate multiple prognostic factors to estimate the likelihood of future events for individual patients based on their prognostic factor values. Evaluating these models crucially involves conducting studies to assess their predictive performance, like discrimination. Systematic reviews and meta-analyses of these validation studies play an essential role in selecting models for clinical practice. METHODS: In this paper, we outline 3 thresholds to determine the target for certainty rating in the discrimination of prognostic models, as observed across a body of validation studies. RESULTS AND CONCLUSION: We propose 3 thresholds when rating the certainty of evidence about a prognostic model's discrimination. The first threshold amounts to rating certainty in the model's ability to classify better than random chance. The other 2 approaches involve setting thresholds informed by other mechanisms for classification: clinician intuition or an alternative prognostic model developed for the same disease area and outcome. The choice of threshold will vary based on the context. Instead of relying on arbitrary discrimination cut-offs, our approach positions the observed discrimination within an informed spectrum, potentially aiding decisions about a prognostic model's practical utility.

SPIN-PM: a consensus framework to evaluate the presence of spin in studies on prediction models.

OBJECTIVES: To develop a framework to identify and evaluate spin practices and its facilitators in studies on clinical prediction model regardless of the modeling technique. STUDY DESIGN AND SETTING: We followed a three-phase consensus process: (1) premeeting literature review to generate items to be included; (2) a series of structured meetings to provide comments discussed and exchanged viewpoints on items to be included with a panel of experienced researchers; and (3) postmeeting review on final list of items and examples to be included. Through this iterative consensus process, a framework was derived after all panel's researchers agreed. RESULTS: This consensus process involved a panel of eight researchers and resulted in SPIN-Prediction Models which consists of two categories of spin (misleading interpretation and misleading transportability), and within these categories, two forms of spin (spin practices and facilitators of spin). We provide criteria and examples. CONCLUSION: We proposed this guidance aiming to facilitate not only the accurate reporting but also an accurate interpretation and extrapolation of clinical prediction models which will likely improve the reporting quality of subsequent research, as well as reduce research waste.

Risk-stratified monitoring for sulfasalazine toxicity: prognostic model development and validation.

BACKGROUND: Sulfasalazine-induced cytopenia, nephrotoxicity and hepatotoxicity is uncommon during long-term treatment. Some guidelines recommend 3 monthly monitoring blood tests indefinitely during long-term treatment while others recommend stopping monitoring after 1 year. To rationalise monitoring, we developed and validated a prognostic model for clinically significant blood, liver or kidney toxicity during established sulfasalazine treatment. DESIGN: Retrospective cohort study. SETTING: UK primary care. Data from Clinical Practice Research Datalink Gold and Aurum formed independent development and validation cohorts. PARTICIPANTS: Age ≥18 years, new diagnosis of an inflammatory condition and sulfasalazine prescription. STUDY PERIOD: 1 January 2007 to 31 December 2019. OUTCOME: Sulfasalazine discontinuation with abnormal monitoring blood-test result. ANALYSIS: Patients were followed up from 6 months after first primary care prescription to the earliest of outcome, drug discontinuation, death, 5 years or 31 December 2019. Penalised Cox regression was performed to develop the risk equation. Multiple imputation handled missing predictor data. Model performance was assessed in terms of calibration and discrimination. RESULTS: 8936 participants were included in the development cohort (473 events, 23 299 person-years) and 5203 participants were included in the validation cohort (280 events, 12 867 person-years). Nine candidate predictors were included. The optimism adjusted R2 D and Royston D statistic in the development data were 0.13 and 0.79, respectively. The calibration slope (95% CI) and Royston D statistic (95% CI) in validation cohort was 1.19 (0.96 to 1.43) and 0.87 (0.67 to 1.07), respectively. CONCLUSION: This prognostic model for sulfasalazine toxicity uses readily available data and should be used to risk-stratify blood-test monitoring during established sulfasalazine treatment.

Community based complex interventions to sustain independence in older people: systematic review and network meta-analysis.

OBJECTIVE: To synthesise evidence of the effectiveness of community based complex interventions, grouped according to their intervention components, to sustain independence for older people. DESIGN: Systematic review and network meta-analysis. DATA SOURCES: Medline, Embase, CINAHL, PsycINFO, CENTRAL, clinicaltrials.gov, and International Clinical Trials Registry Platform from inception to 9 August 2021 and reference lists of included studies. ELIGIBILITY CRITERIA: Randomised controlled trials or cluster randomised controlled trials with ≥24 weeks' follow-up studying community based complex interventions for sustaining independence in older people (mean age ≥65 years) living at home, with usual care, placebo, or another complex intervention as comparators. MAIN OUTCOMES: Living at home, activities of daily living (personal/instrumental), care home placement, and service/economic outcomes at 12 months. DATA SYNTHESIS: Interventions were grouped according to a specifically developed typology. Random effects network meta-analysis estimated comparative effects; Cochrane's revised tool (RoB 2) structured risk of bias assessment. Grading of recommendations assessment, development and evaluation (GRADE) network meta-analysis structured certainty assessment. RESULTS: The review included 129 studies (74 946 participants). Nineteen intervention components, including "multifactorial action from individualised care planning" (a process of multidomain assessment and management leading to tailored actions), were identified in 63 combinations. For living at home, compared with no intervention/placebo, evidence favoured multifactorial action from individualised care planning including medication review and regular follow-ups (routine review) (odds ratio 1.22, 95% confidence interval 0.93 to 1.59; moderate certainty); multifactorial action from individualised care planning including medication review without regular follow-ups (2.55, 0.61 to 10.60; low certainty); combined cognitive training, medication review, nutritional support, and exercise (1.93, 0.79 to 4.77; low certainty); and combined activities of daily living training, nutritional support, and exercise (1.79, 0.67 to 4.76; low certainty). Risk screening or the addition of education and self-management strategies to multifactorial action from individualised care planning and routine review with medication review may reduce odds of living at home. For instrumental activities of daily living, evidence favoured multifactorial action from individualised care planning and routine review with medication review (standardised mean difference 0.11, 95% confidence interval 0.00 to 0.21; moderate certainty). Two interventions may reduce instrumental activities of daily living: combined activities of daily living training, aids, and exercise; and combined activities of daily living training, aids, education, exercise, and multifactorial action from individualised care planning and routine review with medication review and self-management strategies. For personal activities of daily living, evidence favoured combined exercise, multifactorial action from individualised care planning, and routine review with medication review and self-management strategies (0.16, -0.51 to 0.82; low certainty). For homecare recipients, evidence favoured addition of multifactorial action from individualised care planning and routine review with medication review (0.60, 0.32 to 0.88; low certainty). High risk of bias and imprecise estimates meant that most evidence was low or very low certainty. Few studies contributed to each comparison, impeding evaluation of inconsistency and frailty. CONCLUSIONS: The intervention most likely to sustain independence is individualised care planning including medicines optimisation and regular follow-up reviews resulting in multifactorial action. Homecare recipients may particularly benefit from this intervention. Unexpectedly, some combinations may reduce independence. Further research is needed to investigate which combinations of interventions work best for different participants and contexts. REGISTRATION: PROSPERO CRD42019162195.

Development and external validation of the eFalls tool: a multivariable prediction model for the risk of ED attendance or hospitalisation with a fall or fracture in older adults.

BACKGROUND: Falls are common in older adults and can devastate personal independence through injury such as fracture and fear of future falls. Methods to identify people for falls prevention interventions are currently limited, with high risks of bias in published prediction models. We have developed and externally validated the eFalls prediction model using routinely collected primary care electronic health records (EHR) to predict risk of emergency department attendance/hospitalisation with fall or fracture within 1 year. METHODS: Data comprised two independent, retrospective cohorts of adults aged ≥65 years: the population of Wales, from the Secure Anonymised Information Linkage Databank (model development); the population of Bradford and Airedale, England, from Connected Bradford (external validation). Predictors included electronic frailty index components, supplemented with variables informed by literature reviews and clinical expertise. Fall/fracture risk was modelled using multivariable logistic regression with a Least Absolute Shrinkage and Selection Operator penalty. Predictive performance was assessed through calibration, discrimination and clinical utility. Apparent, internal-external cross-validation and external validation performance were assessed across general practices and in clinically relevant subgroups. RESULTS: The model's discrimination performance (c-statistic) was 0.72 (95% confidence interval, CI: 0.68 to 0.76) on internal-external cross-validation and 0.82 (95% CI: 0.80 to 0.83) on external validation. Calibration was variable across practices, with some over-prediction in the validation population (calibration-in-the-large, -0.87; 95% CI: -0.96 to -0.78). Clinical utility on external validation was improved after recalibration. CONCLUSION: The eFalls prediction model shows good performance and could support proactive stratification for falls prevention services if appropriately embedded into primary care EHR systems.

Global disease burden of and risk factors for acute lower respiratory infections caused by respiratory syncytial virus in preterm infants and young children in 2019: a systematic review and meta-analysis of aggregated and individual participant data.

BACKGROUND: Infants and young children born prematurely are at high risk of severe acute lower respiratory infection (ALRI) caused by respiratory syncytial virus (RSV). In this study, we aimed to assess the global disease burden of and risk factors for RSV-associated ALRI in infants and young children born before 37 weeks of gestation. METHODS: We conducted a systematic review and meta-analysis of aggregated data from studies published between Jan 1, 1995, and Dec 31, 2021, identified from MEDLINE, Embase, and Global Health, and individual participant data shared by the Respiratory Virus Global Epidemiology Network on respiratory infectious diseases. We estimated RSV-associated ALRI incidence in community, hospital admission, in-hospital mortality, and overall mortality among children younger than 2 years born prematurely. We conducted two-stage random-effects meta-regression analyses accounting for chronological age groups, gestational age bands (early preterm, <32 weeks gestational age [wGA], and late preterm, 32 to <37 wGA), and changes over 5-year intervals from 2000 to 2019. Using individual participant data, we assessed perinatal, sociodemographic, and household factors, and underlying medical conditions for RSV-associated ALRI incidence, hospital admission, and three severity outcome groups (longer hospital stay [>4 days], use of supplemental oxygen and mechanical ventilation, or intensive care unit admission) by estimating pooled odds ratios (ORs) through a two-stage meta-analysis (multivariate logistic regression and random-effects meta-analysis). This study is registered with PROSPERO, CRD42021269742. FINDINGS: We included 47 studies from the literature and 17 studies with individual participant-level data contributed by the participating investigators. We estimated that, in 2019, 1 650 000 (95% uncertainty range [UR] 1 350 000-1 990 000) RSV-associated ALRI episodes, 533 000 (385 000-730 000) RSV-associated hospital admissions, 3050 (1080-8620) RSV-associated in-hospital deaths, and 26 760 (11 190-46 240) RSV-attributable deaths occurred in preterm infants worldwide. Among early preterm infants, the RSV-associated ALRI incidence rate and hospitalisation rate were significantly higher (rate ratio [RR] ranging from 1·69 to 3·87 across different age groups and outcomes) than for all infants born at any gestational age. In the second year of life, early preterm infants and young children had a similar incidence rate but still a significantly higher hospitalisation rate (RR 2·26 [95% UR 1·27-3·98]) compared with all infants and young children. Although late preterm infants had RSV-associated ALRI incidence rates similar to that of all infants younger than 1 year, they had higher RSV-associated ALRI hospitalisation rate in the first 6 months (RR 1·93 [1·11-3·26]). Overall, preterm infants accounted for 25% (95% UR 16-37) of RSV-associated ALRI hospitalisations in all infants of any gestational age. RSV-associated ALRI in-hospital case fatality ratio in preterm infants was similar to all infants. The factors identified to be associated with RSV-associated ALRI incidence were mainly perinatal and sociodemographic characteristics, and factors associated with severe outcomes from infection were mainly underlying medical conditions including congenital heart disease, tracheostomy, bronchopulmonary dysplasia, chronic lung disease, or Down syndrome (with ORs ranging from 1·40 to 4·23). INTERPRETATION: Preterm infants face a disproportionately high burden of RSV-associated disease, accounting for 25% of RSV hospitalisation burden. Early preterm infants have a substantial RSV hospitalisation burden persisting into the second year of life. Preventive products for RSV can have a substantial public health impact by preventing RSV-associated ALRI and severe outcomes from infection in preterm infants. FUNDING: EU Innovative Medicines Initiative Respiratory Syncytial Virus Consortium in Europe.

Association between pregnancy-related complications and development of type 2 diabetes and hypertension in women: an umbrella review.

BACKGROUND: Despite many systematic reviews and meta-analyses examining the associations of pregnancy complications with risk of type 2 diabetes mellitus (T2DM) and hypertension, previous umbrella reviews have only examined a single pregnancy complication. Here we have synthesised evidence from systematic reviews and meta-analyses on the associations of a wide range of pregnancy-related complications with risk of developing T2DM and hypertension. METHODS: Medline, Embase and Cochrane Database of Systematic Reviews were searched from inception until 26 September 2022 for systematic reviews and meta-analysis examining the association between pregnancy complications and risk of T2DM and hypertension. Screening of articles, data extraction and quality appraisal (AMSTAR2) were conducted independently by two reviewers using Covidence software. Data were extracted for studies that examined the risk of T2DM and hypertension in pregnant women with the pregnancy complication compared to pregnant women without the pregnancy complication. Summary estimates of each review were presented using tables, forest plots and narrative synthesis and reported following Preferred Reporting Items for Overviews of Reviews (PRIOR) guidelines. RESULTS: Ten systematic reviews were included. Two pregnancy complications were identified. Gestational diabetes mellitus (GDM): One review showed GDM was associated with a 10-fold higher risk of T2DM at least 1 year after pregnancy (relative risk (RR) 9.51 (95% confidence interval (CI) 7.14 to 12.67) and although the association differed by ethnicity (white: RR 16.28 (95% CI 15.01 to 17.66), non-white: RR 10.38 (95% CI 4.61 to 23.39), mixed: RR 8.31 (95% CI 5.44 to 12.69)), the between subgroups difference were not statistically significant at 5% significance level. Another review showed GDM was associated with higher mean blood pressure at least 3 months postpartum (mean difference in systolic blood pressure: 2.57 (95% CI 1.74 to 3.40) mmHg and mean difference in diastolic blood pressure: 1.89 (95% CI 1.32 to 2.46) mmHg). Hypertensive disorders of pregnancy (HDP): Three reviews showed women with a history of HDP were 3 to 6 times more likely to develop hypertension at least 6 weeks after pregnancy compared to women without HDP (meta-analysis with largest number of studies: odds ratio (OR) 4.33 (3.51 to 5.33)) and one review reported a higher rate of T2DM after HDP (hazard ratio (HR) 2.24 (1.95 to 2.58)) at least a year after pregnancy. One of the three reviews and five other reviews reported women with a history of preeclampsia were 3 to 7 times more likely to develop hypertension at least 6 weeks postpartum (meta-analysis with the largest number of studies: OR 3.90 (3.16 to 4.82) with one of these reviews reporting the association was greatest in women from Asia (Asia: OR 7.54 (95% CI 2.49 to 22.81), Europe: OR 2.19 (95% CI 0.30 to 16.02), North and South America: OR 3.32 (95% CI 1.26 to 8.74)). CONCLUSIONS: GDM and HDP are associated with a greater risk of developing T2DM and hypertension. Common confounders adjusted for across the included studies in the reviews were maternal age, body mass index (BMI), socioeconomic status, smoking status, pre-pregnancy and current BMI, parity, family history of T2DM or cardiovascular disease, ethnicity, and time of delivery. Further research is needed to evaluate the value of embedding these pregnancy complications as part of assessment for future risk of T2DM and chronic hypertension.

Medical students' perceptions of an artificial intelligence (AI) assisted diagnosing program.

As artificial intelligence (AI) assisted diagnosing systems become accessible and user-friendly, evaluating how first-year medical students perceive such systems holds substantial importance in medical education. This study aimed to assess medical students' perceptions of an AI-assisted diagnostic tool known as 'Glass AI.' Data was collected from first year medical students enrolled in a 1.5-week Cell Physiology pre-clerkship unit. Students voluntarily participated in an activity that involved implementation of Glass AI to solve a clinical case. A questionnaire was designed using 3 domains: 1) immediate experience with Glass AI, 2) potential for Glass AI utilization in medical education, and 3) student deliberations of AI-assisted diagnostic systems for future healthcare environments. 73/202 (36.10%) of students completed the survey. 96% of the participants noted that Glass AI increased confidence in the diagnosis, 43% thought Glass AI lacked sufficient explanation, and 68% expressed risk concerns for the physician workforce. Students expressed future positive outlooks involving AI-assisted diagnosing systems in healthcare, provided strict regulations, are set to protect patient privacy and safety, address legal liability, remove system biases, and improve quality of patient care. In conclusion, first year medical students are aware that AI will play a role in their careers as students and future physicians.

Prognostic factors for liver, blood and kidney adverse events from glucocorticoid sparing immune-suppressing drugs in immune-mediated inflammatory diseases: a prognostic systematic review.

BACKGROUND: Immune-suppressing drugs can cause liver, kidney or blood toxicity. Prognostic factors for these adverse-events are poorly understood. PURPOSE: To ascertain prognostic factors associated with liver, blood or kidney adverse-events in people receiving immune-suppressing drugs. DATA SOURCES: MEDLINE, Web of Science, EMBASE and the Cochrane library (01 January 1995 to 05 January 2023), and supplementary sources. DATA EXTRACTION AND SYNTHESIS: Data were extracted by one reviewer using a modified CHARMS-PF checklist and validated by another. Two independent reviewers assessed risk of bias using Quality in Prognostic factor Studies tool and assessed the quality of evidence using a Grading of Recommendations Assessment, Development and Evaluation-informed framework. RESULTS: Fifty-six studies from 58 papers were included. High-quality evidence of the following associations was identified: elevated liver enzymes (6 studies) and folate non-supplementation (3 studies) are prognostic factors for hepatotoxicity in those treated with methotrexate; that mercaptopurine (vs azathioprine) (3 studies) was a prognostic factor for hepatotoxicity in those treated with thiopurines; that mercaptopurine (vs azathioprine) (3 studies) and poor-metaboliser status (4 studies) were prognostic factors for cytopenia in those treated with thiopurines; and that baseline elevated liver enzymes (3 studies) are a prognostic factor for hepatotoxicity in those treated with anti-tumour necrosis factors. Moderate and low quality evidence for several other demographic, lifestyle, comorbidities, baseline bloods/serologic or treatment-related prognostic factors were also identified. LIMITATIONS: Studies published before 1995, those with less than 200 participants and not published in English were excluded. Heterogeneity between studies included different cut-offs for prognostic factors, use of different outcome definitions and different adjustment factors. CONCLUSIONS: Prognostic factors for target-organ damage were identified which may be further investigated for their potential role in targeted (risk-stratified) monitoring. PROSPERO REGISTRATION NUMBER: CRD42020208049.

Open science practices need substantial improvement in prognostic model studies in oncology using machine learning.

OBJECTIVE: To describe the frequency of open science practices in a contemporary sample of studies developing prognostic models using machine learning methods in the field of oncology. STUDY DESIGN AND SETTING: We conducted a systematic review, searching the MEDLINE database between December 1, 2022, and December 31, 2022, for studies developing a multivariable prognostic model using machine learning methods (as defined by the authors) in oncology. Two authors independently screened records and extracted open science practices. RESULTS: We identified 46 publications describing the development of a multivariable prognostic model. The adoption of open science principles was poor. Only one study reported availability of a study protocol, and only one study was registered. Funding statements and conflicts of interest statements were common. Thirty-five studies (76%) provided data sharing statements, with 21 (46%) indicating data were available on request to the authors and seven declaring data sharing was not applicable. Two studies (4%) shared data. Only 12 studies (26%) provided code sharing statements, including 2 (4%) that indicated the code was available on request to the authors. Only 11 studies (24%) provided sufficient information to allow their model to be used in practice. The use of reporting guidelines was rare: eight studies (18%) mentioning using a reporting guideline, with 4 (10%) using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis statement, 1 (2%) using Minimum Information About Clinical Artificial Intelligence Modeling and Consolidated Standards Of Reporting Trials-Artificial Intelligence, 1 (2%) using Strengthening The Reporting Of Observational Studies In Epidemiology, 1 (2%) using Standards for Reporting Diagnostic Accuracy Studies, and 1 (2%) using Transparent Reporting of Evaluations with Nonrandomized Designs. CONCLUSION: The adoption of open science principles in oncology studies developing prognostic models using machine learning methods is poor. Guidance and an increased awareness of benefits and best practices of open science are needed for prediction research in oncology.

Discontinuation of anti-tumour necrosis factor alpha treatment owing to blood test abnormalities, and cost-effectiveness of alternate blood monitoring strategies.

BACKGROUND: There is no evidence base to support the use of 6-monthly monitoring blood tests for the early detection of liver, blood and renal toxicity during established anti-tumour necrosis factor alpha (TNFα) treatment. OBJECTIVES: To evaluate the incidence and risk factors of anti-TNFα treatment cessation owing to liver, blood and renal side-effects, and to estimate the cost-effectiveness of alternate intervals between monitoring blood tests. METHODS: A secondary care-based retrospective cohort study was performed. Data from the British Association of Dermatologists Biologic and Immunomodulators Register (BADBIR) were used. Patients with at least moderate psoriasis prescribed their first anti-TNFα treatment were included. Treatment discontinuation due to a monitoring blood test abnormality was the primary outcome. Patients were followed-up from start of treatment to the outcome of interest, drug discontinuation, death, 31 July 2021 or up to 5 years, whichever came first. The incidence rate (IR) and 95% confidence intervals (CIs) of anti-TNFα discontinuation with monitoring blood test abnormality was calculated. Multivariate Cox regression was used to examine the association between risk factors and outcome. A mathematical model evaluated costs and quality-adjusted life years (QALYs) associated with increasing the length of time between monitoring blood tests during anti-TNFα treatment. RESULTS: The cohort included 8819 participants [3710 (42.1%) female, mean (SD) age 44.76 (13.20) years] that contributed 25 058 person-years (PY) of follow-up and experienced 125 treatment discontinuations owing to a monitoring blood test abnormality at an IR of 5.85 (95% CI 4.91-6.97)/1000 PY. Of these, 64 and 61 discontinuations occurred within the first year and after the first year of treatment start, at IRs of 8.62 (95% CI 6.74-11.01) and 3.44 (95% CI 2.67-4.42)/1000 PY, respectively. Increasing age (in years), diabetes and liver disease were associated with anti-TNFα discontinuation after a monitoring blood test abnormality [adjusted hazard ratios of 1.02 (95% CI 1.01-1.04), 1.68 (95% CI 1.00-2.81) and 2.27 (95% CI 1.26-4.07), respectively]. Assuming a threshold of £20 000 per QALY gained, no monitoring was most cost-effective, but all extended periods were cost-effective vs. 3- or 6-monthly monitoring. CONCLUSIONS: Anti-TNFα drugs were uncommonly discontinued owing to abnormal monitoring blood tests after the first year of treatment. Extending the duration between monitoring blood tests was cost-effective. Our results produce evidence for specialist society guidance to reduce patient monitoring burden and healthcare costs.

Risk of bias assessments in individual participant data meta-analyses of test accuracy and prediction models: a review shows improvements are needed.

OBJECTIVES: Risk of bias assessments are important in meta-analyses of both aggregate and individual participant data (IPD). There is limited evidence on whether and how risk of bias of included studies or datasets in IPD meta-analyses (IPDMAs) is assessed. We review how risk of bias is currently assessed, reported, and incorporated in IPDMAs of test accuracy and clinical prediction model studies and provide recommendations for improvement. STUDY DESIGN AND SETTING: We searched PubMed (January 2018-May 2020) to identify IPDMAs of test accuracy and prediction models, then elicited whether each IPDMA assessed risk of bias of included studies and, if so, how assessments were reported and subsequently incorporated into the IPDMAs. RESULTS: Forty-nine IPDMAs were included. Nineteen of 27 (70%) test accuracy IPDMAs assessed risk of bias, compared to 5 of 22 (23%) prediction model IPDMAs. Seventeen of 19 (89%) test accuracy IPDMAs used Quality Assessment of Diagnostic Accuracy Studies-2 (QUADAS-2), but no tool was used consistently among prediction model IPDMAs. Of IPDMAs assessing risk of bias, 7 (37%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided details on the information sources (e.g., the original manuscript, IPD, primary investigators) used to inform judgments, and 4 (21%) test accuracy IPDMAs and 1 (20%) prediction model IPDMA provided information or whether assessments were done before or after obtaining the IPD of the included studies or datasets. Of all included IPDMAs, only seven test accuracy IPDMAs (26%) and one prediction model IPDMA (5%) incorporated risk of bias assessments into their meta-analyses. For future IPDMA projects, we provide guidance on how to adapt tools such as Prediction model Risk Of Bias ASsessment Tool (for prediction models) and QUADAS-2 (for test accuracy) to assess risk of bias of included primary studies and their IPD. CONCLUSION: Risk of bias assessments and their reporting need to be improved in IPDMAs of test accuracy and, especially, prediction model studies. Using recommended tools, both before and after IPD are obtained, will address this.

Effect of a doctor working during the festive period on population health: natural experiment using 60 years of Doctor Who episodes (the TARDIS study).

OBJECTIVE: To examine the effect of a (fictional) doctor working during the festive period on population health. DESIGN: Natural experiment. SETTING: England, Wales, and the UK. MAIN OUTCOME MEASURES: Age standardised annual mortality rates in England, Wales, and the UK from 1963, when the BBC first broadcast Doctor Who, a fictional programme with a character called the Doctor who fights villains and intervenes to save others while travelling through space and time. Mortality rates were modelled in a time series analysis accounting for non-linear trends over time, and associations were estimated in relation to a new Doctor Who episode broadcast during the previous festive period, 24 December to 1 January. An interrupted time series analysis modelled the shift in mortality rates from 2005, when festive episodes of Doctor Who could be classed as a yearly Christmas intervention. RESULTS: 31 festive periods from 1963 have featured a new Doctor Who episode, including 14 broadcast on Christmas Day. In time series analyses, an association was found between broadcasts during the festive period and subsequent lower annual mortality rates. In particular, episodes shown on Christmas Day were associated with 0.60 fewer deaths per 1000 person years (95% confidence interval 0.21 to 0.99; P=0.003) in England and Wales and 0.40 fewer deaths per 1000 person years (0.08 to 0.73; P=0.02) in the UK. The interrupted time series analysis showed a strong shift (reduction) in mortality rates from 2005 onwards in association with the Doctor Who Christmas intervention, with a mean 0.73 fewer deaths per 1000 person years (0.21 to 1.26; P=0.01) in England and Wales and a mean 0.62 fewer deaths per 1000 person years (0.16 to 1.09; P=0.01) in the UK. CONCLUSIONS: A new Doctor Who episode shown every festive period, especially on Christmas Day, was associated with reduced mortality rates in England, Wales, and the UK, suggesting that a doctor working over the festive period could lower mortality rates. This finding reinforces why healthcare provision should not be taken for granted and may prompt the BBC and Disney+ to televise new episodes of Doctor Who every festive period, ideally on Christmas Day.

Factors affecting the implementation of calcium supplementation strategies during pregnancy to prevent pre-eclampsia: a mixed-methods systematic review.

OBJECTIVES: Daily calcium supplements are recommended for pregnant women from 20 weeks' gestation to prevent pre-eclampsia in populations with low dietary calcium intake. We aimed to improve understanding of barriers and facilitators for calcium supplement intake during pregnancy to prevent pre-eclampsia. DESIGN: Mixed-method systematic review, with confidence assessed using the Grading of Recommendations, Assessment, Development and Evaluations-Confidence in the Evidence from Reviews of Qualitative research approach. DATA SOURCES: MEDLINE and EMBASE (via Ovid), CINAHL and Global Health (via EBSCO) and grey literature databases were searched up to 17 September 2022. ELIGIBILITY CRITERIA: We included primary qualitative, quantitative and mixed-methods studies reporting implementation or use of calcium supplements during pregnancy, excluding calcium fortification and non-primary studies. No restrictions were imposed on settings, language or publication date. DATA EXTRACTION AND SYNTHESIS: Two independent reviewers extracted data and assessed risk of bias. We analysed the qualitative data using thematic synthesis, and quantitative findings were thematically mapped to qualitative findings. We then mapped the results to behavioural change frameworks to identify barriers and facilitators. RESULTS: Eighteen reports from nine studies were included in this review. Women reported barriers to consuming calcium supplements included limited knowledge about calcium supplements and pre-eclampsia, fears and experiences of side effects, varying preferences for tablets, dosing, working schedules, being away from home and taking other supplements. Receiving information regarding pre-eclampsia and safety of calcium supplement use from reliable sources, alternative dosing options, supplement reminders, early antenatal care, free supplements and support from families and communities were reported as facilitators. Healthcare providers felt that consistent messaging about benefits and risks of calcium, training, and ensuring adequate staffing and calcium supply is available would be able to help them in promoting calcium. CONCLUSION: Relevant stakeholders should consider the identified barriers and facilitators when formulating interventions and policies on calcium supplement use. These review findings can inform implementation to ensure effective and equitable provision and scale-up of calcium interventions. PROSPERO REGISTRATION NUMBER: CRD42021239143.