Global disease burden of and risk factors for acute lower respiratory infections caused by respiratory syncytial virus in preterm infants and young children in 2019: a systematic review and meta-analysis of aggregated and individual participant data.

BACKGROUND: Infants and young children born prematurely are at high risk of severe acute lower respiratory infection (ALRI) caused by respiratory syncytial virus (RSV). In this study, we aimed to assess the global disease burden of and risk factors for RSV-associated ALRI in infants and young children born before 37 weeks of gestation. METHODS: We conducted a systematic review and meta-analysis of aggregated data from studies published between Jan 1, 1995, and Dec 31, 2021, identified from MEDLINE, Embase, and Global Health, and individual participant data shared by the Respiratory Virus Global Epidemiology Network on respiratory infectious diseases. We estimated RSV-associated ALRI incidence in community, hospital admission, in-hospital mortality, and overall mortality among children younger than 2 years born prematurely. We conducted two-stage random-effects meta-regression analyses accounting for chronological age groups, gestational age bands (early preterm, <32 weeks gestational age [wGA], and late preterm, 32 to <37 wGA), and changes over 5-year intervals from 2000 to 2019. Using individual participant data, we assessed perinatal, sociodemographic, and household factors, and underlying medical conditions for RSV-associated ALRI incidence, hospital admission, and three severity outcome groups (longer hospital stay [>4 days], use of supplemental oxygen and mechanical ventilation, or intensive care unit admission) by estimating pooled odds ratios (ORs) through a two-stage meta-analysis (multivariate logistic regression and random-effects meta-analysis). This study is registered with PROSPERO, CRD42021269742. FINDINGS: We included 47 studies from the literature and 17 studies with individual participant-level data contributed by the participating investigators. We estimated that, in 2019, 1 650 000 (95% uncertainty range [UR] 1 350 000-1 990 000) RSV-associated ALRI episodes, 533 000 (385 000-730 000) RSV-associated hospital admissions, 3050 (1080-8620) RSV-associated in-hospital deaths, and 26 760 (11 190-46 240) RSV-attributable deaths occurred in preterm infants worldwide. Among early preterm infants, the RSV-associated ALRI incidence rate and hospitalisation rate were significantly higher (rate ratio [RR] ranging from 1·69 to 3·87 across different age groups and outcomes) than for all infants born at any gestational age. In the second year of life, early preterm infants and young children had a similar incidence rate but still a significantly higher hospitalisation rate (RR 2·26 [95% UR 1·27-3·98]) compared with all infants and young children. Although late preterm infants had RSV-associated ALRI incidence rates similar to that of all infants younger than 1 year, they had higher RSV-associated ALRI hospitalisation rate in the first 6 months (RR 1·93 [1·11-3·26]). Overall, preterm infants accounted for 25% (95% UR 16-37) of RSV-associated ALRI hospitalisations in all infants of any gestational age. RSV-associated ALRI in-hospital case fatality ratio in preterm infants was similar to all infants. The factors identified to be associated with RSV-associated ALRI incidence were mainly perinatal and sociodemographic characteristics, and factors associated with severe outcomes from infection were mainly underlying medical conditions including congenital heart disease, tracheostomy, bronchopulmonary dysplasia, chronic lung disease, or Down syndrome (with ORs ranging from 1·40 to 4·23). INTERPRETATION: Preterm infants face a disproportionately high burden of RSV-associated disease, accounting for 25% of RSV hospitalisation burden. Early preterm infants have a substantial RSV hospitalisation burden persisting into the second year of life. Preventive products for RSV can have a substantial public health impact by preventing RSV-associated ALRI and severe outcomes from infection in preterm infants. FUNDING: EU Innovative Medicines Initiative Respiratory Syncytial Virus Consortium in Europe.

Association between pregnancy-related complications and development of type 2 diabetes and hypertension in women: an umbrella review.

BACKGROUND: Despite many systematic reviews and meta-analyses examining the associations of pregnancy complications with risk of type 2 diabetes mellitus (T2DM) and hypertension, previous umbrella reviews have only examined a single pregnancy complication. Here we have synthesised evidence from systematic reviews and meta-analyses on the associations of a wide range of pregnancy-related complications with risk of developing T2DM and hypertension. METHODS: Medline, Embase and Cochrane Database of Systematic Reviews were searched from inception until 26 September 2022 for systematic reviews and meta-analysis examining the association between pregnancy complications and risk of T2DM and hypertension. Screening of articles, data extraction and quality appraisal (AMSTAR2) were conducted independently by two reviewers using Covidence software. Data were extracted for studies that examined the risk of T2DM and hypertension in pregnant women with the pregnancy complication compared to pregnant women without the pregnancy complication. Summary estimates of each review were presented using tables, forest plots and narrative synthesis and reported following Preferred Reporting Items for Overviews of Reviews (PRIOR) guidelines. RESULTS: Ten systematic reviews were included. Two pregnancy complications were identified. Gestational diabetes mellitus (GDM): One review showed GDM was associated with a 10-fold higher risk of T2DM at least 1 year after pregnancy (relative risk (RR) 9.51 (95% confidence interval (CI) 7.14 to 12.67) and although the association differed by ethnicity (white: RR 16.28 (95% CI 15.01 to 17.66), non-white: RR 10.38 (95% CI 4.61 to 23.39), mixed: RR 8.31 (95% CI 5.44 to 12.69)), the between subgroups difference were not statistically significant at 5% significance level. Another review showed GDM was associated with higher mean blood pressure at least 3 months postpartum (mean difference in systolic blood pressure: 2.57 (95% CI 1.74 to 3.40) mmHg and mean difference in diastolic blood pressure: 1.89 (95% CI 1.32 to 2.46) mmHg). Hypertensive disorders of pregnancy (HDP): Three reviews showed women with a history of HDP were 3 to 6 times more likely to develop hypertension at least 6 weeks after pregnancy compared to women without HDP (meta-analysis with largest number of studies: odds ratio (OR) 4.33 (3.51 to 5.33)) and one review reported a higher rate of T2DM after HDP (hazard ratio (HR) 2.24 (1.95 to 2.58)) at least a year after pregnancy. One of the three reviews and five other reviews reported women with a history of preeclampsia were 3 to 7 times more likely to develop hypertension at least 6 weeks postpartum (meta-analysis with the largest number of studies: OR 3.90 (3.16 to 4.82) with one of these reviews reporting the association was greatest in women from Asia (Asia: OR 7.54 (95% CI 2.49 to 22.81), Europe: OR 2.19 (95% CI 0.30 to 16.02), North and South America: OR 3.32 (95% CI 1.26 to 8.74)). CONCLUSIONS: GDM and HDP are associated with a greater risk of developing T2DM and hypertension. Common confounders adjusted for across the included studies in the reviews were maternal age, body mass index (BMI), socioeconomic status, smoking status, pre-pregnancy and current BMI, parity, family history of T2DM or cardiovascular disease, ethnicity, and time of delivery. Further research is needed to evaluate the value of embedding these pregnancy complications as part of assessment for future risk of T2DM and chronic hypertension.

Prognostic factors for liver, blood and kidney adverse events from glucocorticoid sparing immune-suppressing drugs in immune-mediated inflammatory diseases: a prognostic systematic review.

BACKGROUND: Immune-suppressing drugs can cause liver, kidney or blood toxicity. Prognostic factors for these adverse-events are poorly understood. PURPOSE: To ascertain prognostic factors associated with liver, blood or kidney adverse-events in people receiving immune-suppressing drugs. DATA SOURCES: MEDLINE, Web of Science, EMBASE and the Cochrane library (01 January 1995 to 05 January 2023), and supplementary sources. DATA EXTRACTION AND SYNTHESIS: Data were extracted by one reviewer using a modified CHARMS-PF checklist and validated by another. Two independent reviewers assessed risk of bias using Quality in Prognostic factor Studies tool and assessed the quality of evidence using a Grading of Recommendations Assessment, Development and Evaluation-informed framework. RESULTS: Fifty-six studies from 58 papers were included. High-quality evidence of the following associations was identified: elevated liver enzymes (6 studies) and folate non-supplementation (3 studies) are prognostic factors for hepatotoxicity in those treated with methotrexate; that mercaptopurine (vs azathioprine) (3 studies) was a prognostic factor for hepatotoxicity in those treated with thiopurines; that mercaptopurine (vs azathioprine) (3 studies) and poor-metaboliser status (4 studies) were prognostic factors for cytopenia in those treated with thiopurines; and that baseline elevated liver enzymes (3 studies) are a prognostic factor for hepatotoxicity in those treated with anti-tumour necrosis factors. Moderate and low quality evidence for several other demographic, lifestyle, comorbidities, baseline bloods/serologic or treatment-related prognostic factors were also identified. LIMITATIONS: Studies published before 1995, those with less than 200 participants and not published in English were excluded. Heterogeneity between studies included different cut-offs for prognostic factors, use of different outcome definitions and different adjustment factors. CONCLUSIONS: Prognostic factors for target-organ damage were identified which may be further investigated for their potential role in targeted (risk-stratified) monitoring. PROSPERO REGISTRATION NUMBER: CRD42020208049.

Open science practices need substantial improvement in prognostic model studies in oncology using machine learning.

OBJECTIVE: To describe the frequency of open science practices in a contemporary sample of studies developing prognostic models using machine learning methods in the field of oncology. STUDY DESIGN AND SETTING: We conducted a systematic review, searching the MEDLINE database between December 1, 2022, and December 31, 2022, for studies developing a multivariable prognostic model using machine learning methods (as defined by the authors) in oncology. Two authors independently screened records and extracted open science practices. RESULTS: We identified 46 publications describing the development of a multivariable prognostic model. The adoption of open science principles was poor. Only one study reported availability of a study protocol, and only one study was registered. Funding statements and conflicts of interest statements were common. Thirty-five studies (76%) provided data sharing statements, with 21 (46%) indicating data were available on request to the authors and seven declaring data sharing was not applicable. Two studies (4%) shared data. Only 12 studies (26%) provided code sharing statements, including 2 (4%) that indicated the code was available on request to the authors. Only 11 studies (24%) provided sufficient information to allow their model to be used in practice. The use of reporting guidelines was rare: eight studies (18%) mentioning using a reporting guideline, with 4 (10%) using the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis statement, 1 (2%) using Minimum Information About Clinical Artificial Intelligence Modeling and Consolidated Standards Of Reporting Trials-Artificial Intelligence, 1 (2%) using Strengthening The Reporting Of Observational Studies In Epidemiology, 1 (2%) using Standards for Reporting Diagnostic Accuracy Studies, and 1 (2%) using Transparent Reporting of Evaluations with Nonrandomized Designs. CONCLUSION: The adoption of open science principles in oncology studies developing prognostic models using machine learning methods is poor. Guidance and an increased awareness of benefits and best practices of open science are needed for prediction research in oncology.

Discontinuation of anti-tumour necrosis factor alpha treatment owing to blood test abnormalities, and cost-effectiveness of alternate blood monitoring strategies.

BACKGROUND: There is no evidence base to support the use of 6-monthly monitoring blood tests for the early detection of liver, blood and renal toxicity during established anti-tumour necrosis factor alpha (TNFα) treatment. OBJECTIVES: To evaluate the incidence and risk factors of anti-TNFα treatment cessation owing to liver, blood and renal side-effects, and to estimate the cost-effectiveness of alternate intervals between monitoring blood tests. METHODS: A secondary care-based retrospective cohort study was performed. Data from the British Association of Dermatologists Biologic and Immunomodulators Register (BADBIR) were used. Patients with at least moderate psoriasis prescribed their first anti-TNFα treatment were included. Treatment discontinuation due to a monitoring blood test abnormality was the primary outcome. Patients were followed-up from start of treatment to the outcome of interest, drug discontinuation, death, 31 July 2021 or up to 5 years, whichever came first. The incidence rate (IR) and 95% confidence intervals (CIs) of anti-TNFα discontinuation with monitoring blood test abnormality was calculated. Multivariate Cox regression was used to examine the association between risk factors and outcome. A mathematical model evaluated costs and quality-adjusted life years (QALYs) associated with increasing the length of time between monitoring blood tests during anti-TNFα treatment. RESULTS: The cohort included 8819 participants [3710 (42.1%) female, mean (SD) age 44.76 (13.20) years] that contributed 25 058 person-years (PY) of follow-up and experienced 125 treatment discontinuations owing to a monitoring blood test abnormality at an IR of 5.85 (95% CI 4.91-6.97)/1000 PY. Of these, 64 and 61 discontinuations occurred within the first year and after the first year of treatment start, at IRs of 8.62 (95% CI 6.74-11.01) and 3.44 (95% CI 2.67-4.42)/1000 PY, respectively. Increasing age (in years), diabetes and liver disease were associated with anti-TNFα discontinuation after a monitoring blood test abnormality [adjusted hazard ratios of 1.02 (95% CI 1.01-1.04), 1.68 (95% CI 1.00-2.81) and 2.27 (95% CI 1.26-4.07), respectively]. Assuming a threshold of £20 000 per QALY gained, no monitoring was most cost-effective, but all extended periods were cost-effective vs. 3- or 6-monthly monitoring. CONCLUSIONS: Anti-TNFα drugs were uncommonly discontinued owing to abnormal monitoring blood tests after the first year of treatment. Extending the duration between monitoring blood tests was cost-effective. Our results produce evidence for specialist society guidance to reduce patient monitoring burden and healthcare costs.

Predictors of outcome in sciatica patients following an epidural steroid injection: the POiSE prospective observational cohort study protocol.

INTRODUCTION: Sciatica can be very painful and, in most cases, is due to pressure on a spinal nerve root from a disc herniation with associated inflammation. For some patients, the pain persists, and one management option is a spinal epidural steroid injection (ESI). The aim of an ESI is to relieve leg pain, improve function and reduce the need for surgery. ESIs work well in some patients but not in others, but we cannot identify these patient subgroups currently. This study aims to identify factors, including patient characteristics, clinical examination and imaging findings, that help in predicting who does well and who does not after an ESI. The overall objective is to develop a prognostic model to support individualised patient and clinical decision-making regarding ESI. METHODS: POiSE is a prospective cohort study of 439 patients with sciatica referred by their clinician for an ESI. Participants will receive weekly text messages until 12 weeks following their ESIand then again at 24 weeks following their ESI to collect data on leg pain severity. Questionnaires will be sent to participants at baseline, 6, 12 and 24 weeks after their ESI to collect data on pain, disability, recovery and additional interventions. The prognosis for the cohort will be described. The primary outcome measure for the prognostic model is leg pain at 6 weeks. Prognostic models will also be developed for secondary outcomes of disability and recovery at 6 weeks and additional interventions at 24 weeks following ESI. Statistical analyses will include multivariable linear and logistic regression with mixed effects model. ETHICS AND DISSEMINATION: The POiSE study has received ethical approval (South Central Berkshire B Research Ethics Committee 21/SC/0257). Dissemination will be guided by our patient and public engagement group and will include scientific publications, conference presentations and social media.

Using temporal recalibration to improve the calibration of risk prediction models in competing risk settings when there are trends in survival over time.

We have previously proposed temporal recalibration to account for trends in survival over time to improve the calibration of predictions from prognostic models for new patients. This involves first estimating the predictor effects using data from all individuals (full dataset) and then re-estimating the baseline using a subset of the most recent data whilst constraining the predictor effects to remain the same. In this article, we demonstrate how temporal recalibration can be applied in competing risk settings by recalibrating each cause-specific (or subdistribution) hazard model separately. We illustrate this using an example of colon cancer survival with data from the Surveillance Epidemiology and End Results (SEER) program. Data from patients diagnosed in 1995-2004 were used to fit two models for deaths due to colon cancer and other causes respectively. We discuss considerations that need to be made in order to apply temporal recalibration such as the choice of data used in the recalibration step. We also demonstrate how to assess the calibration of these models in new data for patients diagnosed subsequently in 2005. Comparison was made to a standard analysis (when improvements over time are not taken into account) and a period analysis which is similar to temporal recalibration but differs in the data used to estimate the predictor effects. The 10-year calibration plots demonstrated that using the standard approach over-estimated the risk of death due to colon cancer and the total risk of death and that calibration was improved using temporal recalibration or period analysis.

Overinterpretation of findings in machine learning prediction model studies in oncology: a systematic review.

OBJECTIVES: In biomedical research, spin is the overinterpretation of findings, and it is a growing concern. To date, the presence of spin has not been evaluated in prognostic model research in oncology, including studies developing and validating models for individualized risk prediction. STUDY DESIGN AND SETTING: We conducted a systematic review, searching MEDLINE and EMBASE for oncology-related studies that developed and validated a prognostic model using machine learning published between 1st January, 2019, and 5th September, 2019. We used existing spin frameworks and described areas of highly suggestive spin practices. RESULTS: We included 62 publications (including 152 developed models; 37 validated models). Reporting was inconsistent between methods and the results in 27% of studies due to additional analysis and selective reporting. Thirty-two studies (out of 36 applicable studies) reported comparisons between developed models in their discussion and predominantly used discrimination measures to support their claims (78%). Thirty-five studies (56%) used an overly strong or leading word in their title, abstract, results, discussion, or conclusion. CONCLUSION: The potential for spin needs to be considered when reading, interpreting, and using studies that developed and validated prognostic models in oncology. Researchers should carefully report their prognostic model research using words that reflect their actual results and strength of evidence.

Minimum sample size for developing a multivariable prediction model using multinomial logistic regression.

AIMS: Multinomial logistic regression models allow one to predict the risk of a categorical outcome with > 2 categories. When developing such a model, researchers should ensure the number of participants (n) is appropriate relative to the number of events (Ek) and the number of predictor parameters (pk) for each category k. We propose three criteria to determine the minimum n required in light of existing criteria developed for binary outcomes. PROPOSED CRITERIA: The first criterion aims to minimise the model overfitting. The second aims to minimise the difference between the observed and adjusted R2 Nagelkerke. The third criterion aims to ensure the overall risk is estimated precisely. For criterion (i), we show the sample size must be based on the anticipated Cox-snell R2 of distinct 'one-to-one' logistic regression models corresponding to the sub-models of the multinomial logistic regression, rather than on the overall Cox-snell R2 of the multinomial logistic regression. EVALUATION OF CRITERIA: We tested the performance of the proposed criteria (i) through a simulation study and found that it resulted in the desired level of overfitting. Criterion (ii) and (iii) were natural extensions from previously proposed criteria for binary outcomes and did not require evaluation through simulation. SUMMARY: We illustrated how to implement the sample size criteria through a worked example considering the development of a multinomial risk prediction model for tumour type when presented with an ovarian mass. Code is provided for the simulation and worked example. We will embed our proposed criteria within the pmsampsize R library and Stata modules.

Association of pregnancy complications/risk factors with the development of future long-term health conditions in women: overarching protocol for umbrella reviews.

INTRODUCTION: With good medical care, most pregnancy complications like pre-eclampsia, gestational diabetes, etc resolve after childbirth. However, pregnancy complications are known to be associated with an increased risk of new long-term health conditions for women later in life, such as cardiovascular disease. These umbrella reviews aim to summarise systematic reviews evaluating the association between pregnancy complications and five groups of long-term health conditions: autoimmune conditions, cancers, functional disorders, mental health conditions and metabolic health conditions (diabetes and hypertension). METHODS AND ANALYSIS: We will conduct searches in Medline, Embase and the Cochrane database of systematic reviews without any language restrictions. We will include systematic reviews with or without meta-analyses that studied the association between pregnancy complications and the future risk of the five groups of long-term health conditions in women. Pregnancy complications were identified from existing core outcome sets for pregnancy and after consultation with experts. Two reviewers will independently screen the articles. Data will be synthesised with both narrative and quantitative methods. Where a meta-analysis has been carried out, we will report the combined effect size from individual studies. For binary data, pooled ORs with 95% CIs will be presented. For continuous data, we will use the mean difference with 95% CIs. The findings will be presented in forest plots to assess heterogeneity. The methodological quality of the studies will be evaluated with the AMSTAR 2 tool or the Cochrane risk of bias tool. The corrected covered area method will be used to assess the impact of overlap in reviews. The findings will be used to inform the design of prediction models, which will predict the risk of women developing these five group of health conditions following a pregnancy complication. ETHICS AND DISSEMINATION: No ethical approvals required. Findings will be disseminated through publications in peer-reviewed journals and conference presentations.

Risk of bias of prognostic models developed using machine learning: a systematic review in oncology.

BACKGROUND: Prognostic models are used widely in the oncology domain to guide medical decision-making. Little is known about the risk of bias of prognostic models developed using machine learning and the barriers to their clinical uptake in the oncology domain. METHODS: We conducted a systematic review and searched MEDLINE and EMBASE databases for oncology-related studies developing a prognostic model using machine learning methods published between 01/01/2019 and 05/09/2019. The primary outcome was risk of bias, judged using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). We described risk of bias overall and for each domain, by development and validation analyses separately. RESULTS: We included 62 publications (48 development-only; 14 development with validation). 152 models were developed across all publications and 37 models were validated. 84% (95% CI: 77 to 89) of developed models and 51% (95% CI: 35 to 67) of validated models were at overall high risk of bias. Bias introduced in the analysis was the largest contributor to the overall risk of bias judgement for model development and validation. 123 (81%, 95% CI: 73.8 to 86.4) developed models and 19 (51%, 95% CI: 35.1 to 67.3) validated models were at high risk of bias due to their analysis, mostly due to shortcomings in the analysis including insufficient sample size and split-sample internal validation. CONCLUSIONS: The quality of machine learning based prognostic models in the oncology domain is poor and most models have a high risk of bias, contraindicating their use in clinical practice. Adherence to better standards is urgently needed, with a focus on sample size estimation and analysis methods, to improve the quality of these models.

Methodological conduct of prognostic prediction models developed using machine learning in oncology: a systematic review.

BACKGROUND: Describe and evaluate the methodological conduct of prognostic prediction models developed using machine learning methods in oncology. METHODS: We conducted a systematic review in MEDLINE and Embase between 01/01/2019 and 05/09/2019, for studies developing a prognostic prediction model using machine learning methods in oncology. We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement, Prediction model Risk Of Bias ASsessment Tool (PROBAST) and CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS) to assess the methodological conduct of included publications. Results were summarised by modelling type: regression-, non-regression-based and ensemble machine learning models. RESULTS: Sixty-two publications met inclusion criteria developing 152 models across all publications. Forty-two models were regression-based, 71 were non-regression-based and 39 were ensemble models. A median of 647 individuals (IQR: 203 to 4059) and 195 events (IQR: 38 to 1269) were used for model development, and 553 individuals (IQR: 69 to 3069) and 50 events (IQR: 17.5 to 326.5) for model validation. A higher number of events per predictor was used for developing regression-based models (median: 8, IQR: 7.1 to 23.5), compared to alternative machine learning (median: 3.4, IQR: 1.1 to 19.1) and ensemble models (median: 1.7, IQR: 1.1 to 6). Sample size was rarely justified (n = 5/62; 8%). Some or all continuous predictors were categorised before modelling in 24 studies (39%). 46% (n = 24/62) of models reporting predictor selection before modelling used univariable analyses, and common method across all modelling types. Ten out of 24 models for time-to-event outcomes accounted for censoring (42%). A split sample approach was the most popular method for internal validation (n = 25/62, 40%). Calibration was reported in 11 studies. Less than half of models were reported or made available. CONCLUSIONS: The methodological conduct of machine learning based clinical prediction models is poor. Guidance is urgently needed, with increased awareness and education of minimum prediction modelling standards. Particular focus is needed on sample size estimation, development and validation analysis methods, and ensuring the model is available for independent validation, to improve quality of machine learning based clinical prediction models.

A prognostic model, including quantitative fetal fibronectin, to predict preterm labour: the QUIDS meta-analysis and prospective cohort study.

BACKGROUND: The diagnosis of preterm labour is challenging. False-positive diagnoses are common and result in unnecessary, potentially harmful treatments (e.g. tocolytics, antenatal corticosteroids and magnesium sulphate) and costly hospital admissions. Measurement of fetal fibronectin in vaginal fluid is a biochemical test that can indicate impending preterm birth. OBJECTIVES: To develop an externally validated prognostic model using quantitative fetal fibronectin concentration, in combination with clinical risk factors, for the prediction of spontaneous preterm birth and to assess its cost-effectiveness. DESIGN: The study comprised (1) a qualitative study to establish the decisional needs of pregnant women and their caregivers, (2) an individual participant data meta-analysis of existing studies to develop a prognostic model for spontaneous preterm birth within 7 days in women with symptoms of preterm labour based on quantitative fetal fibronectin and clinical risk factors, (3) external validation of the prognostic model in a prospective cohort study across 26 UK centres, (4) a model-based economic evaluation comparing the prognostic model with qualitative fetal fibronectin, and quantitative fetal fibronectin with cervical length measurement, in terms of cost per QALY gained and (5) a qualitative assessment of the acceptability of quantitative fetal fibronectin. DATA SOURCES/SETTING: The model was developed using data from five European prospective cohort studies of quantitative fetal fibronectin. The UK prospective cohort study was carried out across 26 UK centres. PARTICIPANTS: Pregnant women at 22+0-34+6 weeks' gestation with signs and symptoms of preterm labour. HEALTH TECHNOLOGY BEING ASSESSED: Quantitative fetal fibronectin. MAIN OUTCOME MEASURES: Spontaneous preterm birth within 7 days. RESULTS: The individual participant data meta-analysis included 1783 women and 139 events of spontaneous preterm birth within 7 days (event rate 7.8%). The prognostic model that was developed included quantitative fetal fibronectin, smoking, ethnicity, nulliparity and multiple pregnancy. The model was externally validated in a cohort of 2837 women, with 83 events of spontaneous preterm birth within 7 days (event rate 2.93%), an area under the curve of 0.89 (95% confidence interval 0.84 to 0.93), a calibration slope of 1.22 and a Nagelkerke R2 of 0.34. The economic analysis found that the prognostic model was cost-effective compared with using qualitative fetal fibronectin at a threshold for hospital admission and treatment of ≥ 2% risk of preterm birth within 7 days. LIMITATIONS: The outcome proportion (spontaneous preterm birth within 7 days of test) was 2.9% in the validation study. This is in line with other studies, but having slightly fewer than 100 events is a limitation in model validation. CONCLUSIONS: A prognostic model that included quantitative fetal fibronectin and clinical risk factors showed excellent performance in the prediction of spontaneous preterm birth within 7 days of test, was cost-effective and can be used to inform a decision support tool to help guide management decisions for women with threatened preterm labour. FUTURE WORK: The prognostic model will be embedded in electronic maternity records and a mobile telephone application, enabling ongoing data collection for further refinement and validation of the model. STUDY REGISTRATION: This study is registered as PROSPERO CRD42015027590 and Current Controlled Trials ISRCTN41598423. FUNDING: This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 52. See the NIHR Journals Library website for further project information.

Identifying which women with symptoms of labour will give birth early is challenging, so many women unnecessarily receive therapies aimed at preventing complications in preterm birth. A test called quantitative fetal fibronectin, which uses vaginal swab samples, may help to improve the diagnosis of preterm labour. Fetal fibronectin is a protein that is released from the fetal membranes that surround the developing baby in the womb. The lower the concentration of fetal fibronectin, the less likely the occurrence of preterm birth. Our aim was to see if quantitative fetal fibronectin, in combination with some features of pregnancy (e.g. previous pregnancy history and twin pregnancy), can accurately predict preterm birth in women who have symptoms of preterm labour. We asked women, their partners, doctors and midwives what information would be most useful to them, and how this should be presented. We then analysed previous research data; we used quantitative fetal fibronectin and clinical risk factors together to predict the chance of preterm birth. We explored which features could predict preterm birth most effectively while still being good value to the NHS. To ensure that this risk predictor worked in UK populations, we undertook a research study across 26 UK hospitals. Women who had symptoms of preterm labour were invited to participate. We collected information from these women (approximately 3000 women), including quantitative fetal fibronectin results. We found that a risk predictor comprising quantitative fetal fibronectin and four other features performed best at predicting whether or not preterm birth will occur within the next week for women with symptoms of preterm labour, and that this had potential to be clinically useful and cost-effective. The quantitative fetal fibronectin testing process was acceptable to women, and clinicians found the risk predictor useful. We used our findings to develop a risk calculator to help women and clinicians assess how likely preterm birth is, and decide whether or not to start treatment.

Prediction or causality? A scoping review of their conflation within current observational research.

Etiological research aims to uncover causal effects, whilst prediction research aims to forecast an outcome with the best accuracy. Causal and prediction research usually require different methods, and yet their findings may get conflated when reported and interpreted. The aim of the current study is to quantify the frequency of conflation between etiological and prediction research, to discuss common underlying mistakes and provide recommendations on how to avoid these. Observational cohort studies published in January 2018 in the top-ranked journals of six distinct medical fields (Cardiology, Clinical Epidemiology, Clinical Neurology, General and Internal Medicine, Nephrology and Surgery) were included for the current scoping review. Data on conflation was extracted through signaling questions. In total, 180 studies were included. Overall, 26% (n = 46) contained conflation between etiology and prediction. The frequency of conflation varied across medical field and journal impact factor. From the causal studies 22% was conflated, mainly due to the selection of covariates based on their ability to predict without taking the causal structure into account. Within prediction studies 38% was conflated, the most frequent reason was a causal interpretation of covariates included in a prediction model. Conflation of etiology and prediction is a common methodological error in observational medical research and more frequent in prediction studies. As this may lead to biased estimations and erroneous conclusions, researchers must be careful when designing, interpreting and disseminating their research to ensure this conflation is avoided.

Reporting of prognostic clinical prediction models based on machine learning methods in oncology needs to be improved.

OBJECTIVE: Evaluate the completeness of reporting of prognostic prediction models developed using machine learning methods in the field of oncology. STUDY DESIGN AND SETTING: We conducted a systematic review, searching the MEDLINE and Embase databases between 01/01/2019 and 05/09/2019, for non-imaging studies developing a prognostic clinical prediction model using machine learning methods (as defined by primary study authors) in oncology. We used the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement to assess the reporting quality of included publications. We described overall reporting adherence of included publications and by each section of TRIPOD. RESULTS: Sixty-two publications met the inclusion criteria. 48 were development studies and 14 were development with validation studies. 152 models were developed across all publications. Median adherence to TRIPOD reporting items was 41% [range: 10%-67%] and at least 50% adherence was found in 19% (n=12/62) of publications. Adherence was lower in development only studies (median: 38% [range: 10%-67%]); and higher in development with validation studies (median: 49% [range: 33%-59%]). CONCLUSION: Reporting of clinical prediction models using machine learning in oncology is poor and needs urgent improvement, so readers and stakeholders can appraise the study methods, understand study findings, and reduce research waste.

Development and validation of a risk prediction model of preterm birth for women with preterm labour symptoms (the QUIDS study): A prospective cohort study and individual participant data meta-analysis.

BACKGROUND: Timely interventions in women presenting with preterm labour can substantially improve health outcomes for preterm babies. However, establishing such a diagnosis is very challenging, as signs and symptoms of preterm labour are common and can be nonspecific. We aimed to develop and externally validate a risk prediction model using concentration of vaginal fluid fetal fibronectin (quantitative fFN), in combination with clinical risk factors, for the prediction of spontaneous preterm birth and assessed its cost-effectiveness. METHODS AND FINDINGS: Pregnant women included in the analyses were 22+0 to 34+6 weeks gestation with signs and symptoms of preterm labour. The primary outcome was spontaneous preterm birth within 7 days of quantitative fFN test. The risk prediction model was developed and internally validated in an individual participant data (IPD) meta-analysis of 5 European prospective cohort studies (2009 to 2016; 1,783 women; mean age 29.7 years; median BMI 24.8 kg/m2; 67.6% White; 11.7% smokers; 51.8% nulliparous; 10.4% with multiple pregnancy; 139 [7.8%] with spontaneous preterm birth within 7 days). The model was then externally validated in a prospective cohort study in 26 United Kingdom centres (2016 to 2018; 2,924 women; mean age 28.2 years; median BMI 25.4 kg/m2; 88.2% White; 21% smokers; 35.2% nulliparous; 3.5% with multiple pregnancy; 85 [2.9%] with spontaneous preterm birth within 7 days). The developed risk prediction model for spontaneous preterm birth within 7 days included quantitative fFN, current smoking, not White ethnicity, nulliparity, and multiple pregnancy. After internal validation, the optimism adjusted area under the curve was 0.89 (95% CI 0.86 to 0.92), and the optimism adjusted Nagelkerke R2 was 35% (95% CI 33% to 37%). On external validation in the prospective UK cohort population, the area under the curve was 0.89 (95% CI 0.84 to 0.94), and Nagelkerke R2 of 36% (95% CI: 34% to 38%). Recalibration of the model's intercept was required to ensure overall calibration-in-the-large. A calibration curve suggested close agreement between predicted and observed risks in the range of predictions 0% to 10%, but some miscalibration (underprediction) at higher risks (slope 1.24 (95% CI 1.23 to 1.26)). Despite any miscalibration, the net benefit of the model was higher than "treat all" or "treat none" strategies for thresholds up to about 15% risk. The economic analysis found the prognostic model was cost effective, compared to using qualitative fFN, at a threshold for hospital admission and treatment of ≥2% risk of preterm birth within 7 days. Study limitations include the limited number of participants who are not White and levels of missing data for certain variables in the development dataset. CONCLUSIONS: In this study, we found that a risk prediction model including vaginal fFN concentration and clinical risk factors showed promising performance in the prediction of spontaneous preterm birth within 7 days of test and has potential to inform management decisions for women with threatened preterm labour. Further evaluation of the risk prediction model in clinical practice is required to determine whether the risk prediction model improves clinical outcomes if used in practice. TRIAL REGISTRATION: The study was approved by the West of Scotland Research Ethics Committee (16/WS/0068). The study was registered with ISRCTN Registry (ISRCTN 41598423) and NIHR Portfolio (CPMS: 31277).

Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence.

INTRODUCTION: The Transparent Reporting of a multivariable prediction model of Individual Prognosis Or Diagnosis (TRIPOD) statement and the Prediction model Risk Of Bias ASsessment Tool (PROBAST) were both published to improve the reporting and critical appraisal of prediction model studies for diagnosis and prognosis. This paper describes the processes and methods that will be used to develop an extension to the TRIPOD statement (TRIPOD-artificial intelligence, AI) and the PROBAST (PROBAST-AI) tool for prediction model studies that applied machine learning techniques. METHODS AND ANALYSIS: TRIPOD-AI and PROBAST-AI will be developed following published guidance from the EQUATOR Network, and will comprise five stages. Stage 1 will comprise two systematic reviews (across all medical fields and specifically in oncology) to examine the quality of reporting in published machine-learning-based prediction model studies. In stage 2, we will consult a diverse group of key stakeholders using a Delphi process to identify items to be considered for inclusion in TRIPOD-AI and PROBAST-AI. Stage 3 will be virtual consensus meetings to consolidate and prioritise key items to be included in TRIPOD-AI and PROBAST-AI. Stage 4 will involve developing the TRIPOD-AI checklist and the PROBAST-AI tool, and writing the accompanying explanation and elaboration papers. In the final stage, stage 5, we will disseminate TRIPOD-AI and PROBAST-AI via journals, conferences, blogs, websites (including TRIPOD, PROBAST and EQUATOR Network) and social media. TRIPOD-AI will provide researchers working on prediction model studies based on machine learning with a reporting guideline that can help them report key details that readers need to evaluate the study quality and interpret its findings, potentially reducing research waste. We anticipate PROBAST-AI will help researchers, clinicians, systematic reviewers and policymakers critically appraise the design, conduct and analysis of machine learning based prediction model studies, with a robust standardised tool for bias evaluation. ETHICS AND DISSEMINATION: Ethical approval has been granted by the Central University Research Ethics Committee, University of Oxford on 10-December-2020 (R73034/RE001). Findings from this study will be disseminated through peer-review publications. PROSPERO REGISTRATION NUMBER: CRD42019140361 and CRD42019161764.

Diet and physical activity in pregnancy to prevent gestational diabetes: a protocol for an individual participant data (IPD) meta-analysis on the differential effects of interventions with economic evaluation.

INTRODUCTION: Mothers with gestational diabetes mellitus (GDM) are at increased risk of pregnancy-related complications and developing type 2 diabetes after delivery. Diet and physical activity-based interventions may prevent GDM, but variations in populations, interventions and outcomes in primary trials have limited the translation of available evidence into practice. We plan to undertake an individual participant data (IPD) meta-analysis of randomised trials to assess the differential effects and cost-effectiveness of diet and physical activity-based interventions in preventing GDM and its complications. METHODS: The International Weight Management in Pregnancy Collaborative Network database is a living repository of IPD from randomised trials on diet and physical activity in pregnancy identified through a systematic literature search. We shall update our existing search on MEDLINE, Embase, BIOSIS, LILACS, Pascal, Science Citation Index, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, Database of Abstracts of Reviews of Effects and Health Technology Assessment Database without language restriction to identify relevant trials until March 2021. Primary researchers will be invited to join the Network and share their IPD. Trials including women with GDM at baseline will be excluded. We shall perform a one and two stage random-effect meta-analysis for each intervention type (all interventions, diet-based, physical activity-based and mixed approach) to obtain summary intervention effects on GDM with 95% CIs and summary treatment-covariate interactions. Heterogeneity will be summarised using I2 and tau2 statistics with 95% prediction intervals. Publication and availability bias will be assessed by examining small study effects. Study quality of included trials will be assessed by the Cochrane Risk of Bias tool, and the Grading of Recommendations, Assessment, Development and Evaluations approach will be used to grade the evidence in the results. A model-based economic analysis will be carried out to assess the cost-effectiveness of interventions to prevent GDM and its complications compared with usual care. ETHICS AND DISSEMINATION: Ethics approval is not required. The study is registered on the International Prospective Register of Systematic Reviews (CRD42020212884). Results will be submitted for publication in peer-reviewed journals.

Sample sizes of prediction model studies in prostate cancer were rarely justified and often insufficient.

OBJECTIVE: Developing clinical prediction models (CPMs) on data of sufficient sample size is critical to help minimize overfitting. Using prostate cancer as a clinical exemplar, we aimed to investigate to what extent existing CPMs adhere to recent formal sample size criteria, or historic rules of thumb of events per predictor parameter (EPP)≥10. STUDY DESIGN AND SETTING: A systematic review to identify CPMs related to prostate cancer, which provided enough information to calculate minimum sample size. We compared the reported sample size of each CPM against the traditional 10 EPP rule of thumb and formal sample size criteria. RESULTS: About 211 CPMs were included. Three of the studies justified the sample size used, mostly using EPP rules of thumb. Overall, 69% of the CPMs were derived on sample sizes that surpassed the traditional EPP≥10 rule of thumb, but only 48% surpassed recent formal sample size criteria. For most CPMs, the required sample size based on formal criteria was higher than the sample sizes to surpass 10 EPP. CONCLUSION: Few of the CPMs included in this study justified their sample size, with most justifications being based on EPP. This study shows that, in real-world data sets, adhering to the classic EPP rules of thumb is insufficient to adhere to recent formal sample size criteria.

Temporal recalibration for improving prognostic model development and risk predictions in settings where survival is improving over time.

BACKGROUND: Prognostic models are typically developed in studies covering long time periods. However, if more recent years have seen improvements in survival, then using the full dataset may lead to out-of-date survival predictions. Period analysis addresses this by developing the model in a subset of the data from a recent time window, but results in a reduction of sample size. METHODS: We propose a new approach, called temporal recalibration, to combine the advantages of period analysis and full cohort analysis. This approach develops a model in the entire dataset and then recalibrates the baseline survival using a period analysis sample. The approaches are demonstrated utilizing a prognostic model in colon cancer built using both Cox proportional hazards and flexible parametric survival models with data from 1996-2005 from the Surveillance, Epidemiology, and End Results (SEER) Program database. Comparison of model predictions with observed survival estimates were made for new patients subsequently diagnosed in 2006 and followed-up until 2015. RESULTS: Period analysis and temporal recalibration provided more up-to-date survival predictions that more closely matched observed survival in subsequent data than the standard full cohort models. In addition, temporal recalibration provided more precise estimates of predictor effects. CONCLUSION: Prognostic models are typically developed using a full cohort analysis that can result in out-of-date long-term survival estimates when survival has improved in recent years. Temporal recalibration is a simple method to address this, which can be used when developing and updating prognostic models to ensure survival predictions are more closely calibrated with the observed survival of individuals diagnosed subsequently.