An evaluation of sample size requirements for developing risk prediction models with binary outcomes.

BACKGROUND: Risk prediction models are routinely used to assist in clinical decision making. A small sample size for model development can compromise model performance when the model is applied to new patients. For binary outcomes, the calibration slope (CS) and the mean absolute prediction error (MAPE) are two key measures on which sample size calculations for the development of risk models have been based. CS quantifies the degree of model overfitting while MAPE assesses the accuracy of individual predictions. METHODS: Recently, two formulae were proposed to calculate the sample size required, given anticipated features of the development data such as the outcome prevalence and c-statistic, to ensure that the expectation of the CS and MAPE (over repeated samples) in models fitted using MLE will meet prespecified target values. In this article, we use a simulation study to evaluate the performance of these formulae. RESULTS: We found that both formulae work reasonably well when the anticipated model strength is not too high (c-statistic < 0.8), regardless of the outcome prevalence. However, for higher model strengths the CS formula underestimates the sample size substantially. For example, for c-statistic = 0.85 and 0.9, the sample size needed to be increased by at least 50% and 100%, respectively, to meet the target expected CS. On the other hand, the MAPE formula tends to overestimate the sample size for high model strengths. These conclusions were more pronounced for higher prevalence than for lower prevalence. Similar results were drawn when the outcome was time to event with censoring. Given these findings, we propose a simulation-based approach, implemented in the new R package 'samplesizedev', to correctly estimate the sample size even for high model strengths. The software can also calculate the variability in CS and MAPE, thus allowing for assessment of model stability. CONCLUSIONS: The calibration and MAPE formulae suggest sample sizes that are generally appropriate for use when the model strength is not too high. However, they tend to be biased for higher model strengths, which are not uncommon in clinical risk prediction studies. On those occasions, our proposed adjustments to the sample size calculations will be relevant.

A Bayesian multivariate factor analysis model for causal inference using time-series observational data on mixed outcomes.

Assessing the impact of an intervention by using time-series observational data on multiple units and outcomes is a frequent problem in many fields of scientific research. Here, we propose a novel Bayesian multivariate factor analysis model for estimating intervention effects in such settings and develop an efficient Markov chain Monte Carlo algorithm to sample from the high-dimensional and nontractable posterior of interest. The proposed method is one of the few that can simultaneously deal with outcomes of mixed type (continuous, binomial, count), increase efficiency in the estimates of the causal effects by jointly modeling multiple outcomes affected by the intervention, and easily provide uncertainty quantification for all causal estimands of interest. Using the proposed approach, we evaluate the impact that Local Tracing Partnerships had on the effectiveness of England's Test and Trace programme for COVID-19.

Directions of change in intrinsic case severity across successive SARS-CoV-2 variant waves have been inconsistent.

OBJECTIVES: To determine how the intrinsic severity of successively dominant SARS-CoV-2 variants changed over the course of the pandemic. METHODS: A retrospective cohort analysis in the NHS Greater Glasgow and Clyde (NHS GGC) Health Board. All sequenced non-nosocomial adult COVID-19 cases in NHS GGC with relevant SARS-CoV-2 lineages (B.1.177/Alpha, Alpha/Delta, AY.4.2 Delta/non-AY.4.2 Delta, non-AY.4.2 Delta/Omicron, and BA.1 Omicron/BA.2 Omicron) during analysis periods were included. Outcome measures were hospital admission, ICU admission, or death within 28 days of positive COVID-19 test. We report the cumulative odds ratio; the ratio of the odds that an individual experiences a severity event of a given level vs all lower severity levels for the resident and the replacement variant after adjustment. RESULTS: After adjustment for covariates, the cumulative odds ratio was 1.51 (95% CI: 1.08-2.11) for Alpha versus B.1.177, 2.09 (95% CI: 1.42-3.08) for Delta versus Alpha, 0.99 (95% CI: 0.76-1.27) for AY.4.2 Delta versus non-AY.4.2 Delta, 0.49 (95% CI: 0.22-1.06) for Omicron versus non-AY.4.2 Delta, and 0.86 (95% CI: 0.68-1.09) for BA.2 Omicron versus BA.1 Omicron. CONCLUSIONS: The direction of change in intrinsic severity between successively emerging SARS-CoV-2 variants was inconsistent, reminding us that the intrinsic severity of future SARS-CoV-2 variants remains uncertain.

Comparative Effectiveness of Adalimumab vs Tofacitinib in Patients With Rheumatoid Arthritis in Australia.

Importance: There is a need for observational studies to supplement evidence from clinical trials, and the target trial emulation (TTE) framework can help avoid biases that can be introduced when treatments are compared crudely using observational data by applying design principles for randomized clinical trials. Adalimumab (ADA) and tofacitinib (TOF) were shown to be equivalent in patients with rheumatoid arthritis (RA) in a randomized clinical trial, but to our knowledge, these drugs have not been compared head-to-head using routinely collected clinical data and the TTE framework. Objective: To emulate a randomized clinical trial comparing ADA vs TOF in patients with RA who were new users of a biologic or targeted synthetic disease-modifying antirheumatic drug (b/tsDMARD). Design, Setting, and Participants: This comparative effectiveness study emulating a randomized clinical trial of ADA vs TOF included Australian adults aged 18 years or older with RA in the Optimising Patient Outcomes in Australian Rheumatology (OPAL) data set. Patients were included if they initiated ADA or TOF between October 1, 2015, and April 1, 2021; were new b/tsDMARD users; and had at least 1 component of the disease activity score in 28 joints using C-reactive protein (DAS28-CRP) recorded at baseline or during follow-up. Intervention: Treatment with either ADA (40 mg every 14 days) or TOF (10 mg daily). Main Outcomes and Measures: The main outcome was the estimated average treatment effect, defined as the difference in mean DAS28-CRP among patients receiving TOF compared with those receiving ADA at 3 and 9 months after initiating treatment. Missing DAS28-CRP data were multiply imputed. Stable balancing weights were used to account for nonrandomized treatment assignment. Results: A total of 842 patients were identified, including 569 treated with ADA (387 [68.0%] female; median age, 56 years [IQR, 47-66 years]) and 273 treated with TOF (201 [73.6%] female; median age, 59 years [IQR, 51-68 years]). After applying stable balancing weights, mean DAS28-CRP in the ADA group was 5.3 (95% CI, 5.2-5.4) at baseline, 2.6 (95% CI, 2.5-2.7) at 3 months, and 2.3 (95% CI, 2.2-2.4) at 9 months; in the TOF group, it was 5.3 (95% CI, 5.2-5.4) at baseline, 2.4 (95% CI, 2.2-2.5) at 3 months, and 2.3 (95% CI, 2.1-2.4) at 9 months. The estimated average treatment effect was -0.2 (95% CI, -0.4 to -0.03; P = .02) at 3 months and -0.03 (95% CI, -0.2 to 0.1; P = .60) at 9 months. Conclusions and Relevance: In this study, there was a modest but statistically significant reduction in DAS28-CRP at 3 months for patients receiving TOF compared with those receiving ADA and no difference between treatment groups at 9 months. Three months of treatment with either drug led to clinically relevant average reductions in mean DAS28-CRP, consistent with remission.

Causal inference in survival analysis using longitudinal observational data: Sequential trials and marginal structural models.

Longitudinal observational data on patients can be used to investigate causal effects of time-varying treatments on time-to-event outcomes. Several methods have been developed for estimating such effects by controlling for the time-dependent confounding that typically occurs. The most commonly used is marginal structural models (MSM) estimated using inverse probability of treatment weights (IPTW) (MSM-IPTW). An alternative, the sequential trials approach, is increasingly popular, and involves creating a sequence of "trials" from new time origins and comparing treatment initiators and non-initiators. Individuals are censored when they deviate from their treatment assignment at the start of each "trial" (initiator or noninitiator), which is accounted for using inverse probability of censoring weights. The analysis uses data combined across trials. We show that the sequential trials approach can estimate the parameters of a particular MSM. The causal estimand that we focus on is the marginal risk difference between the sustained treatment strategies of "always treat" vs "never treat." We compare how the sequential trials approach and MSM-IPTW estimate this estimand, and discuss their assumptions and how data are used differently. The performance of the two approaches is compared in a simulation study. The sequential trials approach, which tends to involve less extreme weights than MSM-IPTW, results in greater efficiency for estimating the marginal risk difference at most follow-up times, but this can, in certain scenarios, be reversed at later time points and relies on modelling assumptions. We apply the methods to longitudinal observational data from the UK Cystic Fibrosis Registry to estimate the effect of dornase alfa on survival.

Nowcasting COVID-19 deaths in England by age and region.

Understanding the trajectory of the daily number of COVID-19 deaths is essential to decisions on how to respond to the pandemic, but estimating this trajectory is complicated by the delay between deaths occurring and being reported. In England the delay is typically several days, but it can be weeks. This causes considerable uncertainty about how many deaths occurred in recent days. Here we estimate the deaths per day in five age strata within seven English regions, using a Bayesian model that accounts for reporting-day effects and longer-term changes in the delay distribution. We show how the model can be computationally efficiently fitted when the delay distribution is the same in multiple strata, for example, over a wide range of ages.

A comparison of two frameworks for multi-state modelling, applied to outcomes after hospital admissions with COVID-19.

We compare two multi-state modelling frameworks that can be used to represent dates of events following hospital admission for people infected during an epidemic. The methods are applied to data from people admitted to hospital with COVID-19, to estimate the probability of admission to intensive care unit, the probability of death in hospital for patients before and after intensive care unit admission, the lengths of stay in hospital, and how all these vary with age and gender. One modelling framework is based on defining transition-specific hazard functions for competing risks. A less commonly used framework defines partially-latent subpopulations who will experience each subsequent event, and uses a mixture model to estimate the probability that an individual will experience each event, and the distribution of the time to the event given that it occurs. We compare the advantages and disadvantages of these two frameworks, in the context of the COVID-19 example. The issues include the interpretation of the model parameters, the computational efficiency of estimating the quantities of interest, implementation in software and assessing goodness of fit. In the example, we find that some groups appear to be at very low risk of some events, in particular intensive care unit admission, and these are best represented by using 'cure-rate' models to define transition-specific hazards. We provide general-purpose software to implement all the models we describe in the flexsurv R package, which allows arbitrarily flexible distributions to be used to represent the cause-specific hazards or times to events.

Adjusting for time of infection or positive test when estimating the risk of a post-infection outcome in an epidemic.

When comparing the risk of a post-infection binary outcome, for example, hospitalisation, for two variants of an infectious pathogen, it is important to adjust for calendar time of infection. Typically, the infection time is unknown and positive test time used as a proxy for it. Positive test time may also be used when assessing how risk of the outcome changes over calendar time. We show that if time from infection to positive test is correlated with the outcome, the risk conditional on positive test time is a function of the trajectory of infection incidence. Hence, a risk ratio adjusted for positive test time can be quite different from the risk ratio adjusted for infection time. We propose a simple sensitivity analysis that indicates how risk ratios adjusted for positive test time and infection time may differ. This involves adjusting for a shifted positive test time, shifted to make the difference between it and infection time uncorrelated with the outcome. We illustrate this method by reanalysing published results on the relative risk of hospitalisation following infection with the Alpha versus pre-existing variants of SARS-CoV-2. Results indicate the relative risk adjusted for infection time may be lower than that adjusted for positive test time.

Sensitivity analysis for calibrated inverse probability-of-censoring weighted estimators under non-ignorable dropout.

Inverse probability of censoring weighting is a popular approach to handling dropout in longitudinal studies. However, inverse probability-of-censoring weighted estimators (IPCWEs) can be inefficient and unstable if the weights are estimated by maximum likelihood. To alleviate these problems, calibrated IPCWEs have been proposed, which use calibrated weights that directly optimize covariate balance in finite samples rather than the weights from maximum likelihood. However, the existing calibrated IPCWEs are all based on the unverifiable assumption of sequential ignorability and sensitivity analysis strategies under non-ignorable dropout are lacking. In this paper, we fill this gap by developing an approach to sensitivity analysis for calibrated IPCWEs under non-ignorable dropout. A simple technique is proposed to speed up the computation of bootstrap and jackknife confidence intervals and thus facilitate sensitivity analyses. We evaluate the finite-sample performance of the proposed methods using simulations and apply our methods to data from an international inception cohort study of systemic lupus erythematosus. An R Markdown tutorial to demonstrate the implementation of the proposed methods is provided.

Comparative analysis of the risks of hospitalisation and death associated with SARS-CoV-2 omicron (B.1.1.529) and delta (B.1.617.2) variants in England: a cohort study.

BACKGROUND: The omicron variant (B.1.1.529) of SARS-CoV-2 has demonstrated partial vaccine escape and high transmissibility, with early studies indicating lower severity of infection than that of the delta variant (B.1.617.2). We aimed to better characterise omicron severity relative to delta by assessing the relative risk of hospital attendance, hospital admission, or death in a large national cohort. METHODS: Individual-level data on laboratory-confirmed COVID-19 cases resident in England between Nov 29, 2021, and Jan 9, 2022, were linked to routine datasets on vaccination status, hospital attendance and admission, and mortality. The relative risk of hospital attendance or admission within 14 days, or death within 28 days after confirmed infection, was estimated using proportional hazards regression. Analyses were stratified by test date, 10-year age band, ethnicity, residential region, and vaccination status, and were further adjusted for sex, index of multiple deprivation decile, evidence of a previous infection, and year of age within each age band. A secondary analysis estimated variant-specific and vaccine-specific vaccine effectiveness and the intrinsic relative severity of omicron infection compared with delta (ie, the relative risk in unvaccinated cases). FINDINGS: The adjusted hazard ratio (HR) of hospital attendance (not necessarily resulting in admission) with omicron compared with delta was 0·56 (95% CI 0·54-0·58); for hospital admission and death, HR estimates were 0·41 (0·39-0·43) and 0·31 (0·26-0·37), respectively. Omicron versus delta HR estimates varied with age for all endpoints examined. The adjusted HR for hospital admission was 1·10 (0·85-1·42) in those younger than 10 years, decreasing to 0·25 (0·21-0·30) in 60-69-year-olds, and then increasing to 0·47 (0·40-0·56) in those aged at least 80 years. For both variants, past infection gave some protection against death both in vaccinated (HR 0·47 [0·32-0·68]) and unvaccinated (0·18 [0·06-0·57]) cases. In vaccinated cases, past infection offered no additional protection against hospital admission beyond that provided by vaccination (HR 0·96 [0·88-1·04]); however, for unvaccinated cases, past infection gave moderate protection (HR 0·55 [0·48-0·63]). Omicron versus delta HR estimates were lower for hospital admission (0·30 [0·28-0·32]) in unvaccinated cases than the corresponding HR estimated for all cases in the primary analysis. Booster vaccination with an mRNA vaccine was highly protective against hospitalisation and death in omicron cases (HR for hospital admission 8-11 weeks post-booster vs unvaccinated: 0·22 [0·20-0·24]), with the protection afforded after a booster not being affected by the vaccine used for doses 1 and 2. INTERPRETATION: The risk of severe outcomes following SARS-CoV-2 infection is substantially lower for omicron than for delta, with higher reductions for more severe endpoints and significant variation with age. Underlying the observed risks is a larger reduction in intrinsic severity (in unvaccinated individuals) counterbalanced by a reduction in vaccine effectiveness. Documented previous SARS-CoV-2 infection offered some protection against hospitalisation and high protection against death in unvaccinated individuals, but only offered additional protection in vaccinated individuals for the death endpoint. Booster vaccination with mRNA vaccines maintains over 70% protection against hospitalisation and death in breakthrough confirmed omicron infections. FUNDING: Medical Research Council, UK Research and Innovation, Department of Health and Social Care, National Institute for Health Research, Community Jameel, and Engineering and Physical Sciences Research Council.

Hospitalization and Mortality Risk for COVID-19 Cases With SARS-CoV-2 AY.4.2 (VUI-21OCT-01) Compared to Non-AY.4.2 Delta Variant Sublineages.

To investigate if the AY.4.2 sublineage of the SARS-CoV-2 delta variant is associated with hospitalization and mortality risks that differ from non-AY.4.2 delta risks, we performed a retrospective cohort study of sequencing-confirmed COVID-19 cases in England based on linkage of routine health care datasets. Using stratified Cox regression, we estimated adjusted hazard ratios (aHR) of hospital admission (aHR = 0.85; 95% confidence interval [CI], .77-.94), hospital admission or emergency care attendance (aHR = 0.87; 95% CI, .81-.94), and COVID-19 mortality (aHR = 0.85; 95% CI, .71-1.03). The results indicate that the risks of hospitalization and mortality are similar or lower for AY.4.2 compared to cases with other delta sublineages.

Estimating a time-to-event distribution from right-truncated data in an epidemic: A review of methods.

Time-to-event data are right-truncated if only individuals who have experienced the event by a certain time can be included in the sample. For example, we may be interested in estimating the distribution of time from onset of disease symptoms to death and only have data on individuals who have died. This may be the case, for example, at the beginning of an epidemic. Right truncation causes the distribution of times to event in the sample to be biased towards shorter times compared to the population distribution, and appropriate statistical methods should be used to account for this bias. This article is a review of such methods, particularly in the context of an infectious disease epidemic, like COVID-19. We consider methods for estimating the marginal time-to-event distribution, and compare their efficiencies. (Non-)identifiability of the distribution is an important issue with right-truncated data, particularly at the beginning of an epidemic, and this is discussed in detail. We also review methods for estimating the effects of covariates on the time to event. An illustration of the application of many of these methods is provided, using data on individuals who had died with coronavirus disease by 5 April 2020.

Hospital admission and emergency care attendance risk for SARS-CoV-2 delta (B.1.617.2) compared with alpha (B.1.1.7) variants of concern: a cohort study.

BACKGROUND: The SARS-CoV-2 delta (B.1.617.2) variant was first detected in England in March, 2021. It has since rapidly become the predominant lineage, owing to high transmissibility. It is suspected that the delta variant is associated with more severe disease than the previously dominant alpha (B.1.1.7) variant. We aimed to characterise the severity of the delta variant compared with the alpha variant by determining the relative risk of hospital attendance outcomes. METHODS: This cohort study was done among all patients with COVID-19 in England between March 29 and May 23, 2021, who were identified as being infected with either the alpha or delta SARS-CoV-2 variant through whole-genome sequencing. Individual-level data on these patients were linked to routine health-care datasets on vaccination, emergency care attendance, hospital admission, and mortality (data from Public Health England's Second Generation Surveillance System and COVID-19-associated deaths dataset; the National Immunisation Management System; and NHS Digital Secondary Uses Services and Emergency Care Data Set). The risk for hospital admission and emergency care attendance were compared between patients with sequencing-confirmed delta and alpha variants for the whole cohort and by vaccination status subgroups. Stratified Cox regression was used to adjust for age, sex, ethnicity, deprivation, recent international travel, area of residence, calendar week, and vaccination status. FINDINGS: Individual-level data on 43 338 COVID-19-positive patients (8682 with the delta variant, 34 656 with the alpha variant; median age 31 years [IQR 17-43]) were included in our analysis. 196 (2·3%) patients with the delta variant versus 764 (2·2%) patients with the alpha variant were admitted to hospital within 14 days after the specimen was taken (adjusted hazard ratio [HR] 2·26 [95% CI 1·32-3·89]). 498 (5·7%) patients with the delta variant versus 1448 (4·2%) patients with the alpha variant were admitted to hospital or attended emergency care within 14 days (adjusted HR 1·45 [1·08-1·95]). Most patients were unvaccinated (32 078 [74·0%] across both groups). The HRs for vaccinated patients with the delta variant versus the alpha variant (adjusted HR for hospital admission 1·94 [95% CI 0·47-8·05] and for hospital admission or emergency care attendance 1·58 [0·69-3·61]) were similar to the HRs for unvaccinated patients (2·32 [1·29-4·16] and 1·43 [1·04-1·97]; p=0·82 for both) but the precision for the vaccinated subgroup was low. INTERPRETATION: This large national study found a higher hospital admission or emergency care attendance risk for patients with COVID-19 infected with the delta variant compared with the alpha variant. Results suggest that outbreaks of the delta variant in unvaccinated populations might lead to a greater burden on health-care services than the alpha variant. FUNDING: Medical Research Council; UK Research and Innovation; Department of Health and Social Care; and National Institute for Health Research.

A Bayesian multivariate factor analysis model for evaluating an intervention by using observational time series data on multiple outcomes.

A problem that is frequently encountered in many areas of scientific research is that of estimating the effect of a non-randomized binary intervention on an outcome of interest by using time series data on units that received the intervention ('treated') and units that did not ('controls'). One popular estimation method in this setting is based on the factor analysis (FA) model. The FA model is fitted to the preintervention outcome data on treated units and all the outcome data on control units, and the counterfactual treatment-free post-intervention outcomes of the former are predicted from the fitted model. Intervention effects are estimated as the observed outcomes minus these predicted counterfactual outcomes. We propose a model that extends the FA model for estimating intervention effects by jointly modelling the multiple outcomes to exploit shared variability, and assuming an auto-regressive structure on factors to account for temporal correlations in the outcome. Using simulation studies, we show that the method proposed can improve the precision of the intervention effect estimates and achieve better control of the type I error rate (compared with the FA model), especially when either the number of preintervention measurements or the number of control units is small. We apply our method to estimate the effect of stricter alcohol licensing policies on alcohol-related harms.

Risk of hospital admission for patients with SARS-CoV-2 variant B.1.1.7: cohort analysis.

OBJECTIVE: To evaluate the relation between diagnosis of covid-19 with SARS-CoV-2 variant B.1.1.7 (also known as variant of concern 202012/01) and the risk of hospital admission compared with diagnosis with wild-type SARS-CoV-2 variants. DESIGN: Retrospective cohort analysis. SETTING: Community based SARS-CoV-2 testing in England, individually linked with hospital admission data. PARTICIPANTS: 839 278 patients with laboratory confirmed covid-19, of whom 36 233 had been admitted to hospital within 14 days, tested between 23 November 2020 and 31 January 2021 and analysed at a laboratory with an available TaqPath assay that enables assessment of S-gene target failure (SGTF), a proxy test for the B.1.1.7 variant. Patient data were stratified by age, sex, ethnicity, deprivation, region of residence, and date of positive test. MAIN OUTCOME MEASURES: Hospital admission between one and 14 days after the first positive SARS-CoV-2 test. RESULTS: 27 710 (4.7%) of 592 409 patients with SGTF variants and 8523 (3.5%) of 246 869 patients without SGTF variants had been admitted to hospital within one to 14 days. The stratum adjusted hazard ratio of hospital admission was 1.52 (95% confidence interval 1.47 to 1.57) for patients with covid-19 infected with SGTF variants, compared with those infected with non-SGTF variants. The effect was modified by age (P<0.001), with hazard ratios of 0.93-1.21 in patients younger than 20 years with versus without SGTF variants, 1.29 in those aged 20-29, and 1.45-1.65 in those aged ≥30 years. The adjusted absolute risk of hospital admission within 14 days was 4.7% (95% confidence interval 4.6% to 4.7%) for patients with SGTF variants and 3.5% (3.4% to 3.5%) for those with non-SGTF variants. CONCLUSIONS: The results suggest that the risk of hospital admission is higher for people infected with the B.1.1.7 variant compared with wild-type SARS-CoV-2, likely reflecting a more severe disease. The higher severity may be specific to adults older than 30 years.

Using generalized linear models to implement g-estimation for survival data with time-varying confounding.

Using data from observational studies to estimate the causal effect of a time-varying exposure, repeatedly measured over time, on an outcome of interest requires careful adjustment for confounding. Standard regression adjustment for observed time-varying confounders is unsuitable, as it can eliminate part of the causal effect and induce bias. Inverse probability weighting, g-computation, and g-estimation have been proposed as being more suitable methods. G-estimation has some advantages over the other two methods, but until recently there has been a lack of flexible g-estimation methods for a survival time outcome. The recently proposed Structural Nested Cumulative Survival Time Model (SNCSTM) is such a method. Efficient estimation of the parameters of this model required bespoke software. In this article we show how the SNCSTM can be fitted efficiently via g-estimation using standard software for fitting generalised linear models. The ability to implement g-estimation for a survival outcome using standard statistical software greatly increases the potential uptake of this method. We illustrate the use of this method of fitting the SNCSTM by reanalyzing data from the UK Cystic Fibrosis Registry, and provide example R code to facilitate the use of this approach by other researchers.

Simulating longitudinal data from marginal structural models using the additive hazard model.

Observational longitudinal data on treatments and covariates are increasingly used to investigate treatment effects, but are often subject to time-dependent confounding. Marginal structural models (MSMs), estimated using inverse probability of treatment weighting or the g-formula, are popular for handling this problem. With increasing development of advanced causal inference methods, it is important to be able to assess their performance in different scenarios to guide their application. Simulation studies are a key tool for this, but their use to evaluate causal inference methods has been limited. This paper focuses on the use of simulations for evaluations involving MSMs in studies with a time-to-event outcome. In a simulation, it is important to be able to generate the data in such a way that the correct forms of any models to be fitted to those data are known. However, this is not straightforward in the longitudinal setting because it is natural for data to be generated in a sequential conditional manner, whereas MSMs involve fitting marginal rather than conditional hazard models. We provide general results that enable the form of the correctly specified MSM to be derived based on a conditional data generating procedure, and show how the results can be applied when the conditional hazard model is an Aalen additive hazard or Cox model. Using conditional additive hazard models is advantageous because they imply additive MSMs that can be fitted using standard software. We describe and illustrate a simulation algorithm. Our results will help researchers to effectively evaluate causal inference methods via simulation.

Estimation of required sample size for external validation of risk models for binary outcomes.

Risk-prediction models for health outcomes are used in practice as part of clinical decision-making, and it is essential that their performance be externally validated. An important aspect in the design of a validation study is choosing an adequate sample size. In this paper, we investigate the sample size requirements for validation studies with binary outcomes to estimate measures of predictive performance (C-statistic for discrimination and calibration slope and calibration in the large). We aim for sufficient precision in the estimated measures. In addition, we investigate the sample size to achieve sufficient power to detect a difference from a target value. Under normality assumptions on the distribution of the linear predictor, we obtain simple estimators for sample size calculations based on the measures above. Simulation studies show that the estimators perform well for common values of the C-statistic and outcome prevalence when the linear predictor is marginally Normal. Their performance deteriorates only slightly when the normality assumptions are violated. We also propose estimators which do not require normality assumptions but require specification of the marginal distribution of the linear predictor and require the use of numerical integration. These estimators were also seen to perform very well under marginal normality. Our sample size equations require a specified standard error (SE) and the anticipated C-statistic and outcome prevalence. The sample size requirement varies according to the prognostic strength of the model, outcome prevalence, choice of the performance measure and study objective. For example, to achieve an SE < 0.025 for the C-statistic, 60-170 events are required if the true C-statistic and outcome prevalence are between 0.64-0.85 and 0.05-0.3, respectively. For the calibration slope and calibration in the large, achieving SE < 0.15 would require 40-280 and 50-100 events, respectively. Our estimators may also be used for survival outcomes when the proportion of censored observations is high.

Prediction of five-year mortality after COPD diagnosis using primary care records.

Accurate prognosis information after a diagnosis of chronic obstructive pulmonary disease (COPD) would facilitate earlier and better informed decisions about the use of prevention strategies and advanced care plans. We therefore aimed to develop and validate an accurate prognosis model for incident COPD cases using only information present in general practitioner (GP) records at the point of diagnosis. Incident COPD patients between 2004-2012 over the age of 35 were studied using records from 396 general practices in England. We developed a model to predict all-cause five-year mortality at the point of COPD diagnosis, using 47,964 English patients. Our model uses age, gender, smoking status, body mass index, forced expiratory volume in 1-second (FEV1) % predicted and 16 co-morbidities (the same number as the Charlson Co-morbidity Index). The performance of our chosen model was validated in all countries of the UK (N = 48,304). Our model performed well, and performed consistently in validation data. The validation area under the curves in each country varied between 0.783-0.809 and the calibration slopes between 0.911-1.04. Our model performed better in this context than models based on the Charlson Co-morbidity Index or Cambridge Multimorbidity Score. We have developed and validated a model that outperforms general multimorbidity scores at predicting five-year mortality after COPD diagnosis. Our model includes only data routinely collected before COPD diagnosis, allowing it to be readily translated into clinical practice, and has been made available through an online risk calculator (https://skiddle.shinyapps.io/incidentcopdsurvival/).

A general method for elicitation, imputation, and sensitivity analysis for incomplete repeated binary data.

We develop and demonstrate methods to perform sensitivity analyses to assess sensitivity to plausible departures from missing at random in incomplete repeated binary outcome data. We use multiple imputation in the not at random fully conditional specification framework, which includes one or more sensitivity parameters (SPs) for each incomplete variable. The use of an online elicitation questionnaire is demonstrated to obtain expert opinion on the SPs, and highest prior density regions are used alongside opinion pooling methods to display credible regions for SPs. We demonstrate that substantive conclusions can be far more sensitive to departures from the missing at random assumption (MAR) when control and intervention nonresponders depart from MAR differently, and show that the correlation of arm specific SPs in expert opinion is particularly important. We illustrate these methods on the iQuit in Practice smoking cessation trial, which compared the impact of a tailored text messaging system versus standard care on smoking cessation. We show that conclusions about the effect of intervention on smoking cessation outcomes at 8 week and 6 months are broadly insensitive to departures from MAR, with conclusions significantly affected only when the differences in behavior between the nonresponders in the two trial arms is larger than expert opinion judges to be realistic.